论文标题

有效训练对突触设备的非理想性的熟悉深度信念的净值

Efficient Training of the Memristive Deep Belief Net Immune to Non-Idealities of the Synaptic Devices

论文作者

Wang, Wei, Hoffer, Barak, Greenberg-Toledo, Tzofnat, Li, Yang, Zou, Minhui, Herbelin, Eric, Ronen, Ronny, Xu, Xiaoxin, Zhao, Yulin, Yang, Jianguo, Kvatinsky, Shahar

论文摘要

各种新出现的非挥发性回忆设备的电导状态的可调节性模仿了生物突触的可塑性,这使其在大规模神经形态系统的硬件实现中有希望。通过在一个步骤中执行的横杆阵列中执行的矢量矩阵乘法(VMM)可以极大地加速神经网络的推断。然而,VMM的实施需要复杂的外围电路,并且复杂性进一步提高,因为不理想的设备的非理想性会阻止精确的电导性调整(尤其是在线培训),并且在很大程度上会降低深神经网络(DNNS)的性能。在这里,我们提出了一种有效的在线培训方法的深度信念网(DBN)。所提出的回忆DBN使用随机化的激活,降低了外围电路的复杂性,并使用基于对比度差异(CD)的梯度下降学习算法。模拟VMM和数字CD分别以混合信号硬件的布置进行,使得Memristive DBN高于突触设备的非理想性。在回忆设备上的写操作数量减少了两个数量级。对于MNIST数据集,可以使用各种回忆性突触设备的脉冲突触行为来实现95%〜97%的识别精度。

The tunability of conductance states of various emerging non-volatile memristive devices emulates the plasticity of biological synapses, making it promising in the hardware realization of large-scale neuromorphic systems. The inference of the neural network can be greatly accelerated by the vector-matrix multiplication (VMM) performed within a crossbar array of memristive devices in one step. Nevertheless, the implementation of the VMM needs complex peripheral circuits and the complexity further increases since non-idealities of memristive devices prevent precise conductance tuning (especially for the online training) and largely degrade the performance of the deep neural networks (DNNs). Here, we present an efficient online training method of the memristive deep belief net (DBN). The proposed memristive DBN uses stochastically binarized activations, reducing the complexity of peripheral circuits, and uses the contrastive divergence (CD) based gradient descent learning algorithm. The analog VMM and digital CD are performed separately in a mixed-signal hardware arrangement, making the memristive DBN high immune to non-idealities of synaptic devices. The number of write operations on memristive devices is reduced by two orders of magnitude. The recognition accuracy of 95%~97% can be achieved for the MNIST dataset using pulsed synaptic behaviors of various memristive synaptic devices.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源