Search

RRAM-based computing-in-memory (CIM) chips have shown great potentials to accelerate deep neural networks on edge devices by reducing data transfer between the memory and the computing unit. However, due to the non-ideal characteristics of RRAM, the accuracy of the neural network on the RRAM chip is usually lower than the software. Here we propose an on-chip layer-wise training (LWT) method to alleviate the adverse effect of RRAM imperfections and improve the accuracy of the chip. Using a locally validated dataset, LWT can reduce the communication between the edge and the cloud, which benefits personalized data privacy. The simulation results on the CIFAR-10 dataset show that the LWT method can improve the accuracy of VGG-16 and ResNet-18 by more than 5% and 10%, respectively, with only 25% operations and 35% buffer compared with the back-propagation method. Moreover, the pipe-LWT method is presented to improve the throughput by three times further.

For the first time, this paper develops a novel stochastic computing method by utilizing the inherent random noises of analog RRAM. With the designed analog switching characteristics, the RRAM device can realize the function of sampling from a tunable probabilistic distribution. A Bayesian neural network (BayNN), whose weights are represented by probability distributions, is experimentally demonstrated on the fabricated 160K RRAM array. The measured result achieves 97% accuracy for image classification on MNIST dataset. Moreover, the RRAM based BayNN shows anti-attack capability with inherent device stochastic behavior to detect "adversarial" images, which are generated by adding noises to the original MNIST images and can fool the conventional deep neural networks. This is the first demonstration work for the widely-used BayNN algorithms with emerging devices.

A perspective is provided to compare the reliability requirements of Resistive Random Access Memory (RRAM) for digital memory application and neuromorphic computing application. Retention and endurance are considered as the two key metrics to evaluate the impact on neural network training. Statistical measurements and network simulation demonstrate that the retention and endurance degradation behaviors of analog RRAM are quite unique when implementing on a neuromorphic computing system. New evaluation method is therefore needed for reliability test.

Analog RRAM is attractive for realizing high-density storage and in-memory computing. The endurance requirements of analog type resistive switching are different from binary type switching. In this work, endurance degradation behavior of analog RRAM is studied. It is found that the endurance lifetime would increase as the window decreasing. Higher resistance state has more significant influence on the endurance degradation. Physical mechanism for endurance is elucidated to explain the reason for the impact of switching window on the endurance lifetime.

Resistive random access memory (RRAM) is attractive for neuromorphic computing systems as synaptic weights. In the neural network training, incremental switching occurs between the analog conductance states, thus the analog RRAM devices have unique endurance degradation behaviors compared to the convention digital memory application. In this work, a fast measurement platform is developed to characterize the endurance of incremental switching in analog RRAM. It is found that under weak weight update pulses, the incremental switching cycles of RRAM can be increased for more than 5 orders of magnitude compared with full window switching under strong programming pulses. The 1011-cycle endurance of analog RRAM is proved to be sufficient for training neural networks online for various datasets (from MNIST to ImageNet). However, the nonlinearity and dynamic range of analog RRAM degrade during cycling, which may influence the learning accuracy of the neural network when it re-trains with new datasets.

For the first time, Generative Adversarial Network (GAN) is experimentally demonstrated on 1kb analog RRAM array. After online training, the network can generate different patterns of digital numbers. The intrinsic random noises of analog RRAM device are utilized as the input of the neural network to improve the diversity of the generated numbers. The impacts of read and write noises on the performance of GAN are analyzed. Optimized methodology is developed to mitigate the excessive noise effect on RRAM based GAN. This work proves that RRAM is suitable for the application of GAN. It also paves a new way to take advantage of the non-ideal effects of RRAM devices.

HMAX is a neuroscience-inspired deep learning model, which consists of alternating S layers and C layers, mimicking the functional mechanism of the ventral visual pathway of primates. Recently, sparse coding is introduced to the framework of HMAX for learning the S layer bases, leading to semantic features of natural images. However, when processing large scale datasets, the sparse HMAX is beset with large time and memory consumption during both training and testing. In this paper, we present a distributed algorithm to parallelize the sparse coding in S layers. Experiments on a cluster with eight nodes verify that the algorithm can also learn higher level semantic representations of objects but with much higher efficiency than the original model. The acceleration rate is linearly proportional to the number of nodes, which shows the good scalability of the proposed method.