Search

In this brief, we present a high accuracy training method for inverter-based memristive neural networks (IM-NNs). The method, which relies on accurate modeling of the circuit element characteristics, is called LATIM (Loading-Aware offline Training method for Inverter-based Memristive NNs). In LATIM, an approximation method is proposed to estimate the effective load of the memristive crossbar (as the synapses) while two NNs are utilized to predict the voltage transfer characteristic (VTC) of the inverters (as the activation functions). Efficacy of the proposed method is compared with the recent offline training methods for IM-NNs, called PHAX and RIM. Simulation results reveal that LATIM can predict the output voltage of the IM-NNs, on average, by $14\times $ ($6\times $) and $29\times $ ($4\times $) smaller error for the MNIST and Fashion MNIST datasets, respectively, compared to those of PHAX (RIM) method. In addition, IM-NNs trained by LATIM consume, on average, 62% and 53% lower energy compared to PHAX and RIM methods due to proper sizing of the inverters. [ABSTRACT FROM AUTHOR]

Inverter-based memristive neural networks have the advantages of much lower power consumptions and higher speeds compared to those of digital implementations while they suffer from accuracy degradation due to the circuit elements variations. In this brief, we propose a circuit-level method, called RIM (Resistor-Inverter based Memristive neural network) to mitigate the accuracy degradation of this type of neural network implementation at the presence of variations as well as improve the efficacy of its offline training. In the proposed method, the high slope of the inverter voltage transfer characteristic (VTC) is reduced by adding a grounded resistor to the inverter output node. While decreasing the resistance, generally, suppresses the adverse effects of variations on the network accuracy as well as the loading effect of the memristive crossbar, it may lead to output accuracy decrease under the nominal (no variation) condition. Thus, some trade-off is considered for choosing a proper value for the added resistance. The study reveals that RIM can improve the output accuracy in the range of 10%–96% (51%–74%) in the nominal condition (variations condition). [ABSTRACT FROM AUTHOR]

In this article, the accuracy of inverter-based memristive neural networks (NNs) for function approximation applications is improved under the presence of process variations. The improvement is achieved by using a design approach, called INTERSTICE (Inverter-based Memristive Neural Networks Dis cretization for Function Approximation Applications), which discretizes the output values by employing a classifier. More precisely, in the INTERSTICE approach, the output range is divided into $K$ subranges where each subrange is considered as a class. To train the classifier, the training samples are labeled where each label shows belonging to a specific class. To evaluate the efficacy of the design technique, some function approximation applications such as BlackScholes, FFT, $K$ -means, and Sobel are considered. Compared to PHAX, a recently published inverter-based memristive NN, INTERSTICE provides lower mean squared error (MSE) values in the presence of memristor and transistor variations. More specifically, the improvements in the mean of MSE ($\mu _{\mathrm {MSE}}$) are in the range of 40%–80% when considering 10% variations in the memristor resistance and transistor parameters. In addition, for most of the benchmarks, INTERSTICE improves the $\mu _{\mathrm {MSE}}$ values of the nominal case (the case where all circuit elements are ideal) compared to PHAX. As another advantage compared to the PHAX, in INTERSTICE, digital outputs can be generated based on the selected classes which eliminates the need for an analog-to-digital converter at the output port connected to the digital part of the system. Finally, achieving lower $\mu _{\mathrm {MSE}}$ values using fewer memristors and consuming lower energy is also attainable with this design approach. [ABSTRACT FROM AUTHOR]

In this paper, we propose a training framework for an inverter-based memristive neuromorphic hardware. The framework, which is called PHAX, is a physical characteristics aware one relying on an ex-situ training approach. The considered neuromorphic circuit is highly energy efficient hybrid CMOS-memristive implementation of neuromorphic circuits. To solve the problem of high sensitivity of the training to the mismatches between the high-level mathematical modeling of the neurons and the corresponding physical characteristics, an approach for analytical yet accurate modeling of the memristive crossbar and neuron circuits is suggested. The approach, which is based on SPICE simulations, models the inverter-based neurons using a hyperbolic tangent function. To increase the training efficacy, the backpropagation training algorithm is modified by considering some constraints based on the physical characteristics of the memristive circuit. This modification along with the accurate back-annotation of the physical characteristics considerably improve the effectiveness of the ex-situ training method of the neuromorphic circuit. The results of this paper show an average reduction of $1805{\times }$ in the training runtime compared to that of the in-situ training approach. Furthermore, the results of applying the approach on the kernels of some applications such as image recognition, image processing, and financial analysis reveal that the designed neuromorphic circuits provide an average power saving (speed up) of $1478{\times }$ ( $5.2{\times }$ ) over the ASIC implementation in a 90-nm CMOS technology. [ABSTRACT FROM AUTHOR]

This paper presents a federated learning approach based on utilizing computational resources of the IoT edge devices for training deep neural networks. In this approach, the edge devices and the cloud server collaborate in the training phase while preserving the privacy of the edge device data. Owing to the limited computational power and resources available to the edge devices, instead of the original neural network (NN), we suggest to use a smaller NN generated using a proposed heuristic method. In the proposed approach, the smaller model, which is trained on the edge device, is generated from the main NN model. By the exploiting Knowledge Distillation (K D) approach, the learned knowledge in the server and the edge devices can be exchanged, leading to lower required computation on the server and preserving data privacy of the edge devices. Also, to reduce the knowledge transfer overhead on the communication links between the server and the edge devices, a method for selecting the most valuable data to transfer the knowledge is introduced. The effectiveness of this method is assessed by comparing it to state-of-the-art methods. The results show that the proposed method lowers the communication traffic by up to 250 × and increases the learning accuracy by an average of 8.9 % in the cloud compared to the prior K D -based distributed training approaches in CIFAR-10 dataset. [ABSTRACT FROM AUTHOR]