Search

Artificial intelligence and machine learning are growing computing paradigms, but current algorithms incur undesirable energy costs on conventional hardware platforms, thus motivating the exploration of more efficient neuromorphic architectures. Toward this end, we introduce here a memtransistor with gate-tunable dynamic learning behavior. By fabricating memtransistors from monolayer MoS2 grown on sapphire, the relative importance of the vertical field effect from the gate is enhanced, thereby heightening reconfigurability of the device response. Inspired by biological systems, gate pulses are used to modulate potentiation and depression, resulting in diverse learning curves and simplified spike-timing-dependent plasticity that facilitate unsupervised learning in simulated spiking neural networks. This capability also enables continuous learning, which is a previously underexplored cognitive concept in neuromorphic computing. Overall, this work demonstrates that the reconfigurability of memtransistors provides unique hardware accelerator opportunities for energy efficient artificial intelligence and machine learning.

The increasing complexity of deep learning systems has pushed conventional computing technologies to their limits. While the memristor is one of the prevailing technologies for deep learning acceleration, it is only suited for classical learning layers where only two operands, namely weights and inputs, are processed simultaneously. Meanwhile, to improve the computational efficiency of deep learning for emerging applications, a variety of non-traditional layers requiring concurrent processing of many operands are becoming popular. For example, hypernetworks improve their predictive robustness by simultaneously processing weights and inputs against the application context. Two-electrode memristor grids cannot directly map emerging layers’ higher-order multiplicative neural interactions. Addressing this unmet need, we present crossbar processing using dual-gated memtransistors based on two-dimensional semiconductor MoS2. Unlike the memristor, the resistance states of memtransistors can be persistently programmed and can be actively controlled by multiple gate electrodes. Thus, the discussed memtransistor crossbar enables several advanced inference architectures beyond a conventional passive crossbar. For example, we show that sneak paths can be effectively suppressed in memtransistor crossbars, whereas they limit size scalability in a passive memristor crossbar. Similarly, exploiting gate terminals to suppress crossbar weights dynamically reduces biasing power by ∼20% in memtransistor crossbars for a fully connected layer of AlexNet. On emerging layers such as hypernetworks, collocating multiple operations within the same crossbar cells reduces operating power by ∼15× on the considered network cases.

Artificial intelligence and machine learning are growing computing paradigms, but current algorithms incur undesirable energy costs on conventional hardware platforms, thus motivating the exploration of more efficient neuromorphic architectures. Toward this end, we introduce here a memtransistor with gate-tunable dynamic learning behavior. By fabricating memtransistors from monolayer MoS

This letter discusses mixed-mode magneto tunneling junction (m-MTJ)-based restricted Boltzmann machine (RBM). RBMs are unsupervised learning models, suitable for extracting features from high-dimensional data. The m-MTJ is actuated by the simultaneous actions of voltage-controlled magnetic anisotropy and voltage-controlled spin-transfer torque, where the switching of the free-layer is probabilistic and can be controlled by the two. Using m-MTJ-based activation functions, we present a novel low area/power RBM. We discuss online learning of the presented implementation to negate process variability. For MNIST hand-written dataset, the design achieves ~96% accuracy under expected variability in various components.

This paper discusses the potential of model-hardware co-design to simplify the implementation complexity of compute-in-SRAM deep learning considerably. Although compute-in-SRAM has emerged as a promising approach to improve the energy efficiency of DNN processing, current implementations suffer due to complex and excessive mixed-signal peripherals, such as the need for parallel digital-to-analog converters (DACs) at each input port. Comparatively, our approach inherently obviates complex peripherals by co-designing learning operators to SRAM's operational constraints. For example, our co-designed implementation is DAC-free even for multibit precision DNN processing. Additionally, we also discuss the interaction of our compute-in-SRAM operator with Bayesian inference of DNNs. We show a synergistic interaction of Bayesian inference with our framework, where Bayesian methods allow achieving similar accuracy with much smaller network size. Although each iteration of sample-based Bayesian inference is computationally expensive, the cost is minimized by our compute-in-SRAM approach. Meanwhile, by reducing the network size, Bayesian methods reduce the footprint cost of compute-in-SRAM implementation, which is a crucial concern for the method. We characterize this interaction for deep learning-based pose (position and orientation) estimation for a drone.

We propose a co-design approach for compute-in-memory inference for deep neural networks (DNN). We use multiplication-free function approximators based on $\ell _{1}$ norm along with a co-adapted processing array and compute flow. Using the approach, we overcame many deficiencies in the current art of in-SRAM DNN processing such as the need for digital-to-analog converters (DACs) at each operating SRAM row/column, the need for high precision analog-to-digital converters (ADCs), limited support for multi-bit precision weights, and limited vector-scale parallelism. Our co-adapted implementation seamlessly extends to multi-bit precision weights, it doesn’t require DACs, and it easily extends to higher vector-scale parallelism. We also propose an SRAM-immersed successive approximation ADC (SA-ADC), where we exploit the parasitic capacitance of bit lines of SRAM array as a capacitive DAC. Since the dominant area overhead in SA-ADC comes due to its capacitive DAC, by exploiting the intrinsic parasitic of SRAM array, our approach allows low area implementation of within-SRAM SA-ADC. Our $8\times 62$ SRAM macro, which requires a 5-bit ADC, achieves ~105 tera operations per second per Watt (TOPS/W) with 8-bit input/weight processing at 45 nm CMOS. Our $8\times 30$ SRAM macro, which requires a 4-bit ADC, achieves ~84 TOPS/W. SRAM macros that require lower ADC precision are more tolerant of process variability, however, have lower TOPS/W as well. We evaluated the accuracy and performance of our proposed network for MNIST, CIFAR10, and CIFAR100 datasets. We chose a network configuration which adaptively mixes multiplication-free and regular operators. The network configurations utilize the multiplication-free operator for more than 85% operations from the total. The selected configurations are 98.6% accurate for MNIST, 90.2% for CIFAR10, and 66.9% for CIFAR100. Since most of the operations in the considered configurations are based on proposed SRAM macros, our compute-in-memory’s efficiency benefits broadly translate to the system-level.

In this work, we introduce bitcell array-based support parameters to improve the prediction accuracy of SRAM-based binarized neural network (SRAM-BNN). Our approach enhances the training weight space of SRAM-BNN while requiring minimal overheads to a typical design. More flexibility of the weight space leads to higher prediction accuracy in our design. We adapt row digital-to-analog (DAC) converter, and computing flow in SRAM-BNN for bitcell array-based weight supports. Using the discussed interventions, our scheme also allows a dynamic trade-off of accuracy against energy to address dynamic energy constraints in typical real-time applications. Our approach reduces classification error in MNIST from 1.4% to 0.91%. To reduce the power overheads, we propose a dynamic drop out of support parameters, which also reduces the processing energy of the in-SRAM weight-input product Our architecture can dropout 52% of the bitcell array-based support parameters with only minimal accuracy degradation. We also characterize our design under varying degrees of process variability in the transistors.

The recent trend in adapting ultra-energy-efficient (but error-prone) nanomagnetic devices to non-Boolean computing and information processing (e.g. stochastic/probabilistic computing, neuromorphic, belief networks, etc) has resulted in rapid strides in new computing modalities. Of particular interest are Bayesian networks (BN) which may see revolutionary advances when adapted to a specific type of nanomagnetic devices. Here, we develop a novel nanomagnet-based computing substrate for BN that allows high-speed sampling from an arbitrary Bayesian graph. We show that magneto-tunneling junctions (MTJs) can be used for electrically programmable 'sub-nanosecond' probability sample generation by co-optimizing voltage-controlled magnetic anisotropy and spin transfer torque. We also discuss that just by engineering local magnetostriction in the soft layers of MTJs, one can stochastically couple them for programmable conditional sample generation as well. This obviates the need for extensive energy-inefficient hardware like OP-AMPS, gates, shift-registers, etc to generate the correlations. Based on the above findings, we present an architectural design and computation flow of the MTJ network to map an arbitrary Bayesian graph where we develop circuits to program and induce switching and interactions among MTJs. Our discussed framework can lead to a new generation of stochastic computing hardware for various other computing models, such as stochastic programming and Bayesian deep learning. This can spawn a novel genre of ultra-energy-efficient, extremely powerful computing paradigms, which is a transformational advance.

We present a mixed-mode magneto tunneling junction (m-MTJ)-based Deep Belief Network (DBN). DBNs are unsupervised learning models, suitable for recognition and clustering. m-MTJ is a three-terminal magnetic device with probabilistic free layer switching controlled by the simultaneous actions of voltage-controlled magnetic anisotropy and spin-transfer torque. While DBNs achieve high prediction accuracy even with highly imprecise single-bit weights, the key complexity lies in their activation functions which are stochastic. Using an m-MTJ, we present a novel low area/power DBN neuron with stochastic activation function. We discuss an in-memory computing architecture that allows forward and backward flow of learning dynamics and online learning. Our design achieves ~88.80% accuracy for digit recognition in MNIST even under the worst case variability in nanoscaled m-MTJs.

This work discusses a CMOS-based implementation of low power speaker identification (SI) using Gaussian mixture models (GMMs). Conventionally, GMM-based SI relies on repeated access to log-add look-up table (LUT). With increasing dimensional of speaker models $\mathbf{and}/\mathbf{or}$ number of speakers in the database, accesses to the LUT dominate the overall energy expense for SI. In this work, we discuss piece-wise linear approximations to GMM model that eliminate LUT accesses, thereby limiting model parameter storage in the register files alone, while incurring a minimal accuracy drop. We evaluate our scheme on TIMIT corpus where for text-independent SI and with a two-second test speech, our scheme achieves more than $90\%$ accuracy across test-sets. We discuss the detailed architecture of the control unit, datapath, and key modules in our scheme. Compared to an equivalent design that requires LUT accesses in $\mathrm{off}$ -chip memories, our scheme limits power dissipation for SI to $\sim 600\ \mu \mathbf{W}$ and consumes $4\times$ less energy