Search

Training deep neural networks (DNNs) is computationally intensive but arrays of non-volatile memories like Charge Trap Flash (CTF) can accelerate DNN operations using in-memory computing. Specifically, the Resistive Processing Unit (RPU) architecture uses the voltage-threshold program by stochastic encoded pulse trains and analog memory features to accelerate vector-vector outer product and weight update for the gradient descent algorithms. Although CTF, offering high precision, has been regarded as an excellent choice for implementing RPU, the accumulation of charge due to the applied stochastic pulse trains is ultimately of critical significance in determining the final weight update. In this paper, we report the non-ideal program-time conservation in CTF through pulsing input measurements. We experimentally measure the effect of pulse width and pulse gap, keeping the total ON-time of the input pulse train constant, and report three non-idealities: (1) Cumulative V_T shift reduces when total ON-time is fragmented into a larger number of shorter pulses, (2) Cumulative V_T shift drops abruptly for pulse widths < 2 {\mu}s, (3) Cumulative V_T shift depends on the gap between consecutive pulses and the V_T shift reduction gets recovered for smaller gaps. We present an explanation based on a transient tunneling field enhancement due to blocking oxide trap-charge dynamics to explain these non-idealities. Identifying and modeling the responsible mechanisms and predicting their system-level effects during learning is critical. This non-ideal accumulation is expected to affect algorithms and architectures relying on devices for implementing mathematically equivalent functions for in-memory computing-based acceleration.

Energy-efficient real-time synapses and neurons are essential to enable large-scale neuromorphic computing. In this paper, we propose and demonstrate the Schottky-Barrier MOSFET-based ultra-low power voltage-controlled current source to enable real-time neurons for neuromorphic computing. Schottky-Barrier MOSFET is fabricated on a Silicon-on-insulator platform with polycrystalline Silicon as the channel and Nickel/Platinum as the source/drain. The Poly-Si and Nickel make the back-to-back Schottky junction enabling ultra-low ON current required for energy-efficient neurons.

HfO2-based Ferroelectric field-effect transistor (FeFET) has become a center of attraction for non-volatile memory applications because of their low power, fast switching speed, high scalability, and CMOS compatibility. In this work, we show an n-channel FeFET-based Multibit memory, termed MirrorBit, which effectively doubles the chip density via programming the gradient ferroelectric polarizations in the gate using an appropriate biasing scheme. We have experimentally demonstrated MirrorBit on GlobalFoundries HfO2-based FeFET devices fabricated at 28 nm bulk HKMG CMOS technology. Retention of MirrorBit states has been shown up to $10^5$ s at different temperatures. Also, the endurance is found to be more than $10^3$ cycles. A TCAD simulation is also presented to explain the origin and working of MirrorBit states based on the FeFET model calibrated using the GlobalFoundries FeFET device. We have also proposed the array-level implementation and sensing methodology of the MirrorBit memory. Thus, we have converted 1-bit FeFET into 2-bit FeFET using a particular programming scheme in existing FeFET, without needing any notable fabrication process alteration, to double the chip density for high-density non-volatile memory storage., Comment: 6 pages, 9 figures

Spiking Neural Networks (SNNs) have emerged as a hardware efficient architecture for classification tasks. The challenge of spike-based encoding has been the lack of a universal training mechanism performed entirely using spikes. There have been several attempts to adopt the powerful backpropagation (BP) technique used in non-spiking artificial neural networks (ANN): (1) SNNs can be trained by externally computed numerical gradients. (2) A major advancement towards native spike-based learning has been the use of approximate Backpropagation using spike-time dependent plasticity (STDP) with phased forward/backward passes. However, the transfer of information between such phases for gradient and weight update calculation necessitates external memory and computational access. This is a challenge for standard neuromorphic hardware implementations. In this paper, we propose a stochastic SNN based Back-Prop (SSNN-BP) algorithm that utilizes a composite neuron to simultaneously compute the forward pass activations and backward pass gradients explicitly with spikes. Although signed gradient values are a challenge for spike-based representation, we tackle this by splitting the gradient signal into positive and negative streams. We show that our method approaches BP ANN baseline with sufficiently long spike-trains. Finally, we show that the well-performing softmax cross-entropy loss function can be implemented through inhibitory lateral connections enforcing a Winner Take All (WTA) rule. Our SNN with a 2-layer network shows excellent generalization through comparable performance to ANNs with equivalent architecture and regularization parameters on static image datasets like MNIST, Fashion-MNIST, Extended MNIST, and temporally encoded image datasets like Neuromorphic MNIST datasets. Thus, SSNN-BP enables BP compatible with purely spike-based neuromorphic hardware.

The epitaxial growth of functional materials using a substrate with a graphene layer is a highly desirable method for improving structural quality and obtaining free-standing epitaxial nano-membranes for scientific study, applications, and economical reuse of substrates. However, the aggressive oxidizing conditions typically employed to grow epitaxial perovskite oxides can damage graphene. Here, we demonstrate a technique based on hybrid molecular beam epitaxy that does not require an independent oxygen source to achieve epitaxial growth of complex oxides without damaging the underlying graphene. The technique produces films with self-regulating cation stoichiometry control and epitaxial orientation to the oxide substrate. Furthermore, the films can be exfoliated and transferred to foreign substrates while leaving the graphene on the original substrate. These results open the door to future studies of previously unattainable free-standing nano-membranes grown in an adsorption-controlled manner by hybrid molecular beam epitaxy, and has potentially important implications for the commercial application of perovskite oxides in flexible electronics., Comment: 20 pages, 6 figures

Emerging non-volatile memories have been proposed for a wide range of applications from easing the von-Neumann bottleneck to neuromorphic applications. Specifically, scalable RRAMs based on Pr$_{1-x}$Ca$_x$MnO$_3$ (PCMO) exhibit analog switching have been demonstrated as an integrating neuron, an analog synapse, and a voltage-controlled oscillator. More recently, the inherent stochasticity of memristors has been proposed for efficient hardware implementations of Boltzmann Machines. However, as the problem size scales, the number of neurons increase and controlling the stochastic distribution tightly over many iterations is necessary. This requires parametric control over stochasticity. Here, we characterize the stochastic Set in PCMO RRAMs. We identify that the Set time distribution depends on the internal state of the device (i.e., resistance) in addition to external input (i.e., voltage pulse). This requires the confluence of contradictory properties like stochastic switching as well as deterministic state control in the same device. Unlike, "stochastic-everywhere" filamentary memristors, in PCMO RRAMs, we leverage the (i) stochastic Set in negative polarity and (ii) deterministic analog Reset in positive polarity to demonstrate 100x reduced Set time distribution drift. The impact on Boltzmann Machines' performance is analyzed and as opposed to the "fixed external input stochasticity", the "state-monitored stochasticity" can solve problems 20x larger in size. State monitoring also tunes out the device-to-device variability effect on distributions providing 10x better performance. In addition to the physical insights, this study establishes the use of experimental stochasticity in PCMO RRAMs in stochastic recurrent neural networks reliably over many iterations.

We demonstrate selective area epitaxy of GaAs films using patterned graphene masks on a Ge (001) substrate. The GaAs selectively grows on exposed regions of the Ge substrate, for graphene spacings as large as 10 microns. The selectivity is highly dependent on the growth temperature and annealing time, which we explain in terms of temperature dependent sticking coefficients and surface diffusion. The high nucleation selectivity over several microns sets constraints on experimental realizations of remote epitaxy.

Spiking Neural Networks (SNNs) are gaining widespread momentum in the field of neuromorphic computing. These network systems integrated with neurons and synapses provide computational efficiency by mimicking the human brain. It is desired to incorporate the biological neuronal dynamics, including complex spiking patterns which represent diverse brain activities within the neural networks. Earlier hardware realization of neurons was (1) area intensive because of large capacitors in the circuit design, (2) neuronal spiking patterns were demonstrated with clocked neurons at the device level. To achieve more realistic biological neuron spiking behavior, emerging memristive devices are considered promising alternatives. In this paper, we propose, PrMnO3(PMO) -RRAM device-based neuron. The voltage-controlled electrothermal timescales of the compact PMO RRAM device replace the electrical timescales of charging a large capacitor. The electrothermal timescale is used to implement an integration block with multiple voltage-controlled timescales coupled with a refractory block to generate biological neuronal dynamics. Here, first, a Verilog-A implementation of the thermal device model is demonstrated, which captures the current-temperature dynamics of the PMO device. Second, a driving circuitry is designed to mimic different spiking patterns of cortical neurons, including Intrinsic bursting (IB) and Chattering (CH). Third, a neuron circuit model is simulated, which includes the PMO RRAM device model and the driving circuitry to demonstrate the asynchronous neuron behavior. Finally, a hardware-software hybrid analysis is done in which the PMO RRAM device is experimentally characterized to mimic neuron spiking dynamics. The work presents a realizable and more biologically comparable hardware-efficient solution for large-scale SNNs.

In this work, we aim to devise an end-to-end spiking implementation for contour tracking in 3D media inspired by chemotaxis, where the worm reaches the region which has the given set concentration. For a planer medium, efficient contour tracking algorithms have already been devised, but a new degree of freedom has quite a few challenges. Here we devise an algorithm based on klinokinesis - where the motion of the worm is in response to the stimuli but not proportional to it. Thus the path followed is not the shortest, but we can track the set concentration successfully. We are using simple LIF neurons for the neural network implementation, considering the feasibility of its implementation in the neuromorphic computing hardware., Comment: 12 pages, 8 figures, accepted for the '30th International Conference on Artificial Neural Networks, ICANN2021'

We quantify the mechanisms for manganese (Mn) diffusion through graphene in Mn/graphene/Ge (001) and Mn/graphene/GaAs (001) heterostructures for samples prepared by graphene layer transfer versus graphene growth directly on the semiconductor substrate. These heterostructures are important for applications in spintronics; however, challenges in synthesizing graphene directly on technologically important substrates such as GaAs necessitate layer transfer and anneal steps, which introduce defects into the graphene. \textit{In-situ} photoemission spectroscopy measurements reveal that Mn diffusion through graphene grown directly on a Ge (001) substrate is 1000 times lower than Mn diffusion into samples without graphene ($D_{gr,direct} \sim 4\times10^{-18}$cm$^2$/s, $D_{no-gr} \sim 5 \times 10^{-15}$ cm$^2$/s at 500$^\circ$C). Transferred graphene on Ge suppresses the Mn in Ge diffusion by a factor of 10 compared to no graphene ($D_{gr,transfer} \sim 4\times10^{-16}cm^2/s$). For both transferred and directly-grown graphene, the low activation energy ($E_a \sim 0.1-0.5$ eV) suggests that Mn diffusion through graphene occurs primarily at graphene defects. This is further confirmed as the diffusivity prefactor, $D_0$, scales with the defect density of the graphene sheet. Similar diffusion barrier performance is found on GaAs substrates; however, it is not currently possible to grow graphene directly on GaAs. Our results highlight the importance of developing graphene growth directly on functional substrates, to avoid the damage induced by layer transfer and annealing.

Remote epitaxy is a promising approach for synthesizing exfoliatable crystalline membranes and enabling epitaxy of materials with large lattice mismatch. However, the atomic scale mechanisms for remote epitaxy remain unclear. Here we experimentally demonstrate that GaSb films grow on graphene-terminated GaSb (001) via a seeded lateral epitaxy mechanism, in which pinhole defects in the graphene serve as selective nucleation sites, followed by lateral epitaxy and coalescence into a continuous film. Remote interactions are not necessary in order to explain the growth. Importantly, the small size of the pinholes permits exfoliation of continuous, free-standing GaSb membranes. Due to the chemical similarity between GaSb and other III-V materials, we anticipate this mechanism to apply more generally to other materials. By combining molecular beam epitaxy with \textit{in-situ} electron diffraction and photoemission, plus \textit{ex-situ} atomic force microscopy and Raman spectroscopy, we track the graphene defect generation and GaSb growth evolution a few monolayers at a time. Our results show that the controlled introduction of nanoscale openings in graphene provides a powerful route towards tuning the growth and properties of epitaxial films and membranes on 2D materials.

C. elegans shows chemotaxis using klinokinesis where the worm senses the concentration based on a single concentration sensor to compute the concentration gradient to perform foraging through gradient ascent/descent towards the target concentration followed by contour tracking. The biomimetic implementation requires complex neurons with multiple ion channel dynamics as well as interneurons for control. While this is a key capability of autonomous robots, its implementation on energy-efficient neuromorphic hardware like Intel's Loihi requires adaptation of the network to hardware-specific constraints, which has not been achieved. In this paper, we demonstrate the adaptation of chemotaxis based on klinokinesis to Loihi by implementing necessary neuronal dynamics with only LIF neurons as well as a complete spike-based implementation of all functions e.g. Heaviside function and subtractions. Our results show that Loihi implementation is equivalent to the software counterpart on Python in terms of performance - both during foraging and contour tracking. The Loihi results are also resilient in noisy environments. Thus, we demonstrate a successful adaptation of chemotaxis on Loihi - which can now be combined with the rich array of SNN blocks for SNN based complex robotic control.

Liquid State Machines are brain inspired spiking neural networks (SNNs) with random reservoir connectivity and bio-mimetic neuronal and synaptic models. Reservoir computing networks are proposed as an alternative to deep neural networks to solve temporal classification problems. Previous studies suggest 2nd order (double exponential) synaptic waveform to be crucial for achieving high accuracy for TI-46 spoken digits recognition. The proposal of long-time range (ms) bio-mimetic synaptic waveforms is a challenge to compact and power efficient neuromorphic hardware. In this work, we analyze the role of synaptic orders namely: {\delta} (high output for single time step), 0th (rectangular with a finite pulse width), 1st (exponential fall) and 2nd order (exponential rise and fall) and synaptic timescales on the reservoir output response and on the TI-46 spoken digits classification accuracy under a more comprehensive parameter sweep. We find the optimal operating point to be correlated to an optimal range of spiking activity in the reservoir. Further, the proposed 0th order synapses perform at par with the biologically plausible 2nd order synapses. This is substantial relaxation for circuit designers as synapses are the most abundant components in an in-memory implementation for SNNs. The circuit benefits for both analog and mixed-signal realizations of 0th order synapse are highlighted demonstrating 2-3 orders of savings in area and power consumptions by eliminating Op-Amps and Digital to Analog Converter circuits. This has major implications on a complete neural network implementation with focus on peripheral limitations and algorithmic simplifications to overcome them.

In this paper we present a Spiking Neural Network (SNN) for autonomous navigation, inspired by the chemotaxis network of the worm Caenorhabditis elegans. In particular, we focus on the problem of contour tracking, wherein the bot must reach and subsequently follow a desired concentration setpoint. Past schemes that used only klinokinesis can follow the contour efficiently but take excessive time to reach the setpoint. We address this shortcoming by proposing a novel adaptive klinotaxis mechanism that builds upon a previously proposed gradient climbing circuit. We demonstrate how our klinotaxis circuit can autonomously be configured to perform gradient ascent, gradient descent and subsequently be disabled to seamlessly integrate with the aforementioned klinokinesis circuit. We also incorporate speed regulation (orthokinesis) to further improve contour tracking performance. Thus for the first time, we present a model that successfully integrates klinokinesis, klinotaxis and orthokinesis. We demonstrate via contour tracking simulations that our proposed scheme achieves an 2.4x reduction in the time to reach the setpoint, along with a simultaneous 8.7x reduction in average deviation from the setpoint.

Single-crystalline membranes of functional materials enable the tuning of properties via extreme strain states; however, conventional routes for producing membranes require the use of sacrificial layers and chemical etchants, which can both damage the membrane and limit the ability to make them ultrathin. Here we demonstrate the epitaxial growth of the cubic Heusler compound GdPtSb on graphene-terminated Al$_2$O$_3$ substrates. Despite the presence of the graphene interlayer, the Heusler films have epitaxial registry to the underlying sapphire, as revealed by x-ray diffraction, reflection high energy electron diffraction, and transmission electron microscopy. The weak Van der Waals interactions of graphene enable mechanical exfoliation to yield free-standing GdPtSb membranes, which form ripples when transferred to a flexible polymer handle. Whereas unstrained GdPtSb is antiferromagnetic, measurements on rippled membranes show a spontaneous magnetic moment at room temperature, with a saturation magnetization of 5.2 bohr magneton per Gd. First-principles calculations show that the coupling to homogeneous strain is too small to induce ferromagnetism, suggesting a dominant role for strain gradients. Our membranes provide a novel platform for tuning the magnetic properties of intermetallic compounds via strain (piezomagnetixm and magnetostriction) and strain gradients (flexomagnetism).

Graphene plasmonic resonators have been broadly studied in the terahertz and mid-infrared ranges because of their electrical tunability and large confinement factors which can enable dramatic enhancement of light-matter coupling. In this work, we demonstrate that the characteristic scaling laws of graphene plasmons change for smaller (< 40 nm) plasmonic wavelengths, expanding the operational frequencies of graphene plasmonic resonators into the near-infrared (NIR) and modifying their optical confinement properties. We utilize a novel bottom-up block copolymer lithography method that substantially improves upon top-down methods to create resonators as narrow as 12 nm over centimeter-scale areas. Measurements of these structures reveal that their plasmonic resonances are strongly influenced by non-local and quantum effects, which push their resonant frequency into the NIR (2.2 um), almost double the frequency of previous experimental works. The confinement factors of these resonators, meanwhile, reach 137 +/- 25, amongst the largest reported in literature for an optical cavity. While our findings indicate that the enhancement of some 'forbidden' transitions are an order of magnitude weaker than predicted, the combined NIR response and large confinement of these structures make them an attractive platform to explore ultra-strongly enhanced spontaneous emission., Comment: 4 Figures. Supplement included

Pr$_{0.7}$Ca$_{0.3}$MnO$_3$ (PCMO) based RRAM shows promising memory properties like non-volatility, low variability, multiple resistance states and scalability. From a modeling perspective, the charge carrier DC current modeling of PCMO RRAM by drift diffusion (DD) in the presence of fixed oxygen ion vacancy traps and self-heating (SH) in Technology Computer Aided Design (TCAD) (but without oxygen ionic transport) was able to explain the experimentally observed space charge limited conduction (SCLC) characteristics, prior to resistive switching. Further, transient analysis using DD+SH model was able to reproduce the experimentally observed fast current increase at ~100 ns timescale, prior to resistive switching. However, a complete quantitative transient current transport plus resistive switching model requires the inclusion of ionic transport. We propose a Reaction-Drift (RD) model for oxygen ion vacancy related trap density variation, which is combined with the DD+SH model. Earlier we have shown that the Set transient consists of 3 stages and Reset transient consists of 4 stages experimentally. In this work, the DD+SH+RD model is able to reproduce the entire transient behavior over 10 ns - 1 s range in timescale for both the Set and Reset operations for different applied biases and ambient temperatures. Remarkably, a universal Reset experimental behavior, log(I) is proportional to (m X log(t)) where m~-1/10 is reproduced in simulations. This model is the first model for PCMO RRAMs to significantly reproduce transient Set/Reset behavior. This model establishes the presence of self-heating and ionic-drift limited resistive switching as primary physical phenomena in these RRAMs.

Liquid State Machine (LSM) is a brain-inspired architecture used for solving problems like speech recognition and time series prediction. LSM comprises of a randomly connected recurrent network of spiking neurons. This network propagates the non-linear neuronal and synaptic dynamics. Maass et al. have argued that the non-linear dynamics of LSMs is essential for its performance as a universal computer. Lyapunov exponent (mu), used to characterize the "non-linearity" of the network, correlates well with LSM performance. We propose a complementary approach of approximating the LSM dynamics with a linear state space representation. The spike rates from this model are well correlated to the spike rates from LSM. Such equivalence allows the extraction of a "memory" metric (tau_M) from the state transition matrix. tau_M displays high correlation with performance. Further, high tau_M system require lesser epochs to achieve a given accuracy. Being computationally cheap (1800x time efficient compared to LSM), the tau_M metric enables exploration of the vast parameter design space. We observe that the performance correlation of the tau_M surpasses the Lyapunov exponent (mu), (2-4x improvement) in the high-performance regime over multiple datasets. In fact, while mu increases monotonically with network activity, the performance reaches a maxima at a specific activity described in literature as the "edge of chaos". On the other hand, tau_M remains correlated with LSM performance even as mu increases monotonically. Hence, tau_M captures the useful memory of network activity that enables LSM performance. It also enables rapid design space exploration and fine-tuning of LSM parameters for high performance., Comment: Submitted to IJCNN 2019