Search

Neurophysiological observations confirm that the brain not only is able to detect the impaired synapses (in brain damage) but also it is relatively capable of repairing faulty synapses. It has been shown that retrograde signaling by astrocytes leads to the modulation of synaptic transmission and thus bidirectional collaboration of astrocyte with nearby neurons is an important aspect of self-repairing mechanism. Specifically, the retrograde signaling via astrocyte can increase the transmission probability of the healthy synapses linked to the neuron. Motivated by these findings, in the present research, a CMOS neuromorphic circuit with self-repairing capabilities is proposed based on astrocyte signaling. In this way, the computational model of self-repairing process is hired as a basis for designing a novel analog integrated circuit in the 180-nm CMOS technology. It is illustrated that the proposed analog circuit is able to successfully recompense the damaged synapses by appropriately modifying the voltage signals of the remaining healthy synapses in the wide range of frequency. The proposed circuit occupies 7500- [Formula: see text] silicon area and its power consumption is about [Formula: see text]. This neuromorphic fault-tolerant circuit can be considered as a key candidate for future silicon neuronal systems and implementation of neurorobotic and neuro-inspired circuits.

ObjectiveThe compromise of the hippocampal loop is a hallmark of mesial temporal lobe epilepsy (MTLE); particularly, the hippocampal output to the parahippocampal cortex is disrupted by damage of the CA1. While closed-loop deep brain stimulation (DBS) is the latest frontier to improve drug-refractory MTLE, current approaches do not restore the hippocampal loop, are designed by trial-and-error and heavily rely on seizure detection or prediction algorithms. The objective of this study is to evaluate the efficacy and robustness of bridging hippocampus and cortex via closed-loop stimulation to achieve the functional restoration of the hippocampal loop and control limbic seizures.ApproachIn hippocampus-cortex slices treated with 4-aminopyridine and in which the Schaffer Collaterals are severed, we used interictal discharges originating in the CA3 to trigger stimulation in the subiculum and re-establish the hippocampus output to the cortex. Combining tools from information theory with quantification of ictal activity, we addressed the efficacy of the bridge in restoring the functional connectivity of the hippocampal loop and controlling ictogenesis.Main resultsBridging hippocampus and cortex recovered the functional connectivity of the hippocampal loop, controlled ictogenesis and proved robust to failure mimicking the functional impairment of the CA3 seen in MTLE rodent models and patients. The efficacy and robustness of the bridge stem in mirroring the adaptive properties of the CA3, which acts as biological neuromodulator.SignificanceA DBS device that does not depend on seizure detection/prediction algorithms but relies on endogenous interictal patterns presents the key to advance the conceptual design of current DBS paradigms for epilepsy treatment.

This paper presents a sensory fusion neuromorphic dataset collected with precise temporal synchronization using a set of Address-Event-Representation sensors and tools. The target application is the lip reading of several keywords for different machine learning applications, such as digits, robotic commands, and auxiliary rich phonetic short words. The dataset is enlarged with a spiking version of an audio-visual lip reading dataset collected with frame-based cameras. LIPSFUS is publicly available and it has been validated with a deep learning architecture for audio and visual classification. It is intended for sensory fusion architectures based on both artificial and spiking neural network algorithms., Comment: Submitted to ISCAS2023, 4 pages, plus references, github link provided

For neuromorphic engineering to emulate the human brain, improving memory density with low power consumption is an indispensable but challenging goal. In this regard, emerging RRAMs have attracted considerable interest for their unique qualities like low power consumption, high integration potential, durability, and CMOS compatibility. Using RRAMs to imitate the more analog storage behavior of brain synapses is also a promising strategy for further improving memory density and power efficiency. However, RRAM devices display strong stochastic behavior, together with relaxation effects, making it more challenging to precisely control their multi-level storage capability. To address this, researchers have reported different multi-level programming strategies, mostly involving the precise control of analog parameters like compliance current during write operations and/or programming voltage amplitudes. Here, we present a new fully digital relaxation-aware method for tuning the conductance of analog RRAMs. The method is based on modulating digital pulse widths during erase operations while keeping other parameters fixed, and therefore requires no precise alterations to analog parameters like compliance currents or programming voltage amplitudes. Experimental results, with and without relaxation effect awareness, on a 64 RRAM 1T1R HfOx memory array of cells, fabricated in 130nm CMOS technology, indicate that it is possible to obtain 2-bit memory per cell multi-value storage at the array level, verified 1000 seconds after programming., Comment: 5 pages, 10 figures, 2 tables

Sound source localization is used in various applications such as industrial noise-control, speech detection in mobile phones, speech enhancement in hearing aids and many more. Newest video conferencing setups use sound source localization. The position of a speaker is detected from the difference in the audio waves received by a microphone array. After detection the camera focuses onto the location of the speaker. The human brain is also able to detect the location of a speaker from auditory signals. It uses, among other cues, the difference in amplitude and arrival time of the sound wave at the two ears, called interaural level and time difference. However, the substrate and computational primitives of our brain are different from classical digital computing. Due to its low power consumption of around 20 Watts and its performance in real time the human brain has become a great source of inspiration for emerging technologies. One of these technologies is neuromorphic hardware which implements the fundamental principles of brain computing identified until today using \ac{CMOS} technologies and new devices. In this work we propose the first neuromorphic closed-loop robotic system that uses the interaural time difference for sound source localization in real time. Our system can successfully locate sound sources such as human speech. In a closed-loop experiment, the binaural robotic platform turned immediately into the direction of the sound source with a turning velocity linearly proportional to the angle difference between sound source and pan-tilt unit. After this initial turn, the robotic platform remains at the direction of the sound source. Even though the system only uses very few resources of the available hardware and was only tuned by hand it already reaches performances comparable to other neuromorphic approaches. The sound source localization system presented in this article brings us one step closer towards neuromorphic event-based systems for robotics and embodied computing.

Foveation can be defined as the organic action of directing the gaze towards a visual region of interest, to acquire relevant information selectively. With the recent advent of event cameras, we believe that taking advantage of this visual neuroscience mechanism would greatly improve the efficiency of event-data processing. Indeed, applying foveation to event data would allow to comprehend the visual scene while significantly reducing the amount of raw data to handle. In this respect, we demonstrate the stakes of neuromorphic foveation theoretically and empirically across several computer vision tasks, namely semantic segmentation and classification. We show that foveated event data has a significantly better trade-off between quantity and quality of the information conveyed than high or low resolution event data. Furthermore, this compromise extends even over fragmented datasets. Our code is publicly available online at: github.com/amygruel/FoveationStakes DVS/.

Non-intrusive visual-based applications supporting the communication of people employing sign language for communication are always an open and attractive research field for the human action recognition community. Automatic sign language interpretation is a complex visual recognition task where motion across time distinguishes the sign being performed. In recent years, the development of robust and successful deep-learning techniques has been accompanied by the creation of a large number of databases. The availability of challenging datasets of Sign Language (SL) terms and phrases helps to push the research to develop new algorithms and methods to tackle their automatic recognition. This paper presents ‘SL-Animals-DVS’, an event-based action dataset captured by a Dynamic Vision Sensor (DVS). The DVS records non-fluent signers performing a small set of isolated words derived from SL signs of various animals as a continuous spike flow at very low latency. This is especially suited for SL signs which are usually made at very high speeds. We benchmark the recognition performance on this data using three state-of-the-art Spiking Neural Networks (SNN) recognition systems. SNNs are naturally compatible to make use of the temporal information that is provided by the DVS where the information is encoded in the spike times. The dataset has about 1100 samples of 59 subjects performing 19 sign language signs in isolation at different scenarios, providing a challenging evaluation platform for this emerging technology.

OpenNAS is an open-source tool for automatically generating the source files to create a Neuromorphic Auditory Sensor (NAS) VHDL project for FPGA. OpenNAS guides the user with a friendly interface that allows configuring the NAS’ parameters using a five-step wizard for code generation. OpenNAS provides support to several audio input interfaces (AC’97 audio codec, I2S-ADC and PDM microphones), different processing architectures (cascade and parallel), and neuromorphic output interfaces (parallel AER, SpiNNaker). After NAS generation, users have everything ready for building, simulating, and synthesizing the VHDL project for a target FPGA. OpenNAS is fully modular, which allows providing support to new features in an easy way Ministerio de Economía y Competitividad TEC2016-77785-P (COFNET)

This paper studies the suitability of neuromorphic event-based vision cameras for spaceflight and the effects of neutron radiation on their performance. Neuromorphic event-based vision cameras are novel sensors that implement asynchronous, clockless data acquisition, providing information about the change in illuminance $\ge 120dB$ with sub-millisecond temporal precision. These sensors have huge potential for space applications as they provide an extremely sparse representation of visual dynamics while removing redundant information, thereby conforming to low-resource requirements. An event-based sensor was irradiated under wide-spectrum neutrons at Los Alamos Neutron Science Center and its effects were classified. Radiation-induced damage of the sensor under wide-spectrum neutrons was tested, as was the radiative effect on the signal-to-noise ratio of the output at different angles of incidence from the beam source. We found that the sensor had very fast recovery during radiation, showing high correlation of noise event bursts with respect to source macro-pulses. No statistically significant differences were observed between the number of events induced at different angles of incidence but significant differences were found in the spatial structure of noise events at different angles. The results show that event-based cameras are capable of functioning in a space-like, radiative environment with a signal-to-noise ratio of 3.355. They also show that radiation-induced noise does not affect event-level computation. Finally, we introduce the Event-based Radiation-Induced Noise Simulation Environment (Event-RINSE), a simulation environment based on the noise-modelling we conducted and capable of injecting the effects of radiation-induced noise from the collected data to any stream of events in order to ensure that developed code can operate in a radiative environment. To the best of our knowledge, this is the first time such analysis of neutron-induced noise has been performed on a neuromorphic vision sensor, and this study shows the advantage of using such sensors for space applications.

This paper investigates the use of the body terminal of MOS transistors to improve the linearity of some key circuits used to implement analog and mixed-signal circuits integrated in Fully Depleted Silicon on Insulator (FD-SOI) CMOS. This technology allows to increase the body factor with respect to conventional (bulk) CMOS processes. This effect is analyzed in basic analog building blocks – such as switches, simple-stage transconductors and Voltage-Controlled Ring Oscillators (VCROs). Approximated expressions are derived for the nonlinear characteristics and harmonic distortion of some of these circuits. As an application, transistor-level simulations of two VCRO-based $\Sigma \Delta $ modulators designed in a 28-nm FD-SOI CMOS technology are shown in order to demonstrate the benefits of the presented techniques.

Initially, robots were developed with the aim of making our life easier, carrying out repetitive or dangerous tasks for humans. Although they were able to perform these tasks, the latest generation of robots are being designed to take a step further, by performing more complex tasks that have been carried out by smart animals or humans up to date. To this end, inspiration needs to be taken from biological examples. For instance, insects are able to optimally solve complex environment navigation problems, and many researchers have started to mimic how these insects behave. Recent interest in neuromorphic engineering has motivated us to present a real-time, neuromorphic, spike-based Central Pattern Generator of application in neurorobotics, using an arthropod-like robot. A Spiking Neural Network was designed and implemented on SpiNNaker. The network models a complex, online-change capable Central Pattern Generator which generates three gaits for a hexapod robot locomotion in real-time. Reconfigurable hardware was used to manage both the motors of the robot and the low-latency communication interface with the Spiking Neural Networks. Real-time measurements confirm the simulation results, and locomotion tests show that NeuroPod can perform the gaits without any balance loss or added delay.

Working memory is a fundamental feature of biological brains for perception, cognition, and learning. In addition, learning with working memory, which has been show in conventional artificial intelligence systems through recurrent neural networks, is instrumental to advanced cognitive intelligence. However, it is hard to endow a simple neuron model with working memory, and to understand the biological mechanisms that have resulted in such a powerful ability at the neuronal level. This article presents a novel self-adaptive multicompartment spiking neuron model, referred to as SAM, for spike-based learning with working memory. SAM integrates four major biological principles including sparse coding, dendritic non-linearity, intrinsic self-adaptive dynamics, and spike-driven learning. We first describe SAM’s design and explore the impacts of critical parameters on its biological dynamics. We then use SAM to build spiking networks to accomplish several different tasks including supervised learning of the MNIST dataset using sequential spatiotemporal encoding, noisy spike pattern classification, sparse coding during pattern classification, spatiotemporal feature detection, meta-learning with working memory applied to a navigation task and the MNIST classification task, and working memory for spatiotemporal learning. Our experimental results highlight the energy efficiency and robustness of SAM in these wide range of challenging tasks. The effects of SAM model variations on its working memory are also explored, hoping to offer insight into the biological mechanisms underlying working memory in the brain. The SAM model is the first attempt to integrate the capabilities of spike-driven learning and working memory in a unified single neuron with multiple timescale dynamics. The competitive performance of SAM could potentially contribute to the development of efficient adaptive neuromorphic computing systems for various applications from robotics to edge computing.

Hearing can be considered as one of the most important senses since it plays a key role in the audiovisual learning process. While a lot of effort has been made for achieving good results from the traditional approach of auditory perception, new trends such as neuromorphic computing are showing promising achievements in the implementation of brain structures for sensory perception. In this work, the design and integration of a neuromorphic event based digital model of the auditory ascending pathway within the iCub robotic platform is proposed. This model, which comprises from the cochlea up to the inferior colliculus, replaces the traditional approach for sound processing already implemented on iCub and is able to perform sound recognition and spatial localization in real time, allowing the implementation of auditory attention models in complex scenarios, like the cocktail party problem Agencia Estatal de Investigación PID2019-105556GB-C33/AEI/10.13039/501100011033 (MINDROB) Agencia Estatal de Investigación PCI2019-111841-2/AEI/10.1309/501100011033 (SMALL)

Spiking neural networks (SNNs) are regarded as a promising candidate to deal with the major challenges of current machine learning techniques, including the high energy consumption induced by deep neural networks. However, there is still a great gap between SNNs and the few-shot learning performance of artificial neural networks. Importantly, existing spike-based few-shot learning models do not target robust learning based on spatiotemporal dynamics and superior machine learning theory. In this paper, we propose a novel spike-based framework with the entropy theory, namely, heterogeneous ensemble-based spike-driven few-shot online learning (HESFOL). The proposed HESFOL model uses the entropy theory to establish the gradient-based few-shot learning scheme in a recurrent SNN architecture. We examine the performance of the HESFOL model based on the few-shot classification tasks using spiking patterns and the Omniglot data set, as well as the few-shot motor control task using an end-effector. Experimental results show that the proposed HESFOL scheme can effectively improve the accuracy and robustness of spike-driven few-shot learning performance. More importantly, the proposed HESFOL model emphasizes the application of modern entropy-based machine learning methods in state-of-the-art spike-driven learning algorithms. Therefore, our study provides new perspectives for further integration of advanced entropy theory in machine learning to improve the learning performance of SNNs, which could be of great merit to applied developments with spike-based neuromorphic systems.

Biological nervous systems typically perform the control of numerous degrees of freedom for example in animal limbs. Neuromorphic engineers study these systems by emulating them in hardware for a deeper understanding and its possible application to solve complex problems in engineering and robotics. Central-Pattern-Generators (CPGs) are part of neuro-controllers, typically used at their last steps to produce rhythmic patterns for limbs movement. Different patterns and gaits typically compete through winner-take-all (WTA) circuits to produce the right movements. In this work we present a WTA circuit implemented in a Spiking-Neural-Network (SNN) processor to produce such patterns for controlling a robotic arm in real-time. The robot uses spike-based proportional-integrativederivative (SPID) controllers to keep a commanded joint position from the winner population of neurons of the WTA circuit. Experiments demonstrate the feasibility of robotic control with spiking circuits following brain-inspiration., Comment: 5 pages, 4 figures, submitted to ISCAS2022

In this work, an all-in-one neuromorphic controller system with reduced latency and power consumption for a robotic arm is presented. Biological muscle movement consists of stretching and shrinking fibres via spike-commanded signals that come from motor neurons, which in turn are connected to a central pattern generator neural structure. In addition, biological systems are able to respond to diverse stimuli rather fast and efficiently, and this is based on the way information is coded within neural processes. As opposed to human-created encoding systems, neural ones use neurons and spikes to process the information and make weighted decisions based on a continuous learning process. The Event-Driven Scorbot platform (ED-Scorbot) consists of a 6 Degrees of Freedom (DoF) robotic arm whose controller implements a Spiking Proportional-Integrative- Derivative algorithm, mimicking in this way the previously commented biological systems. In this paper, we present an infrastructure upgrade to the ED-Scorbot platform, replacing the controller hardware, which was comprised of two Spartan Field Programmable Gate Arrays (FPGAs) and a barebone computer, with an edge device, the Xilinx Zynq-7000 SoC (System on Chip) which reduces the response time, power consumption and overall complexity., Comment: 4 pages plus references, 5 figures, submitted to ISCAS2022

Latest advances in CMOS technology, neuromorphic computing and flexible electronics are leading the way to a new era of brain implantable devices that promise to provide innovative and more effective treatments for brain disorders. Real-time signal detection and classification, as well as anticipation of brain electrical activity are yet under development. Microelectrode arrays (MEA) arise as a promising technology enabling detection of local field potentials from multiple locations and permitting the acquisition of more information on brain network electrical activity than conventional electrophysiology techniques. However, whereas most of the electrophysiological studies addressing brain activity have focused on events/patterns analysis, no one has so far addressed the features that might be hidden within the signal baseline. Such features might be particularly relevant in the context of epilepsy, as the signal baseline may carry relevant information for seizure prediction. Here, we present a preliminary processing and analysis of signal baseline acquired through MEA in an in vitro model of limbic seizures. The signal conditioning was implemented using an infinite impulse response (IIR) digital filter. After signal preprocessing, we applied an averaging method to 16 baseline sections to find common patterns and to study the frequency spectrum of this type of signal. We have found signal components between 0.5-2 Hz and peaks at 350, 390 and 650 Hz. In addition, the reconstruction of the averaged signal may provide insights into the main baseline wave patterns. These results might be a preliminary step to study the influence of those components on a biological basis. Based on these results, we propose a possible electronic architecture implementation of the signal processing method.

Deep-learning is a cutting edge theory that is being applied to many fields. For vision applications the Convolutional Neural Networks (CNN) are demanding significant accuracy for classification tasks. Numerous hardware accelerators have populated during the last years to improve CPU or GPU based solutions. This technology is commonly prototyped and tested over FPGAs before being considered for ASIC fabrication for mass production. The use of commercial typical cameras (30fps) limits the capabilities of these systems for high speed applications. The use of dynamic vision sensors (DVS) that emulate the behaviour of a biological retina is taking an incremental importance to improve this applications due to its nature, where the information is represented by a continuous stream of spikes and the frames to be processed by the CNN are constructed collecting a fixed number of these spikes (called events). The faster an object is, the more events are produced by DVS, so the higher is the equivalent frame rate. Therefore, these DVS utilization allows to compute a frame at the maximum speed a CNN accelerator can offer. In this paper we present a VHDL/HLS description of a pipelined design for FPGA able to collect events from an Address-Event-Representation (AER) DVS retina to obtain a normalized histogram to be used by a particular CNN accelerator, called NullHop. VHDL is used to describe the circuit, and HLS for computation blocks, which are used to perform the normalization of a frame needed for the CNN. Results outperform previous implementations of frames collection and normalization using ARM processors running at 800MHz on a Zynq7100 in both latency and power consumption. A measured 67% speed-up factor is presented for a Roshambo CNN real-time experiment running at 160fps peak rate.

This paper presents a biohybrid arrangement made of a commercial microelectrode array (MEA) system for seizure-like activity detection in brain slices. The set-up takes advantage of an embedded fixed-point digital signal processor (DSP) to implement a neuron model and a field-potential to spike converter (FP2SP). The neuron model is biologically plausible and capable of generating various firing modalities. Based on a three-step algorithm, FP2SP extracts spikes from the epileptiform activity generated by brain slices. The seizure detector system is developed by connecting the FP2SP to the model neuron and properly tuning the FP2SP parameters. The results show that all the blocks of this system can operate properly in real-time mode and recognize seizure-like activity.

Neuromorphic computing is a promising technology that realizes computation based on event-based spiking neural networks (SNNs). However, fault-tolerant on-chip learning remains a challenge in neuromorphic systems. This study presents the first scalable neuromorphic fault-tolerant context-dependent learning (FCL) hardware framework. We show how this system can learn associations between stimulation and response in two context-dependent learning tasks from experimental neuroscience, despite possible faults in the hardware nodes. Furthermore, we demonstrate how our novel fault-tolerant neuromorphic spike routing scheme can avoid multiple fault nodes successfully and can enhance the maximum throughput of the neuromorphic network by 0.9%-16.1% in comparison with previous studies. By utilizing the real-time computational capabilities and multiple-fault-tolerant property of the proposed system, the neuronal mechanisms underlying the spiking activities of neuromorphic networks can be readily explored. In addition, the proposed system can be applied in real-time learning and decision-making applications, brain-machine integration, and the investigation of brain cognition during learning.

The emergence of nano-scale memristive devices encouraged many different research areas to exploit their use in multiple applications. One of the proposed applications was to implement synaptic connections in bio-inspired neuromorphic systems. Large-scale neuromorphic hardware platforms are being developed with increasing number of neurons and synapses, having a critical bottleneck in the online learning capabilities. Spike-timing-dependent plasticity (STDP) is a widely used learning mechanism inspired by biology which updates the synaptic weight as a function of the temporal correlation between pre- and post-synaptic spikes. In this work, we demonstrate experimentally that binary stochastic STDP learning can be obtained from a memristor when the appropriate pulses are applied at both sides of the device.

Brain-inspired computing employs devices and architectures that emulate biological functions for more adaptive and energy-efficient systems. Oscillatory neural networks (ONNs) are an alternative approach in emulating biological functions of the human brain and are suitable for solving large and complex associative problems. In this work, we investigate the dynamics of coupled oscillators to implement such ONNs. By harnessing the complex dynamics of coupled oscillatory systems, we forge a novel computation model—information is encoded in the phase of oscillations. Coupled interconnected oscillators can exhibit various behaviors due to the strength of the coupling. In this article, we present a novel method based on subharmonic injection locking (SHIL) for controlling the oscillatory states of coupled oscillators that allow them to lock in frequency with distinct phase differences. Circuit-level simulation results indicate SHIL effectiveness and its applicability to large-scale oscillatory networks for pattern recognition. European Union’s Horizon 2020 871501

The emergence of nano-scale memristive devices encouraged many different research areas to exploit their use in multiple applications. One of the proposed applications was to implement synaptic connections in bio-inspired neuromorphic systems. Large-scale neuromorphic hardware platforms are being developed with increasing number of neurons and synapses, having a critical bottleneck in the online learning capabilities. Spiketiming- dependent plasticity (STDP) is a widely used learning mechanism inspired by biology which updates the synaptic weight as a function of the temporal correlation between pre- and postsynaptic spikes. In this work, we demonstrate experimentally that binary stochastic STDP learning can be obtained from a memristor when the appropriate pulses are applied at both sides of the device. EU H2020 grant 824164 "HERMES" EU H2020 grant 871371 "Memscales" EU H2020 grant 871501 "NeurONN" EU H2020 grant PCI2019-111826-2 "APROVIS3D" EU H2020 grant 899559 "SpinAge" Ministry of Science and Innovation (Spain) PID2019-105556GB-C31 Ministry of Science and Innovation ( Spain) PID2019-103876RB-I00 (CORDION) Ministry of Economy and Competitivity (Spain) / FEDER TEC2015- 63884-C2-1-P (COGNET) Junta de Andalucía (Spain) US-1260118 (Neuro-Radio) Universidad de Sevilla (Spain) VI PPIT

Prostate cancer (PCa) is one of the most commonly diagnosed cancer and one of the leading causes of death among men, with almost 1.41 million new cases and around 375,000 deaths in 2020. Artificial Intelligence algorithms have had a huge impact in medical image analysis, including digital histopathology, where Convolutional Neural Networks (CNNs) are used to provide a fast and accurate diagnosis, supporting experts in this task. To perform an automatic diagnosis, prostate tissue samples are first digitized into gigapixel-resolution whole-slide images. Due to the size of these images, neural networks cannot use them as input and, therefore, small subimages called patches are extracted and predicted, obtaining a patch-level classification. In this work, a novel patch aggregation method based on a custom Wide & Deep neural network model is presented, which performs a slide-level classification using the patch-level classes obtained from a CNN. The malignant tissue ratio, a 10-bin malignant probability histogram, the least squares regression line of the histogram, and the number of malignant connected components are used by the proposed model to perform the classification. An accuracy of 94.24% and a sensitivity of 98.87% were achieved, proving that the proposed system could aid pathologists by speeding up the screening process and, thus, contribute to the fight against PCa., 9 pages, 5 figures

Autonomous driving solutions are based on artificial vision and machine learning for understanding the environment and facilitate decision making tasks. Similar techniques are used for indoor robot navigation. Deep learning architectures, which are usually computationally expensive, are impacting our daily lives. This technology is evolving with a notable improvement of cost-efficiency in terms of energy consumption, enabling AI-edge computing. However, these architectures are usually trained on powerful GPUs, what represents the limit for edge computing. Nevertheless, after this training, efficient edge computing devices can process these architectures locally. Neuromorphic engineering shows off on solving the energy bottleneck problem through bio-inspired sensors, processors and spike-based computation techniques. This work presents a mobile robotic platform commanded through the Robotic Operating System (ROS), which obeys the classification output of an AI-edge CNN accelerator for FPGA connected to a neuromorphic dynamic vision sensor. The classification system is able to process up to 200 fps for 64 \(\times \) 64 histograms collected with 2k events per frame and executing a 5 layer CNN with 18MOPs for indoor robot navigation. A traffic sign dataset has been used for training achieving a measured accuracy of 97.62% and 99.96% in the validation and test datasets respectively.

The growing development of Internet of Things (IoT) devices is producing an increasing use of the electromagnetic spectrum for wireless communications. Cognitive Radio (CR) technology provides communication terminals with the capability to select arbitrary frequency bands dynamically in order to make a more efficient use of the frequency spectrum and bands occupied by different standards and communication protocols. In this work, we propose a system which uses Long Short-Term Memory (LSTM) networks to predict the future occupation of frequency bands and modifies the specifications of the analog and radio-frequency front-end, adapting dynamically to the best communication channel. System-level simulations of a band-pass filter are shown as a case study to validate the presented approach.

This paper presents a pulse-extender, a delay-element, and a current-attenuator as auxiliary circuits that make it possible to have flexible interaction with memristor crossbars in spiking neural networks. In the presynaptic part, the pulse-extender makes the inputs compatible with the pulsed-characterization of memristors. In the post-synaptic part, the current attenuator relaxes the system in terms of requiring low-offset amplifiers and also makes it possible to design neurons with small membrane capacitors. The circuits are fabricated in a CMOS 180nm technology. The measurements verify that these blocks play an important role in reaching an SNN with real-time performance.

Hebbian rule plays an important role in training of artificial neural networks. According to this rule, a synaptic weight between two neurons is increased or decreased depending on the activity of the presynaptic and postsynaptic neurons. In this paper, an oscillatory version of the Hebbian rule is proposed for ONNs and is called Oscillatory Hebbian Rule (OHR). OHR simply expresses the weight change as a function of the phase difference between the presynaptic and postsynaptic neurons. Similar to STDP that weight change is an exponential function of the time difference between the presynaptic and postsynaptic spikes, OHR relates weight change to the phase difference between the presynaptic and postsynaptic neurons using exponential functions. Specifically, when two neurons are in-phase, the weight between them is increased while a weight between two anti-phase neurons is decreased. Simulation results show the capability of OHR for both supervised and unsupervised learning. In supervised learning, a basic block of feedforward architectures is trained as a classifier. When the basic block is used in unsupervised mode, it is capable to learn patterns while the output phase is converged to a specific phase.

Human gestures recognition is a complex visual recognition task where motion across time distinguishes the type of action. Automatic systems tackle this problem using complex machine learning architectures and training datasets. In recent years, the use and success of robust deep learning techniques was compatible with the availability of a great number of these sets. This paper presents SL-Animals-DVS, an event-based action dataset captured by a Dynamic Vision Sensor (DVS). The DVS records humans performing sign language gestures of various animals as a continuous spike flow at very low latency. This is especially suited for sign language gestures which are usually made at very high speeds. We also benchmark the recognition performance on this data using two state-of-the-art Spiking Neural Networks (SNN) recognition systems. SNNs are naturally compatible to make use of the temporal information that is provided by the DVS where the information is encoded in the spike times. The dataset has about 1100 samples of 58 subjects performing 19 sign language gestures in isolation at different scenarios, providing a challenging evaluation platform for this emerging technology.

The brain cortex processes visual information to classify it following a scheme that has been mimicked by Convolutional Neural Networks (CNN). Specialised hardware accelerators are currently used as CPU co-processors for mobile applications. These accelerators are getting closer to the sensors for an edge computation of its output towards a faster and lower power consumption improvements. In this demonstration we use a dynamic vision sensor (inspired in the retina neural cells) as a visual source of the NullHop CNN accelerator deployed on a MPSoC FPGA and placed into a mobile robot for edge-computing the visual information and classify it to properly command a Summit-XL mobile robot for a target destiny. The reduced latency of the used CNN accelerator allows to process several histograms before taking a movement decision. A distance sensor mounted on the robot ensures that the direction change is done at the right distance for a proper path following.

Hybrid memristor-CMOS techniques have been recently proposed to build large-scale neural networks with learning capabilities. The intrinsic characteristics of memristors make them specially suited to implement synaptic connections between layers of spiking neurons, undergoing STDP learning (Spike-Timing-Dependent Plasticity) mechanisms when processing spikes with particular shapes. In a previous work, we proposed a tunable spiking neuron circuit which can generate spikes with controllable shape. In this work, the spike generator circuit has been implemented in FDSOI 28nm technology, and it has demonstrated its capability to produce spikes with pulse widths in the range between $8\mu {s}$ and 100ms.

Many advances have been made in the eld of computer vision. Several recent research trends have focused on mimicking human vision by using a stereo vision system. In multi-camera systems, a calibration process is usually implemented to improve the results accuracy. However, these systems generate a large amount of data to be processed; therefore, a powerful computer is required and, in many cases, this cannot be done in real time. Neuromorphic Engineering attempts to create bio-inspired systems that mimic the information processing that takes place in the human brain. This information is encoded using pulses (or spikes) and the generated systems are much simpler (in computational operations and resources), which allows them to perform similar tasks with much lower power consumption, thus these processes can be developed over specialized hardware with real-time processing. In this work, a bio-inspired stereovision system is presented, where a calibration mechanism for this system is implemented and evaluated using several tests. The result is a novel calibration technique for a neuromorphic stereo vision system, implemented over specialized hardware (FPGA - Field-Programmable Gate Array), which allows obtaining reduced latencies on hardware implementation for stand-alone systems, and working in real time. Ministerio de Economía y Competitividad TEC2016-77785-P Ministerio de Economía y Competitividad TIN2016-80644-P

Recent biological studies have focused on understanding animal interactions and welfare. To help biolo- gists to obtain animals’ behavior information, resources like wireless sensor networks are needed. More- over, large amounts of obtained data have to be processed off-line in order to classify different behaviors.There are recent research projects focused on designing monitoring systems capable of measuring someanimals’ parameters in order to recognize and monitor their gaits or behaviors. However, network unre- liability and high power consumption have limited their applicability.In this work, we present an animal behavior recognition, classification and monitoring system based ona wireless sensor network and a smart collar device, provided with inertial sensors and an embeddedmulti-layer perceptron-based feed-forward neural network, to classify the different gaits or behaviorsbased on the collected information. In similar works, classification mechanisms are implemented in aserver (or base station). The main novelty of this work is the full implementation of a reconfigurableneural network embedded into the animal’s collar, which allows a real-time behavior classification andenables its local storage in SD memory. Moreover, this approach reduces the amount of data transmittedto the base station (and its periodicity), achieving a significantly improving battery life. The system hasbeen simulated and tested in a real scenario for three different horse gaits, using different heuristics andsensors to improve the accuracy of behavior recognition, achieving a maximum of 81%. Junta de Andalucía P12-TIC-1300

Memristive devices have shown great promise to facilitate the acceleration and improve the power efficiency of Deep Learning (DL) systems. Crossbar architectures constructed using these Resistive Random-Access Memory (RRAM) devices can be used to efficiently implement various in-memory computing operations, such as Multiply Accumulate (MAC) and unrolled-convolutions, which are used extensively in Deep Neural Networks (DNNs) and Convolutional Neural Networks (CNNs). However, memristive devices face concerns of aging and non-idealities, which limit the accuracy, reliability, and robustness of Memristive Deep Learning Systems (MDLSs), that should be considered prior to circuit-level realization. This Original Software Publication (OSP) presents MemTorch, an open-source framework for customized large-scale memristive DL simulations, with a refined focus on the co-simulation of device non-idealities. MemTorch also facilitates co-modelling of key crossbar peripheral circuitry. MemTorch adopts a modernized soft-ware engineering methodology and integrates directly with the well-known PyTorch Machine Learning (ML) library, Comment: Accepted for Publication in Neurocomputing

Sound source localization has always been one of the most challenging subjects in different fields of engineering, one of the most important of which being tracking of flying objects. This article focuses on sound source localization using fuzzy fusion and a beamforming method. It proposes a new fuzzy-based algorithm for localizing a sound source using distributed sensor nodes. Eight low-cost sensor nodes have been constructed in this study each of which consists of a microphone array to capture sound waves. Each node is able to record audio signals synchronously on an SD card to evaluate different algorithms offline. However, the sensor nodes are designed to be able to estimate the location of the sound source in real-time. In the proposed algorithm, every node estimates the direction of the sound source. Moreover, a calibration algorithm is used for extracting the orientation of sensor nodes to calibrate the estimated directions. The calibrated directions are fuzzified and then used for localizing the sound source by fuzzy fusion. An experiment was designed based on localizing a flying quadcopter as a moving sound source to evaluate the performance of the proposed algorithm. The flying trajectory was then estimated and compared with the target trajectory extracted from the GPS module mounted on the quadcopter. Comparing the estimated sound source with the target location, a mean distance error of ${6.03}{m}$ was achieved in a wide-range outdoor environment with the size of ${240}\times {160}\times {80} \,\,{m}^{{3}}$ . The achieved mean distance error is reasonable regarding the mean precision of the GPS module. The practical results illustrate the effectiveness of the proposed approach in localizing a sound source in a wide-range outdoor environment.

Neural networks have enabled great advances in recent times due mainly to improved parallel computing capabilities in accordance to Moore’s Law, which allowed reducing the time needed for the parameter learning of complex, multi-layered neural architectures. However, with silicon technology reaching its physical limits, new types of computing paradigms are needed to increase the power efficiency of learning algorithms, especially for dealing with deep spatio-temporal knowledge on embedded applications. With the goal of mimicking the brain’s power efficiency, new hardware architectures such as the SpiNNaker board have been built. Furthermore, recent works have shown that networks using spiking neurons as learning units can match classical neural networks in supervised tasks. In this paper, we show that the implementation of state-of-the-art models on both the MNIST and the event-based NMNIST digit recognition datasets is possible on neuromorphic hardware. We use two approaches, by directly converting a classical neural network to its spiking version and by training a spiking network from scratch. For both cases, software simulations and implementations into a SpiNNaker 103 machine were performed. Numerical results approaching the state of the art on digit recognition are presented, and a new method to decrease the spike rate needed for the task is proposed, which allows a significant reduction of the spikes (up to 34 times for a fully connected architecture) while preserving the accuracy of the system. With this method, we provide new insights on the capabilities offered by networks of spiking neurons to efficiently encode spatio-temporal information. Consejo Nacional de Ciencia Y Tecnología (México) FC2016-1961 European Union's Horizon 2020 No 824164 HERMES Ministerio de Ciencia, Innovación y Universidades TEC2015-63884-C2-1-P

Reading several ReRAMs simultaneously in a neuromorphic circuit increases power consumption and limits scalability. Applying small inference read pulses is a vain attempt when offset voltages of the read-out circuit are decisively more. This paper presents an experimental validation of a three-stage calibration scheme to calibrate the DC offset voltage across the rows of the memristive crossbar. The proposed method is based on biasing the body terminal of one of the differential pair MOSFETs of the buffer through a series of cascaded resistor banks arranged in three stages- coarse, fine and finer stages. The circuit is designed in a 130 nm CMOS technology, where the OxRAM-based binary memristors are built on top of it. A dedicated PCB and other auxiliary boards have been designed for testing the chip. Experimental results validate the presented approach, which is only limited by mismatch and electrical noise. EU H2020 grant 687299 NeuRAM3 EU H2020 grant 824164 HERMES EU H2020 grant 871501 NeurONN EU H2020 grant 871371 MeM-Scales Spanish Ministry of Economy and Competitiveness TEC2015-63884-C2-1-P (COGNET) Spanish Ministry of Economy and Competitiveness G0086 ICON Universidad de Sevilla (España) VI PPIT

In this chapter we present results on system-level integration of memristive devices with neuromorphic circuits and systems. Specifically we present an overview of the current state-of-the-art hybrid memristive-complementary metal–oxide–semiconductor (CMOS) mixed-signal neuromorphic circuits for learning and plasticity and present perspectives toward integration of memristive devices in neuromorphic spiking neural network architectures. We focus on neuromorphic circuits and architectures that allow for a relatively natural integration of memristive devices, irrespective of the specific characteristics of the specific memristive device technology adopted. We address the cointegration of memristive devices with on-chip learning mechanisms, using both analog and digital CMOS circuits, to build a solid background of the functionality of neuromorphic circuits explaining how memristive devices can be implemented on them. Furthermore we also address the system-level integration of such architectures in multicore and multichip systems, for connecting them to input and output devices, such as sensors, actuators, and conventional CMOS processing devices.

One of the biggest struggles while working with artificial neural networks is being able to come up with models which closely match biological observations. Biological neural networks seem to capable of creating and pruning dendritic spines, leading to synapses being changed, which results in higher learning capability. The latter forms the basis of the present study in which a new ionic model for reservoir-like networks, consisting of spiking neurons, is introduced. High plasticity of this model makes learning possible with a fewer number of neurons. In order to study the effect of the applied stimulus in an ionic liquid space through time, a diffusion operator is used which somehow compensates for the separation between spatial and temporal coding in spiking neural networks and therefore, makes the mentioned model suitable for spatiotemporal patterns. Inspired by partial structural changes in the human brain over the years, the proposed model evolves during the learning process. The effect of topological evolution on the proposed model’s performance for some classification problems is studied in this paper. Several datasets have been used to evaluate the performance of the proposed model compared to the original LSM. Classification results via separation and accuracy values have shown that the proposed ionic liquid outperforms the original LSM

The paper presents a realization on a digital signal processor (DSP) of an Izhikevich Neural Network operating with biologically plausible real-time constants. The paper demonstrates the real-time realization of different neuron behavioral modes, i.e., regular-spiking, chattering, bursting, and fast-spiking under proper parametrization. Real-time spike-timing-dependent-plasticity has also been embedded in the neural network realization. The paper studies the maximum array size that can be implemented on a TMS320C6455 microprocessor to be able to reproduce correctly the real-time dynamics of the different behaviors. The TMS320C6454, from the same DSP family as TMS320C6455, is embedded in a commercial microelectrode array system for real-time interaction with biological neural cell cultures. As demonstrator, a simple classification of two binary patterns has been implemented. Upon learning activation, the system robustly unsupervisely learns to differentiate the two patterns.

We present a fully configurable spiking convolutional node that can be used to ensemble large arbitrary Convolutional Neural Networks (ConvNets) on FPGA. The node includes a rate saturation mechanism to implement a refractory period. Using this node, a 4-layer ConvNet with 22 convolutional nodes and a decision block has been implemented in an FPGA. This network has been trained for recognition of poker card symbols and tested with a stimulus with 40 poker cards recorded by a silicon retina. In this paper, we study different strategies for the decision block to maximize the recognition rate with minimum power consumption.