Search

Sound source localization (SSL) is used in various applications such as industrial noise-control, speech detection in mobile phones, speech enhancement in hearing aids and many more. Newest video conferencing setups use SSL. The position of a speaker is detected from the difference in the audio waves received by a microphone array. After detection the camera focuses onto the location of the speaker. The human brain is also able to detect the location of a speaker from auditory signals. It uses, among other cues, the difference in amplitude and arrival time of the sound wave at the two ears, called interaural level and time difference. However, the substrate and computational primitives of our brain are different from classical digital computing. Due to its low power consumption of around 20 W and its performance in real time the human brain has become a great source of inspiration for emerging technologies. One of these technologies is neuromorphic hardware which implements the fundamental principles of brain computing identified until today using complementary metal-oxide-semiconductor technologies and new devices. In this work we propose the first neuromorphic closed-loop robotic system that uses the interaural time difference for SSL in real time. Our system can successfully locate sound sources such as human speech. In a closed-loop experiment, the robotic platform turned immediately into the direction of the sound source with a turning velocity linearly proportional to the angle difference between sound source and binaural microphones. After this initial turn, the robotic platform remains at the direction of the sound source. Even though the system only uses very few resources of the available hardware, consumes around 1 W, and was only tuned by hand, meaning it does not contain any learning at all, it already reaches performances comparable to other neuromorphic approaches. The SSL system presented in this article brings us one step closer towards neuromorphic event-based systems for robotics and embodied computing.

During the past few decades, a significant amount of research effort has been dedicated toward developing skin-inspired sensors for real-time human motion monitoring and next-generation robotic devices. Although several flexible and wearable sensors have been developed in the past, the need of the hour is developing accurate, reliable, sophisticated, facile yet inexpensive flexible sensors coupled with neuromorphic systems or spiking neural networks to encode tactile information without the need for complex digital architectures, thus achieving true skin-like sensing with limited resources. In this work, we propose an approach entailing carbon nanofiber-polydimethylsiloxane composite-based piezoresistive sensors, coupled with spiking neural networks, to mimic skin-like sensing. The strain and pressure sensors have been combined with appropriately designed neural networks to encode analog voltages to spikes to recreate bioinspired tactile sensing and proprioception. To further validate the proprioceptive capability of the system, a gesture tracking smart glove, combined with a spiking neural network, was demonstrated. Wearable and flexible sensors with accompanying neural networks such as the ones proposed in this work will pave the way for a future generation of skin-mimetic sensors for advanced prosthetic devices, apparel integrable smart sensors for human motion monitoring, and human-machine interfaces.

Attractor networks are widely understood to be a reoccurring primitive that underlies cognitive function. Stabilising activity in spiking attractor networks however remains a difficult task, especially when implemented in analog integrated circuits (aIC). We introduce here a novel circuit implementation of a hard Winner-Take-All (hWTA) mechanism, in which competing neurons’ refractory circuits are coupled together, and thus their spiking is forced to be mutually exclusive. We demonstrate stable persistent-firing attractor dynamics in a small on-chip network consisting of hWTA-connected neurons and excitatory recurrent synapses. Its utility within larger networks is demonstrated in simulation, and shown to support overlapping attractors and be robust to synaptic weight mismatch. The realised hWTA mechanism is thus useful for stabilising activity in spiking networks composed of unreliable components, without the need for careful parameter tuning.

Spatio-temporal pattern recognition is a fundamental ability of the brain which is required for numerous real-world activities. Recent deep learning approaches have reached outstanding accuracies in such tasks, but their implementation on conventional embedded solutions is still very computationally and energy expensive. Tactile sensing in robotic applications is a representative example where real-time processing and energy efficiency are required. Following a brain-inspired computing approach, we propose a new benchmark for spatio-temporal tactile pattern recognition at the edge through Braille letter reading. We recorded a new Braille letters dataset based on the capacitive tactile sensors of the iCub robot's fingertip. We then investigated the importance of spatial and temporal information as well as the impact of event-based encoding on spike-based computation. Afterward, we trained and compared feedforward and recurrent Spiking Neural Networks (SNNs) offline using Backpropagation Through Time (BPTT) with surrogate gradients, then we deployed them on the Intel Loihi neuromorphic chip for fast and efficient inference. We compared our approach to standard classifiers, in particular to the Long Short-Term Memory (LSTM) deployed on the embedded NVIDIA Jetson GPU, in terms of classification accuracy, power, and energy consumption together with computational delay. Our results show that the LSTM reaches ~97% of accuracy, outperforming the recurrent SNN by ~17% when using continuous frame-based data instead of event-based inputs. However, the recurrent SNN on Loihi with event-based inputs is ~500 times more energy-efficient than the LSTM on Jetson, requiring a total power of only ~30 mW. This work proposes a new benchmark for tactile sensing and highlights the challenges and opportunities of event-based encoding, neuromorphic hardware, and spike-based computing for spatio-temporal pattern recognition at the edge.

We report the design in CMOS technology and the experimental characterization of an analog spiking neural network with on-chip unsupervised learning. Long-term synaptic memory is implemented using a floating-gate device in a standard 150 nm CMOS process. The neurons are operated with a voltage supply of only 0.4V, allowing an extremely low power dissipation with an energy dissipation per synaptic operation of about 55 fJ. The CMOS chip includes the circuits for implementing real-time learning of the network based on the Spike Time Dependent Plasticity algorithm. During the learning, the neurons produce pulses of ±4.5 V that change the synaptic weight by activating tunneling currents to change the charge in the floating gates.

The present invention relates to an event-driven integrated circuit, comprising a sensor, an interface system and a processor. By means of the solution, events with addresses are asynchronously generated and processed. The interface system comprises a replication module, a fusion module, a secondary sampling module, a region of interest module, an event routing module, etc., which constitute a programmable daisy chain. The sensor, the interface system and the processor are coupled on a single chip by means of an adapter board, and different bare dies can be manufactured by using different processes. By means of the solution, signal loss and noise interference in the prior art can be eliminated, and the technical effects of high-speed processing of signals, smaller footprints of chips, reduced manufacturing costs, etc., are achieved, thereby solving the technical problems in the prior art of a large chip area and a low signal processing capability. In addition, by means of the smart design of the interface system, the functions and configurability of the interface system are enriched, thereby providing various advantages in terms of power consumption, functions and speed in subsequent processing.

The field of artificial intelligence has significantly advanced over the past decades, inspired by discoveries from the fields of biology and neuroscience. The idea of this work is inspired by the process of self-organization of cortical areas in the human brain from both afferent and lateral/internal connections. In this work, we develop an original brain-inspired neural model associating Self-Organizing Maps (SOM) and Hebbian learning in the Reentrant SOM (ReSOM) model. The framework is applied to multimodal classification problems. Compared to existing methods based on unsupervised learning with post-labeling, the model enhances the state-of-the-art results. This work also demonstrates the distributed and scalable nature of the model through both simulation results and hardware execution on a dedicated FPGA-based platform named SCALP (Self-configurable 3D Cellular Adaptive Platform). SCALP boards can be interconnected in a modular way to support the structure of the neural model. Such a unified software and hardware approach enables the processing to be scaled and allows information from several modalities to be merged dynamically. The deployment on hardware boards provides performance results of parallel execution on several devices, with the communication between each board through dedicated serial links. The proposed unified architecture, composed of the ReSOM model and the SCALP hardware platform, demonstrates a significant increase in accuracy thanks to multimodal association, and a good trade-off between latency and power consumption compared to a centralized GPU implementation.

Neuromorphic systems are a viable alternative to conventional systems for real-time tasks with constrained resources. Their low power consumption, compact hardware realization, and low-latency response characteristics are the key ingredients of such systems. Furthermore, the event-based signal processing approach can be exploited for reducing the computational load and avoiding data loss due to its inherently sparse representation of sensed data and adaptive sampling time. In event-based systems, the information is commonly coded by the number of spikes within a specific temporal window. However, the temporal information of event-based signals can be difficult to extract when using rate coding. In this work, we present a novel digital implementation of the model, called time difference encoder (TDE), for temporal encoding on event-based signals, which translates the time difference between two consecutive input events into a burst of output events. The number of output events along with the time between them encodes the temporal information. The proposed model has been implemented as a digital circuit with a configurable time constant, allowing it to be used in a wide range of sensing tasks that require the encoding of the time difference between events, such as optical flow-based obstacle avoidance, sound source localization, and gas source localization. This proposed bioinspired model offers an alternative to the Jeffress model for the interaural time difference estimation, which is validated in this work with a sound source lateralization proof-of-concept system. The model was simulated and implemented on a field-programmable gate array (FPGA), requiring 122 slice registers of hardware resources and less than 1 mW of power consumption. Ministerio de Economía y Competitividad TEC2016-77785-P (COFNET) Agencia Estatal de Investigación PID2019-105556GB-C33/AEI/10.13039/501100011033 (MINDROB)

The invention discloses a data updating method, a storage space setting method and device, a chip and equipment. In order to solve the technical problems that in the prior art, the classification number of data reading layers in a neural mimicry chip is fixed, the occupied area of the chip is large, and shared storage cannot be achieved, a pulse sequence output by a pulse neural network is read through the data reading layers of a memory. The method comprises the steps of: analyzing the pulse sequence to determine a prediction type of the pulse neural network for the input data; determining a target storage address corresponding to the prediction type in a storage address sequence of a memory; and updating the pulse aggregation analysis information stored in the first storage space and the pulse counting statistical information stored in the second storage space according to the pulse sequence. Based on the technical means, the technical effects of improving the data storage and updating efficiency in the dynamic readout layer, dynamically allocating the moving average value memory, reducing the chip area, dynamically configuring the number of output classes and the like are achieved.

Humans can distinguish fabrics by their textures, even when they are finer than the density of tactile sensors. Evidence suggests that this ability is produced by the nervous system using an active touch strategy. When the finger slides over a texture, the nervous system converts the texture's spatial period into an equivalent spiking frequency. Many studies focused on modeling the biological encoding part that translates the spatial frequency into a temporal spiking frequency, but few explored the decoding part. In this work, we propose a novel approach based on a spiking neural network able to detect the frequency of an input signal. Inspired by biological evidence, our architecture detects the range in which the encoded frequency dwells and could therefore decode the texture's spatial period. The network has been designed to be composed of existing neuromorphic spiking primitives. This property enables a straightforward implementation on integrated silicon circuits, allowing the texture decoding at the edge of the sensor.

Ferroelectric tunneling junctions (FTJ) are considered to be the intrinsically most energy efficient memristors. In this work, specific electrical features of ferroelectric hafnium-zirconium oxide based FTJ devices are investigated. Moreover, the impact on the design of FTJ-based circuits for edge computing applications is discussed by means of two example circuits.

Event-based vision sensors encode local pixel-wise brightness changes in streams of events rather than image frames and yield sparse, energy-efficient encodings of scenes, in addition to low latency, high dynamic range, and lack of motion blur. Recent progress in object recognition from event-based sensors has come from conversions of deep neural networks, trained with backpropagation. However, using these approaches for event streams requires a transformation to a synchronous paradigm, which not only loses computational efficiency, but also misses opportunities to extract spatio-temporal features. In this article we propose a hybrid architecture for end-to-end training of deep neural networks for event-based pattern recognition and object detection, combining a spiking neural network (SNN) backbone for efficient event-based feature extraction, and a subsequent analog neural network (ANN) head to solve synchronous classification and detection tasks. This is achieved by combining standard backpropagation with surrogate gradient training to propagate gradients through the SNN. Hybrid SNN-ANNs can be trained without conversion, and result in highly accurate networks that are substantially more computationally efficient than their ANN counterparts. We demonstrate results on event-based classification and object detection datasets, in which only the architecture of the ANN heads need to be adapted to the tasks, and no conversion of the event-based input is necessary. Since ANNs and SNNs require different hardware paradigms to maximize their efficiency, we envision that SNN backbone and ANN head can be executed on different processing units, and thus analyze the necessary bandwidth to communicate between the two parts. Hybrid networks are promising architectures to further advance machine learning approaches for event-based vision, without having to compromise on efficiency.

Compared to classic robotics, biological nervous systems respond to stimuli in a fast and efficient way regarding the body motor actions. Decision making, once the sensory information arrives to the brain, is in the order of ms, while the whole process from sensing to movement requires tens of ms. Classic robotic systems usually require complex computational abilities. Key differences between biological systems and robotic machines lie in the way information is coded and transmitted. A neuron is the “basic” element that constitutes biological nervous systems. Neurons communicate in an event-driven way through small currents or ionic pulses (spikes). When neurons are arranged in networks, they allow not only for the processing of sensory information, but also for the actuation over the muscles in the same spiking manner. This paper presents the application of a classic motor control model (proportional-integral-derivative) developed with the biological spike processing principle, including the motor actuation with time enlarged spikes instead of the classic pulse-width-modulation. This closed-loop control model, called spike-based PID controller (sPID), was improved and adapted for a dual FPGA-based system to control the four joints of a bioinspired light robot (BioRob X5), called event-driven BioRob (ED-BioRob). The use of spiking signals allowed the system to achieve a current consumption bellow 1A for the entire 4 DoF working at the same time. Furthermore, the robot joints commands can be received from a population of silicon-neurons running on the Dynap-SE platform. Thus, our proposal aims to bridge the gap between a general purpose processing analog neuromorphic hardware and the spiking actuation of a robotic platform. Spanish Ministry of Education and Science / European Regional Development Fund COFNET TEC2016-77785-P Spanish Ministry of Education and Science / European Regional Development Fund MIND-ROB PID2019-105556GBC33 Spanish Ministry of Education and Science / European Regional Development Fund CHIST-ERA project SMALL PCI2019-111841-2

The invention relates to an event-driven spiking convolutional neural network, comprising a plurality of layers, wherein each layer comprises - a kernel module configured to store and to process in an event-driven fashion kernel values of at least one convolution kernel, - a neuron module configured to store and to process in an event-driven fashion neuron states of neurons of the network, and particularly to output spike events generated from the updated neurons, - a memory mapper configured to determine neurons to which an incoming a spike event from a source layer projects to by means of a convolution with the at least one convolution kernel and wherein neuron states of said determined neurons are to be updated with applicable kernel values of the at least one convolution kernel, wherein the memory mapper is configured to process incoming spike events in an event-driven fashion.

Spiking neural networks (SNNs) are potentially highly efficient models for inference on fully parallel neuromorphic hardware, but existing training methods that convert conventional artificial neural networks (ANNs) into SNNs are unable to exploit these advantages. Although ANN-to-SNN conversion has achieved state-of-the-art accuracy for static image classification tasks, the following subtle but important difference in the way SNNs and ANNs integrate information over time makes the direct application of conversion techniques for sequence processing tasks challenging. Whereas all connections in SNNs have a certain propagation delay larger than zero, ANNs assign different roles to feed-forward connections, which immediately update all neurons within the same time step, and recurrent connections, which have to be rolled out in time and are typically assigned a delay of one time step. Here, we present a novel method to obtain highly accurate SNNs for sequence processing by modifying the ANN training before conversion, such that delays induced by ANN rollouts match the propagation delays in the targeted SNN implementation. Our method builds on the recently introduced framework of streaming rollouts, which aims for fully parallel model execution of ANNs and inherently allows for temporal integration by merging paths of different delays between input and output of the network. The resulting networks achieve state-of-the-art accuracy for multiple event-based benchmark datasets, including N-MNIST, CIFAR10-DVS, N-CARS, and DvsGesture, and through the use of spatio-temporal shortcut connections yield low-latency approximate network responses that improve over time as more of the input sequence is processed. In addition, our converted SNNs are consistently more energy-efficient than their corresponding ANNs.

The brain is able to solve complex tasks in real time by combining different sensory cues with previously acquired knowledge. Inspired by the brain, we designed a neuromorphic demonstrator which combines auditory and visual input to find an obstacle free direction closest to the sound source. The system consists of two event-based sensors (the eDVS for vision and the NAS for audition) mounted onto a pan-tilt unit and a spiking neural network implemented on the SpiNNaker platform. By combining the different sensory information, the demonstrator is able to point at a sound source direction while avoiding obstacles in real time.

International audience; During the last years, Deep Neural Networks have reached the highest performances in image classification Nevertheless,such a success is mostly based on supervised and off-line learning: they require thus huge labeled datasets for learning,and once it is done, they cannot adapt to any change in the data from the environment. In the context of brain-inspired computing, we apply Kohonen-based Self-Organizing Maps for unsupervised learning without labels, and we explore original extensions such as the Dynamic SOM that enables continuous learning and the Pruning Cellular SOM that includes synaptic pruning in neuromorphic circuits. After presenting the three models and the experimental setup for MNIST classification, we compare different methods for automatic labeling based onvery few labeled data (1% of the training dataset), and then we compare the performances of the three Kohonen-based Self-Organizing Maps with STDP-based Spiking Neural Networks in terms of accuracy, dynamicity and scalability.

The demonstration shows a mobile application, called "RELAX", for hand gesture classification using multi-sensors fusion. In particular, we integrated the data collected by an electromyography (EMG) sensor with the events produced by an event-based vision sensor, the Dynamic Vision Sensor (DVS). The application runs real-time on any Android smartphone and it is able to recognize five gestures with an accuracy of up to 85%.This demonstration is associated with the track Bio-Inspired and Neuromorphic Circuits and Systems. Associated paper submission identifier: 8114.

Best demo Award; International audience; The discrimination of human gestures using wearable solutions is extremely important as a supporting technique for assisted living, healthcare of the elderly and neurorehabilitation. This paper presents a mobile electromyography (EMG)analysis framework to be an auxiliary component in physiotherapy sessions or as a feedback for neuroprosthesis calibration. We implemented a framework that allows the integration of multisensors, EMG and visual information, to perform sensor fusion and to improve the accuracy of hand gesture recognition tasks. In particular, we used an event-based camera adapted to run on the limited computational resources of mobile phones. We introduced a new publicly available dataset of sensor fusion for hand gesture recognition recorded from 10 subjects and used it to train the recognition models offline. We compare the online results of the hand gesture recognition using the fusion approach with the individual sensors with an improvement in the accuracy of 13% and 11%, for EMG and vision respectively, reaching 85%.

We demonstrate an event-driven Deep Learning (DL) hardware software ecosystem. The user-friendly software tools port models from Keras (popular machine learning libraries), automaticaly convert DL models to Spiking equivalents, i.e. Spiking Convolutional Neural Networks (SCNNs) and run spiking simulations of the converted models on the hardware emulator for testing and prototyping. More importantly, the software ports the converted models onto a novel, ultra-low power, real-time, event-driven ASIC SCNN Chip: DynapCNN. An interactive demonstration of a real-time face recognition system built using the above pipeline is shown as an example.

International audience; Artificial neural networks are experiencing today an unprecedented interest thanks to two main changes: the explosion of open data that is necessary for their training, and the increasing computing power of today's computers that makes the training part possible in a reasonable time. The recent results of deep neural networks on image classification has given neural networks the leading role in machine learning algorithms and artificial intelligence research.However, most applications such as smart devices or autonomous vehicles require an embedded implementation of neural networks. Their implementation in CPU/GPU remains too expensive, mostly in energy consumption, due to the non-adaptation of the hardware to the computation model, which becomes a limit to their use. It is therefore necessary to design neuromorphic architectures, i.e. hardware accelerators that fit to the parallel and distributed computation paradigm of neural networks for reducing their hardware cost implementation. We mainly focus on the optimization of energy consumption to enable integration in embedded systems.For this purpose, we implement two models of artificial neural networks coming from two different scientific domains: the multi-layer perceptron derived from machine learning and the spiking neural network inspired from neuroscience. We compare the performances of both approaches in terms of accuracy and hardware cost to find out the most attractive architecture for the design of embedded artificial intelligence.

We use a large-scale analog neuromorphic system to encode the hidden-layer activations of a single-layer feed forward network with random weights. The random activations of the network are implemented using the device mismatch inherent to analog circuits. We show that these activations produced by analog VLSI implementations of integrate and fire neurons are suited to solve multi dimensional, non linear regression tasks. Exploitation of the device mismatch eliminates the storage requirements for the random network weights.

It contains associated material.-- The 1000 Genomes Project Consortium, [Background] Population differentiation has proved to be effective for identifying loci under geographically localized positive selection, and has the potential to identify loci subject to balancing selection. We have previously investigated the pattern of genetic differentiation among human populations at 36.8 million genomic variants to identify sites in the genome showing high frequency differences. Here, we extend this dataset to include additional variants, survey sites with low levels of differentiation, and evaluate the extent to which highly differentiated sites are likely to result from selective or other processes., [Results] We demonstrate that while sites with low differentiation represent sampling effects rather than balancing selection, sites showing extremely high population differentiation are enriched for positive selection events and that one half may be the result of classic selective sweeps. Among these, we rediscover known examples, where we actually identify the established functional SNP, and discover novel examples including the genes ABCA12, CALD1 and ZNF804, which we speculate may be linked to adaptations in skin, calcium metabolism and defense, respectively., [Conclusions] We identify known and many novel candidate regions for geographically restricted positive selection, and suggest several directions for further research. © 2014 Colonna et al., This work was supported by The Wellcome Trust (098051), an Italian National Research Council (CNR) short-term mobility fellowship from the 2013 program to VC, and an EMBO Short Term Fellowship ASTF 324–2010 to VC.

This paper presents the self-organized neuromorphic architecture named SOMA. The objective is to study neural-based self-organization in computing systems and to prove the feasibility of a self-organizing hardware structure. Considering that these properties emerge from large scale and fully connected neural maps, we will focus on the definition of a self-organizing hardware architecture based on digital spiking neurons that offer hardware efficiency. From a biological point of view, this corresponds to a combination of the so-called synaptic and structural plasticities. We intend to define computational models able to simultaneously self-organize at both computation and communication levels, and we want these models to be hardware-compliant, fault tolerant and scalable by means of a neuro-cellular structure., Comment: Published in IEEE World Congress on Computational Intelligence (WCCI), International Workshop: Neuromorphic Hardware in Practice and Use (NHPU), Jul. 2018, Rio de Janeiro, Brazil