Search

Neuromorphic engineering endeavors to integrate the computational prowess and efficiency inherent in biological neuronal systems, such as the brain, into contemporary technological systems, primarily through the deployment of spiking neural networks. This research delineates the development and implementation of a bio-inspired sequential hippocampus memory model, which can effectively learn and sequentially recall memories, within a robotic infrastructure. The hippocampus memory model, implemented on the SpiNNaker platform, has been tactically utilized to control a 4-joint event-based robot arm, the ED-ScorBot, by learning and then recalling trajectories via a sequence of memories regarding joint positions. The conveyed spiking information from SpiNNaker is interpreted by an FPGA in real-time to command the event-driven motors of the robotic arm, integrating learned trajectories into physical robotic movement. An empirical exploration validates the model’s capability to govern the robotic arm’s trajectory with precision and dependability while simultaneously demonstrating the potential for incorporating spike-based memory models in robotic applications. This synergistic convergence of neuromorphic engineering and robotics illustrates a viable pathway towards sophisticated, efficient, and adaptable robotic systems capable of learning and reproducing complex tasks, with significant implications for future developments in autonomous robotic applications.

Configurable synaptic delays are a basic feature in many neuromorphic neural network hardware accelerators. However, they have been rarely used in model implementations, despite their promising impact on performance and efficiency in tasks that exhibit complex (temporal) dynamics, as it has been unclear how to optimize them. In this work, we propose a framework to train and deploy, in digital neuromorphic hardware, highly performing spiking neural network models (SNNs) where apart from the synaptic weights, the per-synapse delays are also co-optimized. Leveraging spike-based back-propagation-through-time, the training accounts for both platform constraints, such as synaptic weight precision and the total number of parameters per core, as a function of the network size. In addition, a delay pruning technique is used to reduce memory footprint with a low cost in performance. We evaluate trained models in two neuromorphic digital hardware platforms: Intel Loihi and Imec Seneca. Loihi offers synaptic delay support using the so-called Ring-Buffer hardware structure. Seneca does not provide native hardware support for synaptic delays. A second contribution of this paper is therefore a novel area- and memory-efficient hardware structure for acceleration of synaptic delays, which we have integrated in Seneca. The evaluated benchmark involves several models for solving the SHD (Spiking Heidelberg Digits) classification task, where minimal accuracy degradation during the transition from software to hardware is demonstrated. To our knowledge, this is the first work showcasing how to train and deploy hardware-aware models parameterized with synaptic delays, on multicore neuromorphic hardware accelerators.

Prostate cancer is the second most frequent cancer in men worldwide after lung cancer. Its diagnosis is based on the identification of the Gleason score that evaluates the abnormality of cells in glands through the analysis of the different Gleason patterns within tissue samples. The recent advancements in computational pathology, a domain aiming at developing algorithms to automatically analyze digitized histopathology images, lead to a large variety and availability of datasets and algorithms for Gleason grading and scoring. However, there is no clear consensus on which methods are best suited for each problem in relation to the characteristics of data and labels. This paper provides a systematic comparison on nine datasets with state-of-the-art training approaches for deep neural networks (including fully-supervised learning, weakly-supervised learning, semi-supervised learning, Additive-MIL, Attention-Based MIL, Dual-Stream MIL, TransMIL and CLAM) applied to Gleason grading and scoring tasks. The nine datasets are collected from pathology institutes and openly accessible repositories. The results show that the best methods for Gleason grading and Gleason scoring tasks are fully supervised learning and CLAM, respectively, guiding researchers to the best practice to adopt depending on the task to solve and the labels that are available.

With the rise of artificial intelligence, neural network simulations of biological neuron models are being explored to reduce the footprint of learning and inference in resource-constrained task scenarios. A mainstream type of such networks are spiking neural networks (SNNs) based on simplified Integrate and Fire models for which several hardware accelerators have emerged. Among them, the ReckOn chip was introduced as a recurrent SNN allowing for both online training and execution of tasks based on arbitrary sensory modalities, demonstrated for vision, audition, and navigation. As a fully digital and open-source chip, we adapted ReckOn to be implemented on a Xilinx Multiprocessor System on Chip system (MPSoC), facilitating its deployment in embedded systems and increasing the setup flexibility. We present an overview of the system, and a Python framework to use it on a Pynq ZU platform. We validate the architecture and implementation in the new scenario of robotic arm control, and show how the simulated accuracy is preserved with a peak performance of 3.8M events processed per second., Comment: Under review at ICECS'24

Configurable synaptic delays are a basic feature in many neuromorphic neural network hardware accelerators. However, they have been rarely used in model implementations, despite their promising impact on performance and efficiency in tasks that exhibit complex (temporal) dynamics, as it has been unclear how to optimize them. In this work, we propose a framework to train and deploy, in digital neuromorphic hardware, highly performing spiking neural network models (SNNs) where apart from the synaptic weights, the per-synapse delays are also co-optimized. Leveraging spike-based back-propagation-through-time, the training accounts for both platform constraints, such as synaptic weight precision and the total number of parameters per core, as a function of the network size. In addition, a delay pruning technique is used to reduce memory footprint with a low cost in performance. We evaluate trained models in two neuromorphic digital hardware platforms: Intel Loihi and Imec Seneca. Loihi offers synaptic delay support using the so-called Ring-Buffer hardware structure. Seneca does not provide native hardware support for synaptic delays. A second contribution of this paper is therefore a novel area- and memory-efficient hardware structure for acceleration of synaptic delays, which we have integrated in Seneca. The evaluated benchmark involves several models for solving the SHD (Spiking Heidelberg Digits) classification task, where minimal accuracy degradation during the transition from software to hardware is demonstrated. To our knowledge, this is the first work showcasing how to train and deploy hardware-aware models parameterized with synaptic delays, on multicore neuromorphic hardware accelerators.

Analog implementation of Oscillatory Neural Networks (ONNs) has the potential to implement fast and ultra-low-power computing capabilities. One of the drawbacks of analog implementation is component mismatches which cause desynchronization and instability in ONNs. Emerging devices like memristors and VO2 are particularly prone to variations. In this paper, we study the effect of component mismatches on the performance of differential ONNs (DONNs). Mismatches were considered in two main blocks: differential oscillatory neurons and synaptic circuits. To measure DONN tolerance to mismatches in each block, performance was evaluated with mismatches being present separately in each block. Memristor-bridge circuits with four memristors were used as the synaptic circuits. The differential oscillatory neurons were based on VO2 -devices. The simulation results showed that DONN performance was more vulnerable to mismatches in the components of the differential oscillatory neurons than to mismatches in the synaptic circuits. DONNs were found to tolerate up to 20% mismatches in the memristance of the synaptic circuits. However, mismatches in the differential oscillatory neurons resulted in non-uniformity of the natural frequencies, causing desynchronization and instability. Simulations showed that 0.5% relative standard deviation (RSD) in natural frequencies can reduce DONN performance dramatically. In addition, sensitivity analyses showed that the high threshold voltage of VO2-devices is the most sensitive parameter for frequency non-uniformity and desynchronization.

Analog implementation of Oscillatory Neural Networks (ONNs) has the potential to implement fast and ultra-low-power computing capabilities. One of the drawbacks of analog implementation is component mismatches which cause desynchronization and instability in ONNs. Emerging devices like memristors and VO2 are particularly prone to variations. In this paper, we study the effect of component mismatches on the performance of differential ONNs (DONNs). Mismatches were considered in two main blocks: differential oscillatory neurons and synaptic circuits. To measure DONN tolerance to mismatches in each block, performance was evaluated with mismatches being present separately in each block. Memristor-bridge circuits with four memristors were used as the synaptic circuits. The differential oscillatory neurons were based on VO2 -devices. The simulation results showed that DONN performance was more vulnerable to mismatches in the components of the differential oscillatory neurons than to mismatches in the synaptic circuits. DONNs were found to tolerate up to 20% mismatches in the memristance of the synaptic circuits. However, mismatches in the differential oscillatory neurons resulted in non-uniformity of the natural frequencies, causing desynchronization and instability. Simulations showed that 0.5% relative standard deviation (RSD) in natural frequencies can reduce DONN performance dramatically. In addition, sensitivity analyses showed that the high threshold voltage of VO2-devices is the most sensitive parameter for frequency non-uniformity and desynchronization.

This paper presents a sensory fusion neuromorphic dataset collected with precise temporal synchronization using a set of Address-Event-Representation sensors and tools. The target application is the lip reading of several keywords for different machine learning applications, such as digits, robotic commands, and auxiliary rich phonetic short words. The dataset is enlarged with a spiking version of an audio-visual lip reading dataset collected with frame-based cameras. LIPSFUS is publicly available and it has been validated with a deep learning architecture for audio and visual classification. It is intended for sensory fusion architectures based on both artificial and spiking neural network algorithms.

The brain, this 3-pound mass of tissue that can easily be held in one's palm, has an inherent computational complexity that has always inspired efforts to endorse machines with some of its remarkable characteristics. Ironically, the brain computes in its own way, compared to analog or digital computers, despite sharing key concepts with both. It employs analog computation but digital communication, through spikes, both of which improve robustness to noise. This unique combination defines a new computational paradigm that we have just started to explore. The reasons why neuromorphic systems are one of the fastest growing applications are not purely scientific, but mainly technological. For 50 years, the principle guiding computations has been Moore's Law, a macroscopic observation that we will always find ways to engineer faster, smaller and cheaper chips. But there are several reasons why Moore's Law is no longer able to keep up. First, physics: As we downsize transistors close to the atomic scale, it becomes difficult to regulate electron flow. Electrons do not necessarily adhere to Newtonian physics and may pass through the transistor barriers, a phenomenon called quantum tunneling. This makes our computer architectures inefficient. Second, we have long accepted the existence of a trade-off between computing faster and consuming less power, but this has never been a problem until we started approaching the physical limits of fabricating transistors. And the final nail in Moore's Law coffin is put by deep learning. Our computational needs are now orders of magnitude higher than what our systems can deliver. The von-Neumann paradigm is already having more than its fair share of inefficiency—to match its brilliance. And the reason is simple. Computers have been designed with feasibility, not efficiency, at their center. And nowhere are the effects of a bad design more imminent—or the opportunity for an alternative design more compelling – than in emerging technologie

Background: Among all the cancers known today, prostate cancer is one of the most commonly diagnosed in men. With modern advances in medicine, its mortality has been considerably reduced. However, it is still a leading type of cancer in terms of deaths.The diagnosis of prostate cancer is mainly conducted by biopsy test. From this test, Whole Slide Images are obtained, from which pathologists diagnose the cancer according to the Gleason scale. Within this scale from 1 to 5, grade 3and above is considered malignant tissue. Several studies have shown an inter- observer discrepancy between pathologists in assigning the value of the Gleason scale. Due to the recent advances in artificial intelligence, its application to the computational pathology field with the aim of supporting and providing a second opinion to the professional is of great interest. Method:In this work, the inter-observer variability of a local dataset of 80 whole-slide images annotated by a team of 5 pathologists from the same group was analyzed at both area and label level. Four approaches were followed to train six different Convolutional Neural Network architectures, which were evaluated on the same data set on which the inter-observer variability was analyzed. Results: An inter-observer variability of 0.6946k was obtained, with 46% discrepancy in terms of area size of the annotations performed by the pathologists. The best trained models achieved 0.826±0.014k on the test set when trained with data from the same source. Conclusions: The obtained results show that deep learning-based automatic diagnosi s systems could help reduce the widely-known inter-observer variability that is present among pathologists and support them in their decision, serving as a second opinion or as a triage tool for medical centers.

The brain, this 3-pound mass of tissue that can easily be held in one's palm, has an inherent computational complexity that has always inspired efforts to endorse machines with some of its remarkable characteristics. Ironically, the brain computes in its own way, compared to analog or digital computers, despite sharing key concepts with both. It employs analog computation but digital communication, through spikes, both of which improve robustness to noise. This unique combination defines a new computational paradigm that we have just started to explore. The reasons why neuromorphic systems are one of the fastest growing applications are not purely scientific, but mainly technological. For 50 years, the principle guiding computations has been Moore's Law, a macroscopic observation that we will always find ways to engineer faster, smaller and cheaper chips. But there are several reasons why Moore's Law is no longer able to keep up. First, physics: As we downsize transistors close to the atomic scale, it becomes difficult to regulate electron flow. Electrons do not necessarily adhere to Newtonian physics and may pass through the transistor barriers, a phenomenon called quantum tunneling. This makes our computer architectures inefficient. Second, we have long accepted the existence of a trade-off between computing faster and consuming less power, but this has never been a problem until we started approaching the physical limits of fabricating transistors. And the final nail in Moore's Law coffin is put by deep learning. Our computational needs are now orders of magnitude higher than what our systems can deliver. The von-Neumann paradigm is already having more than its fair share of inefficiency—to match its brilliance. And the reason is simple. Computers have been designed with feasibility, not efficiency, at their center. And nowhere are the effects of a bad design more imminent—or the opportunity for an alternative design more compelling – than in emerging technologie

This paper presents a sensory fusion neuromorphic dataset collected with precise temporal synchronization using a set of Address-Event-Representation sensors and tools. The target application is the lip reading of several keywords for different machine learning applications, such as digits, robotic commands, and auxiliary rich phonetic short words. The dataset is enlarged with a spiking version of an audio-visual lip reading dataset collected with frame-based cameras. LIPSFUS is publicly available and it has been validated with a deep learning architecture for audio and visual classification. It is intended for sensory fusion architectures based on both artificial and spiking neural network algorithms.

Analog implementation of Oscillatory Neural Networks (ONNs) has the potential to implement fast and ultra-low-power computing capabilities. One of the drawbacks of analog implementation is component mismatches which cause desynchronization and instability in ONNs. Emerging devices like memristors and VO2 are particularly prone to variations. In this paper, we study the effect of component mismatches on the performance of differential ONNs (DONNs). Mismatches were considered in two main blocks: differential oscillatory neurons and synaptic circuits. To measure DONN tolerance to mismatches in each block, performance was evaluated with mismatches being present separately in each block. Memristor-bridge circuits with four memristors were used as the synaptic circuits. The differential oscillatory neurons were based on VO2 -devices. The simulation results showed that DONN performance was more vulnerable to mismatches in the components of the differential oscillatory neurons than to mismatches in the synaptic circuits. DONNs were found to tolerate up to 20% mismatches in the memristance of the synaptic circuits. However, mismatches in the differential oscillatory neurons resulted in non-uniformity of the natural frequencies, causing desynchronization and instability. Simulations showed that 0.5% relative standard deviation (RSD) in natural frequencies can reduce DONN performance dramatically. In addition, sensitivity analyses showed that the high threshold voltage of VO2-devices is the most sensitive parameter for frequency non-uniformity and desynchronization.

Alternative paradigms to the von Neumann computing scheme are currently arousing huge interest. Oscillatory neural networks (ONNs) using emerging phase-change materials like VO2 constitute an energy-efficient, massively parallel, brain-inspired, in-memory computing approach. The encoding of information in the phase pattern of frequency-locked, weakly coupled oscillators makes it possible to exploit their rich non-linear dynamics and their synchronization phenomena for computing. A single fully connected ONN layer can implement an auto-associative memory comparable to that of a Hopfield network, hence Hebbian learning rule is the most widely adopted method for configuring ONNs for such applications, despite its well-known limitations. An extensive amount of literature is available about learning in Hopfield networks, with information regarding many different learning algorithms that perform better than the Hebbian rule. However, not all of these algorithms are useful for ONN training due to the constraints imposed by their physical implementation. This paper evaluates different learning methods with respect to their suitability for ONNs. It proposes a new approach, which is compared against previous works. The proposed method has been shown to produce competitive results in terms of pattern recognition accuracy with reduced precision in synaptic weights, and to be suitable for online learning.

Background: Among all the cancers known today, prostate cancer is one of the most commonly diagnosed in men. With modern advances in medicine, its mortality has been considerably reduced. However, it is still a leading type of cancer in terms of deaths.The diagnosis of prostate cancer is mainly conducted by biopsy test. From this test, Whole Slide Images are obtained, from which pathologists diagnose the cancer according to the Gleason scale. Within this scale from 1 to 5, grade 3and above is considered malignant tissue. Several studies have shown an inter- observer discrepancy between pathologists in assigning the value of the Gleason scale. Due to the recent advances in artificial intelligence, its application to the computational pathology field with the aim of supporting and providing a second opinion to the professional is of great interest. Method:In this work, the inter-observer variability of a local dataset of 80 whole-slide images annotated by a team of 5 pathologists from the same group was analyzed at both area and label level. Four approaches were followed to train six different Convolutional Neural Network architectures, which were evaluated on the same data set on which the inter-observer variability was analyzed. Results: An inter-observer variability of 0.6946k was obtained, with 46% discrepancy in terms of area size of the annotations performed by the pathologists. The best trained models achieved 0.826±0.014k on the test set when trained with data from the same source. Conclusions: The obtained results show that deep learning-based automatic diagnosi s systems could help reduce the widely-known inter-observer variability that is present among pathologists and support them in their decision, serving as a second opinion or as a triage tool for medical centers.

The brain, this 3-pound mass of tissue that can easily be held in one's palm, has an inherent computational complexity that has always inspired efforts to endorse machines with some of its remarkable characteristics. Ironically, the brain computes in its own way, compared to analog or digital computers, despite sharing key concepts with both. It employs analog computation but digital communication, through spikes, both of which improve robustness to noise. This unique combination defines a new computational paradigm that we have just started to explore. The reasons why neuromorphic systems are one of the fastest growing applications are not purely scientific, but mainly technological. For 50 years, the principle guiding computations has been Moore's Law, a macroscopic observation that we will always find ways to engineer faster, smaller and cheaper chips. But there are several reasons why Moore's Law is no longer able to keep up. First, physics: As we downsize transistors close to the atomic scale, it becomes difficult to regulate electron flow. Electrons do not necessarily adhere to Newtonian physics and may pass through the transistor barriers, a phenomenon called quantum tunneling. This makes our computer architectures inefficient. Second, we have long accepted the existence of a trade-off between computing faster and consuming less power, but this has never been a problem until we started approaching the physical limits of fabricating transistors. And the final nail in Moore's Law coffin is put by deep learning. Our computational needs are now orders of magnitude higher than what our systems can deliver. The von-Neumann paradigm is already having more than its fair share of inefficiency—to match its brilliance. And the reason is simple. Computers have been designed with feasibility, not efficiency, at their center. And nowhere are the effects of a bad design more imminent—or the opportunity for an alternative design more compelling – than in emerging technologie

Oscillatory neural networks (ONNs) exhibit a high potential for energy-efficient computing. In ONNs, neurons are implemented with oscillators and synapses with resistive and/or capacitive coupling between pairs of oscillators. Computing is carried out on the basis of the rich, complex, non-linear synchronization dynamics of a system of coupled oscillators. The exploited synchronization phenomena in ONNs are an example of fully parallel collective computing. A fast system’s convergence to stable states, which correspond to the desired processed information, enables an energy-efficient solution if small area and low-power oscillators are used, specifically when they are built on the basis of the hysteresis exhibited by phase-transition materials such as VO2. In recent years, there have been numerous studies on ONNs using VO2. Most of them report simulation results. Although in some cases experimental results are also shown, they do not implement the design techniques that other works on electrical simulations report that allow to improve the behavior of the ONNs. Experimental validation of these approaches is necessary. Therefore, in this study, we describe an ONN realized in a commercial CMOS technology in which the oscillators are built using a circuit that we have developed to emulate the VO2 device. The purpose is to be able to study in-depth the synchronization dynamics of relaxation oscillators similar to those that can be performed with VO2 devices. The fabricated circuit is very flexible. It allows programming the synapses to implement different ONNs, calibrating the frequency of the oscillators, or controlling their initialization. It uses differential oscillators and resistive synapses, equivalent to the use of memristors. In this article, the designed and fabricated circuits are described in detail, and experimental results are shown. Specifically, its satisfactory operation as an associative memory is demonstrated. The experiments carried out allow us to c

This paper presents a sensory fusion neuromorphic dataset collected with precise temporal synchronization using a set of Address-Event-Representation sensors and tools. The target application is the lip reading of several keywords for different machine learning applications, such as digits, robotic commands, and auxiliary rich phonetic short words. The dataset is enlarged with a spiking version of an audio-visual lip reading dataset collected with frame-based cameras. LIPSFUS is publicly available and it has been validated with a deep learning architecture for audio and visual classification. It is intended for sensory fusion architectures based on both artificial and spiking neural network algorithms.

Spin-Hall-effect nano-oscillators are promising beyond the CMOS devices currently available, and can potentially be used to emulate the functioning of neurons in computational neuromorphic systems. As they oscillate in the 4–20 GHz range, they could potentially be used for building highly accelerated neural hardware platforms. However, due to their extremely low signal level and high impedance at their output, as well as their microwave-range operating frequency, discerning whether the SHNO is oscillating or not carries a great challenge when its state read-out circuit is implemented using CMOS technologies. This paper presents the first CMOS front-end read-out circuitry, implemented in 180 nm, working at a SHNO oscillation frequency up to 4.7 GHz, managing to discern SHNO amplitudes of 100 µV even for an impedance as large as 300 Ω and a noise figure of 5.3 dB300 Ω. A design flow of this front end is presented, as well as the architecture of each of its blocks. The study of the low-noise amplifier is deepened for its intrinsic difficulties in the design, satisfying the characteristics of SHNOs.

Analog implementation of Oscillatory Neural Networks (ONNs) has the potential to implement fast and ultra-low-power computing capabilities. One of the drawbacks of analog implementation is component mismatches which cause desynchronization and instability in ONNs. Emerging devices like memristors and VO2are particularly prone to variations. In this paper, we study the effect of component mismatches on the performance of differential ONNs (DONNs). Mismatches were considered in two main blocks: differential oscillatory neurons and synaptic circuits. To measure DONN tolerance to mismatches in each block, performance was evaluated with mismatches being present separately in each block. Memristor-bridge circuits with four memristors were used as the synaptic circuits. The differential oscillatory neurons were based on VO2-devices. The simulation results showed that DONN performance was more vulnerable to mismatches in the components of the differential oscillatory neurons than to mismatches in the synaptic circuits. DONNs were found to tolerate up to 20% mismatches in the memristance of the synaptic circuits. However, mismatches in the differential oscillatory neurons resulted in non-uniformity of the natural frequencies, causing desynchronization and instability. Simulations showed that 0.5% relative standard deviation (RSD) in natural frequencies can reduce DONN performance dramatically. In addition, sensitivity analyses showed that the high threshold voltage of VO2-devices is the most sensitive parameter for frequency non-uniformity and desynchronization.

For neuromorphic engineering to emulate the human brain, improving memory density with low power consumption is an indispensable but challenging goal. In this regard, emerging RRAMs have attracted considerable interest for their unique qualities like low power consumption, high integration potential, durability, and CMOS compatibility. Using RRAMs to imitate the more analog storage behavior of brain synapses is also a promising strategy for further improving memory density and power efficiency. However, RRAM devices display strong stochastic behavior, together with relaxation effects, making it more challenging to precisely control their multi-level storage capability. To address this, researchers have reported different multi-level programming strategies, mostly involving the precise control of analog parameters like compliance current during write operations and/or programming voltage amplitudes. Here, we present a new fully digital relaxation-aware method for tuning the conductance of analog RRAMs. The method is based on modulating digital pulse widths during erase operations while keeping other parameters fixed, and therefore requires no precise alterations to analog parameters like compliance currents or programming voltage amplitudes. Experimental results, with and without relaxation effect awareness, on a 64 RRAM 1T1R HfOx memory array of cells, fabricated in 130nm CMOS technology, indicate that it is possible to obtain 2-bit memory per cell multi-value storage at the array level, verified 1000 seconds after programming., Comment: 5 pages, 10 figures, 2 tables

In-camera event denoising reduces the data rate of event cameras by filtering out noise at the source. A lightweight multilayer perceptron denoising filter (MLPF) provides state-of-the-art low-cost denoising accuracy. It processes a small neighborhood of pixels from the timestamp image around each event to discriminate signal and noise events. This paper proposes two digital logic implementations of the MLPF denoiser and quantifies their resource cost, power, and latency. The hardware MLPF quantizes the weights and hidden unit activations to 4 bits and has about 1k weights with about 40% sparsity. The Area-Under-Curve Receiver Operating Characteristic accuracy is nearly indistinguishable from that of the floating point network. The FPGA MLPF processes each event in 10 clock cycles. In FPGA, it uses 3.5k flip flops and 11.5k LUTs. Our ASIC implementation in 65nm digital technology for a 346x260 pixel camera occupies an area of 4.3mm^2 and consumes 4nJ of energy per event at event rates up to 25MHz. The MLPF can be easily integrated into an event camera using an FPGA or as an ASIC directly on the camera chip or in the same package. This denoising could dramatically reduce the energy consumed by the communication and host processor and open new areas of always-on event camera application under scavenged and battery power. Code: https://github.com/SensorsINI/dnd_hls, Comment: Accepted to 2023 CVPRW Workshop on Event-Based Vision

This paper presents a sensory fusion neuromorphic dataset collected with precise temporal synchronization using a set of Address-Event-Representation sensors and tools. The target application is the lip reading of several keywords for different machine learning applications, such as digits, robotic commands, and auxiliary rich phonetic short words. The dataset is enlarged with a spiking version of an audio-visual lip reading dataset collected with frame-based cameras. LIPSFUS is publicly available and it has been validated with a deep learning architecture for audio and visual classification. It is intended for sensory fusion architectures based on both artificial and spiking neural network algorithms., Comment: Submitted to ISCAS2023, 4 pages, plus references, github link provided

[ES] Sensor de visión dinámico foveado electrónicamente. El sensor de visión comprende una matriz (1) de píxeles (5) sobre los que se aplica un mecanismo de foveación, definiendo una serie de regiones de baja resolución de píxeles agrupados (macro-píxeles), de manera que trabajan como un único pixel (5) aislado, obteniéndose información del grupo de píxeles (5) y no de cada pixel (5) de manera individual. Gracias a las regiones de baja resolución de macro-pixeles se logra un ahorro de energía y ancho de banda de datos en favor de las regiones de alta resolución no agrupadas o foveadas. Las regiones de píxeles agrupados son configurables con señales electrónicas externas. Además, se pueden activar electrónicamente múltiples regiones de alta resolución o foveación y tamaños de región., [EN] The present invention relates to a vision sensor comprising a matrix (1) of pixels (5) on which a foveation mechanism is used, defining a series of low resolution regions of grouped pixels (macro-pixels) such that they operate as a single isolated pixel (5), information being obtained from the groups of pixels (5) and not from each pixel (5) individually. Due to the low resolution regions of macro-pixels, energy and data bandwidth savings are achieved in favour of the high resolution regions that are not grouped or foveated. The regions of grouped pixels can be configured with external electronic signals. In addition, multiple high resolution or foveation regions as well as region sizes can be electronically activated.

Oscillatory neural networks (ONNs) exhibit a high potential for energy-efficient computing. In ONNs, neurons are implemented with oscillators and synapses with resistive and/or capacitive coupling between pairs of oscillators. Computing is carried out on the basis of the rich, complex, non-linear synchronization dynamics of a system of coupled oscillators. The exploited synchronization phenomena in ONNs are an example of fully parallel collective computing. A fast system's convergence to stable states, which correspond to the desired processed information, enables an energy-efficient solution if small area and low-power oscillators are used, specifically when they are built on the basis of the hysteresis exhibited by phase-transition materials such as VO2. In recent years, there have been numerous studies on ONNs using VO2. Most of them report simulation results. Although in some cases experimental results are also shown, they do not implement the design techniques that other works on electrical simulations report that allow to improve the behavior of the ONNs. Experimental validation of these approaches is necessary. Therefore, in this study, we describe an ONN realized in a commercial CMOS technology in which the oscillators are built using a circuit that we have developed to emulate the VO2 device. The purpose is to be able to study in-depth the synchronization dynamics of relaxation oscillators similar to those that can be performed with VO2 devices. The fabricated circuit is very flexible. It allows programming the synapses to implement different ONNs, calibrating the frequency of the oscillators, or controlling their initialization. It uses differential oscillators and resistive synapses, equivalent to the use of memristors. In this article, the designed and fabricated circuits are described in detail, and experimental results are shown. Specifically, its satisfactory operation as an associative memory is demonstrated. The experiments carried out allow us to c

Alternative paradigms to the von Neumann computing scheme are currently arousing huge interest. Oscillatory neural networks (ONNs) using emerging phase-change materials like VO2 constitute an energy-efficient, massively parallel, brain-inspired, in-memory computing approach. The encoding of information in the phase pattern of frequency-locked, weakly coupled oscillators makes it possible to exploit their rich non-linear dynamics and their synchronization phenomena for computing. A single fully connected ONN layer can implement an auto-associative memory comparable to that of a Hopfield network, hence Hebbian learning rule is the most widely adopted method for configuring ONNs for such applications, despite its well-known limitations. An extensive amount of literature is available about learning in Hopfield networks, with information regarding many different learning algorithms that perform better than the Hebbian rule. However, not all of these algorithms are useful for ONN training due to the constraints imposed by their physical implementation. This paper evaluates different learning methods with respect to their suitability for ONNs. It proposes a new approach, which is compared against previous works. The proposed method has been shown to produce competitive results in terms of pattern recognition accuracy with reduced precision in synaptic weights, and to be suitable for online learning.

Coupled nano-oscillators are attracting increasing interest because of their potential to perform computation efficiently, enabling new applications in computing and information processing. The potential of phase transition devices for such dynamical systems has recently been recognized. This paper investigates the implementation of coupled VO2-based oscillator networks to solve combinatorial optimization problems. The target problem is mapped to an Ising model, which is solved by the synchronization dynamics of the system. Different factors that impact the probability of the system reaching the ground state of the Ising Hamiltonian and, therefore, the optimum solution to the corresponding optimization problem, are analyzed. The simulation-based analysis has led to the proposal of a novel Second-Harmonic Injection Locking (SHIL) schedule. Its main feature is that SHIL signal amplitude is repeatedly smoothly increased and decreased. Reducing SHIL strength is the mechanism that enables escaping from local minimum energy states. Our experiments show better results in terms of success probability than previously reported approaches. An experimental Oscillatory Ising Machine (OIM) has been built to validate our proposal.

[EN] The present invention relates to a vision sensor comprising a matrix (1) of pixels (5) on which a foveation mechanism is used, defining a series of low resolution regions of grouped pixels (macro-pixels) such that they operate as a single isolated pixel (5), information being obtained from the groups of pixels (5) and not from each pixel (5) individually. Due to the low resolution regions of macro-pixels, energy and data bandwidth savings are achieved in favour of the high resolution regions that are not grouped or foveated. The regions of grouped pixels can be configured with external electronic signals. In addition, multiple high resolution or foveation regions as well as region sizes can be electronically activated., [ES] Sensor de visión dinámico foveado electrónicamente. El sensor de visión comprende una matriz (1) de píxeles (5) sobre los que se aplica un mecanismo de foveación, definiendo una serie de regiones de baja resolución de píxeles agrupados (macro-píxeles), de manera que trabajan como un único pixel (5) aislado, obteniéndose información del grupo de píxeles (5) y no de cada pixel (5) de manera individual. Gracias a las regiones de baja resolución de macro-pixeles se logra un ahorro de energía y ancho de banda de datos en favor de las regiones de alta resolución no agrupadas o foveadas. Las regiones de píxeles agrupados son configurables con señales electrónicas externas. Además, se pueden activar electrónicamente múltiples regiones de alta resolución o foveación y tamaños de región., [FR] Le capteur de vision comprend une matrice (1) de pixels (5) sur lesquels est appliqué un mécanisme de fovéation, définissant une série de régions de basse résolution de pixels regroupés (macro-pixels), de sorte qu'ils travaillent comme un seul pixel (5) isolé, ce qui permet d'obtenir des informations du groupe de pixels (5) et non de chaque pixel (5) de manière individuelle. Grâce aux régions de basse résolution de micro-pixels, on obtient une économie d'énergie et de largeur de bande de données en faveur des régions de haute résolution non regroupées ou fovéées. Les régions de pixels regroupés sont configurables avec des signaux électroniques externes. En outre, de multiples régions de haute résolution ou fovéation et de tailles de région multiples peuvent être activées électroniquement.

Sound source localization (SSL) is used in various applications such as industrial noise-control, speech detection in mobile phones, speech enhancement in hearing aids and many more. Newest video conferencing setups use SSL. The position of a speaker is detected from the difference in the audio waves received by a microphone array. After detection the camera focuses onto the location of the speaker. The human brain is also able to detect the location of a speaker from auditory signals. It uses, among other cues, the difference in amplitude and arrival time of the sound wave at the two ears, called interaural level and time difference. However, the substrate and computational primitives of our brain are different from classical digital computing. Due to its low power consumption of around 20 W and its performance in real time the human brain has become a great source of inspiration for emerging technologies. One of these technologies is neuromorphic hardware which implements the fundamental principles of brain computing identified until today using complementary metal-oxide-semiconductor technologies and new devices. In this work we propose the first neuromorphic closed-loop robotic system that uses the interaural time difference for SSL in real time. Our system can successfully locate sound sources such as human speech. In a closed-loop experiment, the robotic platform turned immediately into the direction of the sound source with a turning velocity linearly proportional to the angle difference between sound source and binaural microphones. After this initial turn, the robotic platform remains at the direction of the sound source. Even though the system only uses very few resources of the available hardware, consumes around 1 W, and was only tuned by hand, meaning it does not contain any learning at all, it already reaches performances comparable to other neuromorphic approaches. The SSL system presented in this article brings us one step closer towards neuromorphic

Muscles are stretched with bursts of spikes that come from motor neurons connected to the cerebellum through the spinal cord. Then, alpha motor neurons directly innervate the muscles to complete the motor command coming from upper biological structures. Nevertheless, classical robotic systems usually require complex computational capabilities and relative high-power consumption to process their control algorithm, which requires information from the robot’s proprioceptive sensors. The way in which the information is encoded and transmitted is an important difference between biological systems and robotic machines. Neuromorphic engineering mimics these behaviors found in biology into engineering solutions to produce more efficient systems and for a better understanding of neural systems. This paper presents the application of a Spike-based Proportional-Integral-Derivative controller to a 6-DoF Scorbot ER-VII robotic arm, feeding the motors with Pulse-Frequency-Modulation instead of Pulse-Width-Modulation, mimicking the way in which motor neurons act over muscles. The presented frameworks allow the robot to be commanded and monitored locally or remotely from both a Python software running on a computer or from a spike-based neuromorphic hardware. Multi-FPGA and single-PSoC solutions are compared. These frameworks are intended for experimental use of the neuromorphic community as a testbed platform and for dataset recording for machine learning purposes.

In-camera event denoising reduces the data rate of event cameras by filtering out noise at the source. A lightweight multilayer perceptron denoising filter (MLPF) provides state-of-the-art low-cost denoising accuracy. It processes a small neighborhood of pixels from the timestamp image around each event to discriminate signal and noise events. This paper proposes two digital logic implementations of the MLPF denoiser and quantifies their resource cost, power, and latency. The hardware MLPF quantizes the weights and hidden unit activations to 4 bits and has about 1k weights with about 40% sparsity. The Area-Under-Curve Receiver Operating Characteristic accuracy is nearly indistinguishable from that of the floating point network. The FPGA MLPF processes each event in 10 clock cycles. In FPGA, it uses 3.5k flip flops and 11.5k LUTs. Our ASIC implementation in 65nm digital technology for a 346 × 260 pixel camera occupies an area of 4.3mm 2 and consumes 4nJ of energy per event at event rates up to 25MHz. The MLPF can be easily integrated into an event camera using an FPGA or as an ASIC directly on the camera chip or in the same package. This denoising could dramatically reduce the energy consumed by the communication and host processor and open new areas of always-on event camera application under scavenged and battery power.Code: https://github.com/SensorsINI/dnd_hls

Sound source localization (SSL) is used in various applications such as industrial noise-control, speech detection in mobile phones, speech enhancement in hearing aids and many more. Newest video conferencing setups use SSL. The position of a speaker is detected from the difference in the audio waves received by a microphone array. After detection the camera focuses onto the location of the speaker. The human brain is also able to detect the location of a speaker from auditory signals. It uses, among other cues, the difference in amplitude and arrival time of the sound wave at the two ears, called interaural level and time difference. However, the substrate and computational primitives of our brain are different from classical digital computing. Due to its low power consumption of around 20 W and its performance in real time the human brain has become a great source of inspiration for emerging technologies. One of these technologies is neuromorphic hardware which implements the fundamental principles of brain computing identified until today using complementary metal-oxide-semiconductor technologies and new devices. In this work we propose the first neuromorphic closed-loop robotic system that uses the interaural time difference for SSL in real time. Our system can successfully locate sound sources such as human speech. In a closed-loop experiment, the robotic platform turned immediately into the direction of the sound source with a turning velocity linearly proportional to the angle difference between sound source and binaural microphones. After this initial turn, the robotic platform remains at the direction of the sound source. Even though the system only uses very few resources of the available hardware, consumes around 1 W, and was only tuned by hand, meaning it does not contain any learning at all, it already reaches performances comparable to other neuromorphic approaches. The SSL system presented in this article brings us one step closer towards neuromorphic

Muscles are stretched with bursts of spikes that come from motor neurons connected to the cerebellum through the spinal cord. Then, alpha motor neurons directly innervate the muscles to complete the motor command coming from upper biological structures. Nevertheless, classical robotic systems usually require complex computational capabilities and relative high-power consumption to process their control algorithm, which requires information from the robot’s proprioceptive sensors. The way in which the information is encoded and transmitted is an important difference between biological systems and robotic machines. Neuromorphic engineering mimics these behaviors found in biology into engineering solutions to produce more efficient systems and for a better understanding of neural systems. This paper presents the application of a Spike-based Proportional-Integral-Derivative controller to a 6-DoF Scorbot ER-VII robotic arm, feeding the motors with Pulse-Frequency-Modulation instead of Pulse-Width-Modulation, mimicking the way in which motor neurons act over muscles. The presented frameworks allow the robot to be commanded and monitored locally or remotely from both a Python software running on a computer or from a spike-based neuromorphic hardware. Multi-FPGA and single-PSoC solutions are compared. These frameworks are intended for experimental use of the neuromorphic community as a testbed platform and for dataset recording for machine learning purposes.

The role of axonal synaptic delays in the efficacy and performance of artificial neural networks has been largely unexplored. In step-based analog-valued neural network models (ANNs), the concept is almost absent. In their spiking neuroscience-inspired counterparts, there is hardly a systematic account of their effects on model performance in terms of accuracy and number of synaptic operations.This paper proposes a methodology for accounting for axonal delays in the training loop of deep Spiking Neural Networks (SNNs), intending to efficiently solve machine learning tasks on data with rich temporal dependencies. We then conduct an empirical study of the effects of axonal delays on model performance during inference for the Adding task, a benchmark for sequential regression, and for the Spiking Heidelberg Digits dataset (SHD), commonly used for evaluating event-driven models. Quantitative results on the SHD show that SNNs incorporating axonal delays instead of explicit recurrent synapses achieve state-of-the-art, over 90% test accuracy while needing less than half trainable synapses. Additionally, we estimate the required memory in terms of total parameters and energy consumption of accomodating such delay-trained models on a modern neuromorphic accelerator. These estimations are based on the number of synaptic operations and the reference GF-22nm FDX CMOS technology. As a result, we demonstrate that a reduced parameterization, which incorporates axonal delays, leads to approximately 90% energy and memory reduction in digital hardware implementations for a similar performance in the aforementioned task.

In this paper we present a memristor-inspired computational method for obtaining a type of running spectrogram or fingerprint of epileptiform activity generated by rodent hippocampal spheroids. It can be used to compute on the fly and with low computational cost an alert-level signal for epileptiform events onset. Here, we describe the computational method behind this fingerprint technique and illustrate it using epileptiform events recorded from hippocampal spheroids using a microelectrode array system., Comment: published in 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS)

Hearing can be considered as one of the most important senses since it plays a key role in the audiovisual learning process. While a lot of effort has been made for achieving good results from the traditional approach of auditory perception, new trends such as neuromorphic computing are showing promising achievements in the implementation of brain structures for sensory perception. In this work, the design and integration of a neuromorphic event based digital model of the auditory ascending pathway within the iCub robotic platform is proposed. This model, which comprises from the cochlea up to the inferior colliculus, replaces the traditional approach for sound processing already implemented on iCub and is able to perform sound recognition and spatial localization in real time, allowing the implementation of auditory attention models in complex scenarios, like the cocktail party problem

Brain-inspired computing employs devices and architectures that emulate biological functions for more adaptive and energy-efficient systems. Oscillatory neural networks (ONNs) are an alternative approach in emulating biological functions of the human brain and are suitable for solving large and complex associative problems. In this work, we investigate the dynamics of coupled oscillators to implement such ONNs. By harnessing the complex dynamics of coupled oscillatory systems, we forge a novel computation model—information is encoded in the phase of oscillations. Coupled interconnected oscillators can exhibit various behaviors due to the strength of the coupling. In this article, we present a novel method based on subharmonic injection locking (SHIL) for controlling the oscillatory states of coupled oscillators that allow them to lock in frequency with distinct phase differences. Circuit-level simulation results indicate SHIL effectiveness and its applicability to large-scale oscillatory networks for pattern recognition.

Neuromorphic systems are a viable alternative to conventional systems for real-time tasks with constrained resources. Their low power consumption, compact hardware realization, and low-latency response characteristics are the key ingredients of such systems. Furthermore, the event-based signal processing approach can be exploited for reducing the computational load and avoiding data loss due to its inherently sparse representation of sensed data and adaptive sampling time. In event-based systems, the information is commonly coded by the number of spikes within a specific temporal window. However, the temporal information of event-based signals can be difficult to extract when using rate coding. In this work, we present a novel digital implementation of the model, called time difference encoder (TDE), for temporal encoding on event-based signals, which translates the time difference between two consecutive input events into a burst of output events. The number of output events along with the time between them encodes the temporal information. The proposed model has been implemented as a digital circuit with a configurable time constant, allowing it to be used in a wide range of sensing tasks that require the encoding of the time difference between events, such as optical flow-based obstacle avoidance, sound source localization, and gas source localization. This proposed bioinspired model offers an alternative to the Jeffress model for the interaural time difference estimation, which is validated in this work with a sound source lateralization proof-of-concept system. The model was simulated and implemented on a field-programmable gate array (FPGA), requiring 122 slice registers of hardware resources and less than 1 mW of power consumption.

Liquid State Machines (LSMs) are computing reservoirs composed of recurrently connected Spiking Neural Networks which have attracted research interest for their modeling capacity of biological structures and as promising pattern recognition tools suitable for their implementation in neuromorphic processors, benefited from the modest use of computing resources in their training process. However, it has been difficult to optimize LSMs for solving complex tasks such as event-based computer vision and few implementations in large-scale neuromorphic processors have been attempted. In this work, we show that offline-trained LSMs implemented in the SpiNNaker neuromorphic processor are able to classify visual events, achieving state-of-the-art performance in the event-based N-MNIST dataset. The training of the readout layer is performed using a recent adaptation of back-propagation-through-time (BPTT) for SNNs, while the internal weights of the reservoir are kept static. Results show that mapping our LSM from a Deep Learning framework to SpiNNaker does not affect the performance of the classification task. Additionally, we show that weight quantization, which substantially reduces the memory footprint of the LSM, has a small impact on its performance.

Neuromorphic systems are a viable alternative to conventional systems for real-time tasks with constrained resources. Their low power consumption, compact hardware realization, and low-latency response characteristics are the key ingredients of such systems. Furthermore, the event-based signal processing approach can be exploited for reducing the computational load and avoiding data loss due to its inherently sparse representation of sensed data and adaptive sampling time. In event-based systems, the information is commonly coded by the number of spikes within a specific temporal window. However, the temporal information of event-based signals can be difficult to extract when using rate coding. In this work, we present a novel digital implementation of the model, called time difference encoder (TDE), for temporal encoding on event-based signals, which translates the time difference between two consecutive input events into a burst of output events. The number of output events along with the time between them encodes the temporal information. The proposed model has been implemented as a digital circuit with a configurable time constant, allowing it to be used in a wide range of sensing tasks that require the encoding of the time difference between events, such as optical flow-based obstacle avoidance, sound source localization, and gas source localization. This proposed bioinspired model offers an alternative to the Jeffress model for the interaural time difference estimation, which is validated in this work with a sound source lateralization proof-of-concept system. The model was simulated and implemented on a field-programmable gate array (FPGA), requiring 122 slice registers of hardware resources and less than 1 mW of power consumption.

Liquid State Machines (LSMs) are computing reservoirs composed of recurrently connected Spiking Neural Networks which have attracted research interest for their modeling capacity of biological structures and as promising pattern recognition tools suitable for their implementation in neuromorphic processors, benefited from the modest use of computing resources in their training process. However, it has been difficult to optimize LSMs for solving complex tasks such as event-based computer vision and few implementations in large-scale neuromorphic processors have been attempted. In this work, we show that offline-trained LSMs implemented in the SpiNNaker neuromorphic processor are able to classify visual events, achieving state-of-the-art performance in the event-based N-MNIST dataset. The training of the readout layer is performed using a recent adaptation of back-propagation-through-time (BPTT) for SNNs, while the internal weights of the reservoir are kept static. Results show that mapping our LSM from a Deep Learning framework to SpiNNaker does not affect the performance of the classification task. Additionally, we show that weight quantization, which substantially reduces the memory footprint of the LSM, has a small impact on its performance.

En este proyecto se trata el control automático y eficaz de un sistema de riego. Para ello, se ha consultado los tipos de sistema de riegos más comunes utilizados en la actualidad, siendo el sistema de riego por goteo uno de los más extendidos y propicio a este tipo de control. Para realizar este control del riego se ha decido usar una potente placa controladora stm32 que permite la toma de información del exterior por sensores, administrarla y gestionarla eficientemente por una programación de multitarea en tiempo real. Todo esto en conjunto con una interfaz amigable para el usuario permite configurar fácilmente programas de riego y mostrar información actualizada de la situación del riego., This project deals with the automatic and efficient control of an irrigation system. To do this, we have consulted the most common types of irrigation systems used today, being the drip irrigation system one of the most widespread and conducive to this type of control. To carry out this irrigation control it has been decided to use a powerful stm32 controller board that allows the collection of information from the outside by sensors, administer and manage it efficiently by a multitasking programming in real time. All this in conjunction with a user friendly interface allows to easily configure irrigation schedules and display updated information of the irrigation situation.

In this work, it is intended to advance on the development of the neuromorphic audio processing systems in robots through the implementation of an open-source neuromorphic cochlea, event-based models of primary auditory nuclei, and their potential use for real-time robotics applications. First, the main gaps when working with neuromorphic cochleae were identified. Among them, the accessibility and usability of such sensors can be considered as a critical aspect. Silicon cochleae could not be as flexible as desired for some applications. However, FPGA-based sensors can be considered as an alternative for fast prototyping and proof-of-concept applications. Therefore, a software tool was implemented for generating open-source, user-configurable Neuromorphic Auditory Sensor models that can be deployed in any FPGA, removing the aforementioned barriers for the neuromorphic research community. Next, the biological principles of the animals' auditory system were studied with the aim of continuing the development of the Neuromorphic Auditory Sensor. More specifically, the principles of binaural hearing were deeply studied for implementing event-based models to perform real-time sound source localization tasks. Two different approaches were followed to extract inter-aural time differences from event-based auditory signals. On the one hand, a digital, event-based design of the Jeffress model was implemented. On the other hand, a novel digital implementation of the Time Difference Encoder model was designed and implemented on FPGA. Finally, three different robotic platforms were used for evaluating the performance of the proposed real-time neuromorphic audio processing architectures. An audio-guided central pattern generator was used to control a hexapod robot in real-time using spiking neural networks on SpiNNaker. Then, a sensory integration application was implemented combining sound source localization and obstacle avoidance for autonomous robots navigation. Lastly, the Neu, En este trabajo se pretende avanzar en el desarrollo de los sistemas de procesamiento de audio neuromórficos en robots a través de la implementación de una cóclea neuromórfica de código abierto, modelos basados en eventos de los núcleos auditivos primarios, y su potencial uso para aplicaciones de robótica en tiempo real. En primer lugar, se identificaron los principales problemas a la hora de trabajar con cócleas neuromórficas. Entre ellos, la accesibilidad y usabilidad de dichos sensores puede considerarse un aspecto crítico. Los circuitos integrados analógicos que implementan modelos cocleares pueden no pueden ser tan flexibles como se desea para algunas aplicaciones específicas. Sin embargo, los sensores basados en FPGA pueden considerarse una alternativa para el desarrollo rápido y flexible de prototipos y aplicaciones de prueba de concepto. Por lo tanto, en este trabajo se implementó una herramienta de software para generar modelos de sensores auditivos neuromórficos de código abierto y configurables por el usuario, que pueden desplegarse en cualquier FPGA, eliminando las barreras mencionadas para la comunidad de investigación neuromórfica. A continuación, se estudiaron los principios biológicos del sistema auditivo de los animales con el objetivo de continuar con el desarrollo del Sensor Auditivo Neuromórfico (NAS). Más concretamente, se estudiaron en profundidad los principios de la audición binaural con el fin de implementar modelos basados en eventos para realizar tareas de localización de fuentes sonoras en tiempo real. Se siguieron dos enfoques diferentes para extraer las diferencias temporales interaurales de las señales auditivas basadas en eventos. Por un lado, se implementó un diseño digital basado en eventos del modelo Jeffress. Por otro lado, se diseñó una novedosa implementación digital del modelo de codificador de diferencias temporales y se implementó en FPGA. Por último, se utilizaron tres plataformas robóticas diferentes para evaluar el rendimi

En este proyecto se presenta un brazo robótico controlado remotamente por un guante. Este brazo está acoplado a una mano robótica que pretende imitar los movimientos de una mano real. Para ello se sirve de 3 servomotores por dedos, proporcionando una mayor libertad en el movimiento. También permite la rotación en dos ejes de la muñeca, con dos servomotores de mayor tamaño. La mano robótica monta sensores de presión limitando el cierre tras haber superado un umbral. El guante mide la apertura y cierre de la mano usando potenciómetros acoplados a un mecanismo diseñado para esto. También dispone de un giroscopio que se encarga de medir la rotación de la muñeca. El guante incorpora una pantalla oled, disponiendo al usuario de la información que se envía al brazo robótico. Esta pantalla también será útil para informar del proceso de calibración necesario al montar sensores analógicos. El envío y recepción inalámbricos de datos se hará mediante módulos Xbee, que cumplen con el protocolo de comunicaciones Zigbee., This project presents a robotic arm remotely controlled by a glove. This arm is coupled to a robotic hand that aims to imitate the movements of a real hand. It uses 3 servomotors per finger, providing greater freedom of movement. It also allows rotation on two axes of the wrist, with two bigger servomotors. The robotic hand mounts pressure sensors which limit the closure after a threshold has been exceeded. The glove measures the opening and closing of the hand using potentiometers coupled to a mechanism designed for this purpose. It also has a gyroscope that measures the rotation of the wrist. The glove incorporates an oled display, providing the user with information that is sent to the robotic arm. This display will also be useful to inform the user of the calibration process required when mount analog sensors. The wireless sending and receiving of data will be done via Xbee modules, which comply with the Zigbee communications protocol.

Hearing can be considered as one of the most important senses since it plays a key role in the audiovisual learning process. While a lot of effort has been made for achieving good results from the traditional approach of auditory perception, new trends such as neuromorphic computing are showing promising achievements in the implementation of brain structures for sensory perception. In this work, the design and integration of a neuromorphic event based digital model of the auditory ascending pathway within the iCub robotic platform is proposed. This model, which comprises from the cochlea up to the inferior colliculus, replaces the traditional approach for sound processing already implemented on iCub and is able to perform sound recognition and spatial localization in real time, allowing the implementation of auditory attention models in complex scenarios, like the cocktail party problem

Event cameras (or silicon retinas) represent a new kind of sensor that measure pixel-wise changes in brightness and output asynchronous events accordingly. This novel technology allows for a sparse and energy-efficient recording and storage of visual information. While this type of data is sparse by definition, the event flow can be very high, up to 25M events per second, which requires significant processing resources to handle and therefore impedes embedded applications. Neuromorphic computer vision and event sensor based applications are receiving an increasing interest from the computer vision community (classification, detection, tracking, segmentation, etc.), especially for robotics or autonomous driving scenarios. Downscaling event data is an important feature in a system, especially if embedded, so as to be able to adjust the complexity of data to the available resources such as processing capability and power consumption. To the best of our knowledge, this works is the first attempt to formalize event data downscaling. In order to study the impact of spatial resolution downscaling, we compare several features of the resulting data, such as the total number of events, event density, information entropy, computation time and optical consistency as assessment criteria. Our code is available online at https://github.com/amygruel/EvVisu.

Brain-inspired computing employs devices and architectures that emulate biological functions for more adaptive and energy-efficient systems. Oscillatory neural networks (ONNs) are an alternative approach in emulating biological functions of the human brain and are suitable for solving large and complex associative problems. In this work, we investigate the dynamics of coupled oscillators to implement such ONNs. By harnessing the complex dynamics of coupled oscillatory systems, we forge a novel computation model—information is encoded in the phase of oscillations. Coupled interconnected oscillators can exhibit various behaviors due to the strength of the coupling. In this article, we present a novel method based on subharmonic injection locking (SHIL) for controlling the oscillatory states of coupled oscillators that allow them to lock in frequency with distinct phase differences. Circuit-level simulation results indicate SHIL effectiveness and its applicability to large-scale oscillatory networks for pattern recognition.

The advent of nanoscale memristors raised hopes of being able to build CMOL (CMOS/nanowire/molecular) type ultra-dense in-memory-computing circuit architectures. In CMOL, nanoscale memristors would be fabricated at the intersection of nanowires. The CMOL concept can be exploited in neuromorphic hardware by fabricating lower density neurons on CMOS and placing massive analog synaptic connectivity with nanowire and nanoscale-memristor fabric post-fabricated on top. However, technical problems have hindered such developments for presently available reliable commercial monolithic CMOS-memristor technologies. On one hand, each memristor needs a MOS selector transistor in series to guarantee forming and programming operations in large arrays. This results in compound MOS-memristor synapses (called 1T1R) which are no longer synapses at the crossing of nanowires. On the other hand, memristors do not yet constitute highly reliable, stable analog memories for massive analog-weight synapses with gradual learning. Here we demonstrate a pseudo-CMOL monolithic chip core that circumvents the two technical problems mentioned above by: (a) exploiting a CMOL-like geometrical chip layout technique to improve density despite the 1T1R limitation, and (b) exploiting a binary weight stochastic Spike-Timing-Dependent-Plasticity (STDP) learning rule that takes advantage of the more reliable binary memory capability of the memristors used. Experimental results are provided for a spiking neural network (SNN) CMOL-core with 64 input neurons, 64 output neurons and 4096 1T1R synapses, fabricated in 130nm CMOS with 200nm-sized Ti/HfOx/TiN memristors on top. The CMOL-core uses query-driven event read-out, which allows for memristor variability insensitive computations. Experimental system-level demonstrations are provided for plain template matching tasks, as well as regularized stochastic binary STDP feature-extraction learning, obtaining perfect recognition in hardware for a 4-letter recognition