Search

A single neuron receives an extensive array of synaptic inputs through its dendrites, raising the fundamental question of how these inputs undergo integration and summation, culminating in the initiation of spikes in the soma. Experimental and computational investigations have revealed various modes of integration operations that include linear, superlinear, and sublinear summation. Interestingly, distinct neuron types exhibit diverse patterns of dendritic integration contingent upon the spatial distribution of dendrites. The functional implications of these specific integration modalities remain largely unexplored. In this study, we employ the Purkinje cell as a model system to investigate these intricate questions. Our findings reveal that Purkinje cells (PCs) generally exhibit sublinear summation across their expansive dendrites. The degree of sublinearity is dynamically modulated by both spatial and temporal input. Strong sublinearity necessitates that the synaptic distribution in PCs be globally scattered sensitive, whereas weak sublinearity facilitates the generation of complex firing patterns in PCs. Leveraging dendritic branches characterized by strong sublinearity as computational units, we demonstrate that a neuron can adeptly address the feature-binding problem. Collectively, these results offer a systematic perspective on the functional role of dendritic sublinearity, providing inspiration for a broader understanding of dendritic integration across various neuronal types.

The visual pathway involves complex networks of cells and regions which contribute to the encoding and processing of visual information. While some aspects of visual perception are understood, there are still many unanswered questions regarding the exact mechanisms of visual encoding and the organization of visual information along the pathway. This chapter discusses the importance of visual perception and the challenges associated with understanding how visual information is encoded and represented in the brain. Furthermore, this chapter introduces the concept of neuroprostheses: devices designed to enhance or replace bodily functions, and highlights the importance of constructing computational models of the visual pathway in the implementation of such devices. A number of such models, employing the use of deep learning models, are outlined, and their value to understanding visual coding and natural vision is discussed.

In digital cameras, we find a major limitation: the image and video form inherited from a film camera obstructs it from capturing the rapidly changing photonic world. Here, we present vidar, a bit sequence array where each bit represents whether the accumulation of photons has reached a threshold, to record and reconstruct the scene radiance at any moment. By employing only consumer-level CMOS sensors and integrated circuits, we have developed a vidar camera that is 1,000x faster than conventional cameras. By treating vidar as spike trains in biological vision, we have further developed a spiking neural network-based machine vision system that combines the speed of the machine and the mechanism of biological vision, achieving high-speed object detection and tracking 1,000x faster than human vision. We demonstrate the utility of the vidar camera and the super vision system in an assistant referee and target pointing system. Our study is expected to fundamentally revolutionize the image and video concepts and related industries, including photography, movies, and visual media, and to unseal a new spiking neural network-enabled speed-free machine vision era.

Neuroprosthesis, as one type of precision medicine device, is aiming for manipulating neuronal signals of the brain in a closed-loop fashion, together with receiving stimulus from the environment and controlling some part of our brain/body. In terms of vision, incoming information can be processed by the brain in millisecond interval. The retina computes visual scenes and then sends its output as neuronal spikes to the cortex for further computation. Therefore, the neuronal signal of interest for retinal neuroprosthesis is spike. Closed-loop computation in neuroprosthesis includes two stages: encoding stimulus to neuronal signal, and decoding it into stimulus. Here we review some of the recent progress about visual computation models that use spikes for analyzing natural scenes, including static images and dynamic movies. We hypothesize that for a better understanding of computational principles in the retina, one needs a hypercircuit view of the retina, in which different functional network motifs revealed in the cortex neuronal network should be taken into consideration for the retina. Different building blocks of the retina, including a diversity of cell types and synaptic connections, either chemical synapses or electrical synapses (gap junctions), make the retina an ideal neuronal network to adapt the computational techniques developed in artificial intelligence for modeling of encoding/decoding visual scenes. Altogether, one needs a systems approach of visual computation with spikes to advance the next generation of retinal neuroprosthesis as an artificial visual system., Comment: 15 pages, 5 figures

Light field modulation matrix is closely related to the quality of reconstructed image in ghost imaging. The orthogonality light field modulation matrix with better noise immunity and high quality reconstructed image is urgently needed in the practical application of ghost imaging. In this work, we propose a Gold matrix ghost imaging method with excellent imaging and anti-noise performance, which is proud of the pseudo-randomness and orthogonality of the deterministic Gold matrix. Numerical simulation and experimental results show that the Gold matrix has the characteristics of both random speckle and Hadamard matrix, and the reconstructed images are superior to the two modulation matrixes under the condition of noise, which shows better imaging robustness., Comment: 9 pages, 7 figures

Neural coding is one of the central questions in systems neuroscience for understanding how the brain processes stimulus from the environment, moreover, it is also a cornerstone for designing algorithms of brain-machine interface, where decoding incoming stimulus is highly demanded for better performance of physical devices. Traditionally researchers have focused on functional magnetic resonance imaging (fMRI) data as the neural signals of interest for decoding visual scenes. However, our visual perception operates in a fast time scale of millisecond in terms of an event termed neural spike. There are few studies of decoding by using spikes. Here we fulfill this aim by developing a novel decoding framework based on deep neural networks, named spike-image decoder (SID), for reconstructing natural visual scenes, including static images and dynamic videos, from experimentally recorded spikes of a population of retinal ganglion cells. The SID is an end-to-end decoder with one end as neural spikes and the other end as images, which can be trained directly such that visual scenes are reconstructed from spikes in a highly accurate fashion. Our SID also outperforms on the reconstruction of visual stimulus compared to existing fMRI decoding models. In addition, with the aid of a spike encoder, we show that SID can be generalized to arbitrary visual scenes by using the image datasets of MNIST, CIFAR10, and CIFAR100. Furthermore, with a pre-trained SID, one can decode any dynamic videos to achieve real-time encoding and decoding of visual scenes by spikes. Altogether, our results shed new light on neuromorphic computing for artificial visual systems, such as event-based visual cameras and visual neuroprostheses., Comment: 35 pages, 10 figures

Recent studies have suggested that the cognitive process of the human brain is realized as probabilistic inference and can be further modeled by probabilistic graphical models like Markov random fields. Nevertheless, it remains unclear how probabilistic inference can be implemented by a network of spiking neurons in the brain. Previous studies have tried to relate the inference equation of binary Markov random fields to the dynamic equation of spiking neural networks through belief propagation algorithm and reparameterization, but they are valid only for Markov random fields with limited network structure. In this paper, we propose a spiking neural network model that can implement inference of arbitrary binary Markov random fields. Specifically, we design a spiking recurrent neural network and prove that its neuronal dynamics are mathematically equivalent to the inference process of Markov random fields by adopting mean-field theory. Furthermore, our mean-field approach unifies previous works. Theoretical analysis and experimental results, together with the application to image denoising, demonstrate that our proposed spiking neural network can get comparable results to that of mean-field inference., Comment: Accepted in Neural Networks

Deep convolutional neural networks (CNNs) have demonstrated impressive performance on many visual tasks. Recently, they became useful models for the visual system in neuroscience. However, it is still not clear what are learned by CNNs in terms of neuronal circuits. When a deep CNN with many layers is used for the visual system, it is not easy to compare the structure components of CNNs with possible neuroscience underpinnings due to highly complex circuits from the retina to higher visual cortex. Here we address this issue by focusing on single retinal ganglion cells with biophysical models and recording data from animals. By training CNNs with white noise images to predict neuronal responses, we found that fine structures of the retinal receptive field can be revealed. Specifically, convolutional filters learned are resembling biological components of the retinal circuit. This suggests that a CNN learning from one single retinal cell reveals a minimal neural network carried out in this cell. Furthermore, when CNNs learned from different cells are transferred between cells, there is a diversity of transfer learning performance, which indicates that CNNs are cell-specific. Moreover, when CNNs are transferred between different types of input images, here white noise v.s. natural images, transfer learning shows a good performance, which implies that CNNs indeed capture the full computational ability of a single retinal cell for different inputs. Taken together, these results suggest that CNNs could be used to reveal structure components of neuronal circuits, and provide a powerful model for neural system identification., Comment: updated version

Neuronal circuits formed in the brain are complex with intricate connection patterns. Such complexity is also observed in the retina as a relatively simple neuronal circuit. A retinal ganglion cell receives excitatory inputs from neurons in previous layers as driving forces to fire spikes. Analytical methods are required that can decipher these components in a systematic manner. Recently a method termed spike-triggered non-negative matrix factorization (STNMF) has been proposed for this purpose. In this study, we extend the scope of the STNMF method. By using the retinal ganglion cell as a model system, we show that STNMF can detect various computational properties of upstream bipolar cells, including spatial receptive field, temporal filter, and transfer nonlinearity. In addition, we recover synaptic connection strengths from the weight matrix of STNMF. Furthermore, we show that STNMF can separate spikes of a ganglion cell into a few subsets of spikes where each subset is contributed by one presynaptic bipolar cell. Taken together, these results corroborate that STNMF is a useful method for deciphering the structure of neuronal circuits., Comment: updated version