Search

[Display omitted] Cognitive neuroscience aims to develop computational models that can accurately predict and explain neural responses to sensory inputs in the cortex. Recent studies attempt to leverage the representation power of deep neural networks (DNNs) to predict the brain response and suggest a correspondence between artificial and biological neural networks in their feature representations. However, typical voxel-wise encoding models tend to rely on specific networks designed for computer vision tasks, leading to suboptimal brain-wide correspondence during cognitive tasks. To address this challenge, this work proposes a novel approach that upgrades voxel-wise encoding models through multi-level integration of features from DNNs and information from brain networks. Our approach combines DNN feature-level ensemble learning and brain atlas-level model integration, resulting in significant improvements in predicting whole-brain neural activity during naturalistic video perception. Furthermore, this multi-level integration framework enables a deeper understanding of the brain's neural representation mechanism, accurately predicting the neural response to complex visual concepts. We demonstrate that neural encoding models can be optimized by leveraging a framework that integrates both data-driven approaches and theoretical insights into the functional structure of the cortical networks. [ABSTRACT FROM AUTHOR]

In motor cortex, behaviorally relevant neural responses are entangled with irrelevant signals, which complicates the study of encoding and decoding mechanisms. It remains unclear whether behaviorally irrelevant signals could conceal some critical truth. One solution is to accurately separate behaviorally relevant and irrelevant signals at both single-neuron and single-trial levels, but this approach remains elusive due to the unknown ground truth of behaviorally relevant signals. Therefore, we propose a framework to define, extract, and validate behaviorally relevant signals. Analyzing separated signals in three monkeys performing different reaching tasks, we found neural responses previously considered to contain little information actually encode rich behavioral information in complex nonlinear ways. These responses are critical for neuronal redundancy and reveal movement behaviors occupy a higher-dimensional neural space than previously expected. Surprisingly, when incorporating often-ignored neural dimensions, behaviorally relevant signals can be decoded linearly with comparable performance to nonlinear decoding, suggesting linear readout may be performed in motor cortex. Our findings prompt that separating behaviorally relevant signals may help uncover more hidden cortical mechanisms.

Neural dynamics can reflect intrinsic dynamics or dynamic inputs, such as sensory inputs or inputs from other brain regions. To avoid misinterpreting temporally structured inputs as intrinsic dynamics, dynamical models of neural activity should account for measured inputs. However, incorporating measured inputs remains elusive in joint dynamical modeling of neural-behavioral data, which is important for studying neural computations of behavior. We first show how training dynamical models of neural activity while considering behavior but not input or input but not behavior may lead to misinterpretations. We then develop an analytical learning method for linear dynamical models that simultaneously accounts for neural activity, behavior, and measured inputs. The method provides the capability to prioritize the learning of intrinsic behaviorally relevant neural dynamics and dissociate them from both other intrinsic dynamics and measured input dynamics. In data from a simulated brain with fixed intrinsic dynamics that performs different tasks, the method correctly finds the same intrinsic dynamics regardless of the task while other methods can be influenced by the task. In neural datasets from three subjects performing two different motor tasks with task instruction sensory inputs, the method reveals low-dimensional intrinsic neural dynamics that are missed by other methods and are more predictive of behavior and/or neural activity. The method also uniquely finds that the intrinsic behaviorally relevant neural dynamics are largely similar across the different subjects and tasks, whereas the overall neural dynamics are not. These input-driven dynamical models of neural-behavioral data can uncover intrinsic dynamics that may otherwise be missed. [ABSTRACT FROM AUTHOR]

This research investigates the implementation of complex-exponential-based neurons in FPGA, which can pave the way for implementing bio-inspired spiking neural networks to compensate for the existing computational constraints in conventional artificial neural networks. The increasing use of extensive neural networks and the complexity of models in handling big data lead to higher power consumption and delays. Hence, finding solutions to reduce computational complexity is crucial for addressing power consumption challenges. The complex exponential form effectively encodes oscillating features like frequency, amplitude, and phase shift, streamlining the demanding calculations typical of conventional artificial neurons through levering the simple phase addition of complex exponential functions. The article implements such a two-neuron and a multi-neuron neural model using the Xilinx System Generator and Vivado Design Suite, employing 8-bit, 16-bit, and 32-bit fixed-point data format representations. The study evaluates the accuracy of the proposed neuron model across different FPGA implementations while also providing a detailed analysis of operating frequency, power consumption, and resource usage for the hardware implementations. BRAM-based Vivado designs outperformed Simulink regarding speed, power, and resource efficiency. Specifically, the Vivado BRAM-based approach supported up to 128 neurons, showcasing optimal LUT and FF resource utilization. Such outcomes accommodate choosing the optimal design procedure for implementing spiking neural networks on FPGAs. [ABSTRACT FROM AUTHOR]

Expanding wearable technologies to artificial tactile perception will be of significance for intelligent human–machine interface, as neuromorphic sensing devices are promising candidates due to their low energy consumption and highly effective operating properties. Skin‐compatible and conformable features are required for the purpose of realizing wearable artificial tactile perception. Here, we report an intrinsically stretchable, skin‐integrated neuromorphic system with triboelectric nanogenerators as tactile sensing and organic electrochemical transistors as information processing. The integrated system provides desired sensing, synaptic, and mechanical characteristics, such as sensitive response (~0.04 kPa−1) to low‐pressure, short‐ and long‐term synaptic plasticity, great switching endurance (>10 000 pulses), symmetric weight update, together with high stretchability of 100% strain. With neural encoding, demonstrations are capable of recognizing, extracting, and encoding features of tactile information. This work provides a feasible approach to wearable, skin‐conformable neuromorphic sensing system with great application prospects in intelligent robotics and replacement prosthetics. [ABSTRACT FROM AUTHOR]

Abstract Expanding wearable technologies to artificial tactile perception will be of significance for intelligent human–machine interface, as neuromorphic sensing devices are promising candidates due to their low energy consumption and highly effective operating properties. Skin‐compatible and conformable features are required for the purpose of realizing wearable artificial tactile perception. Here, we report an intrinsically stretchable, skin‐integrated neuromorphic system with triboelectric nanogenerators as tactile sensing and organic electrochemical transistors as information processing. The integrated system provides desired sensing, synaptic, and mechanical characteristics, such as sensitive response (~0.04 kPa−1) to low‐pressure, short‐ and long‐term synaptic plasticity, great switching endurance (>10 000 pulses), symmetric weight update, together with high stretchability of 100% strain. With neural encoding, demonstrations are capable of recognizing, extracting, and encoding features of tactile information. This work provides a feasible approach to wearable, skin‐conformable neuromorphic sensing system with great application prospects in intelligent robotics and replacement prosthetics.

Both experimenter-controlled stimuli and stimulus-independent variables impact cortical neural activity. A major hurdle to understanding neural representation is distinguishing between qualitatively different causes of the fluctuating population activity. We applied an unsupervised low-rank tensor decomposition analysis to the recorded population activity in the visual cortex of awake mice in response to repeated presentations of naturalistic visual stimuli. We found that neurons covaried largely independently of individual neuron stimulus response reliability and thus encoded both stimulus-driven and stimulus-independent variables. Importantly, a neuron's response reliability and the neuronal coactivation patterns substantially reorganized for different external visual inputs. Analysis of recurrent balanced neural network models revealed that both the stimulus specificity and the mixed encoding of qualitatively different variables can arise from clustered external inputs. These results establish that coactive neurons with diverse response reliability mediate a mixed representation of stimulus-driven and stimulus-independent variables in the visual cortex.NEW & NOTEWORTHY V1 neurons covary largely independently of individual neuron's response reliability. A single neuron's response reliability imposes only a weak constraint on its encoding capabilities. Visual stimulus instructs a neuron's reliability and coactivation pattern. Network models revealed using clustered external inputs.

A central question related to virtually all social decisions is how animals integrate sex-specific cues from conspecifics. Using microendoscopic calcium imaging in mice, we find that sex information is represented in the dorsal medial prefrontal cortex (dmPFC) across excitatory and inhibitory neurons. These cells form a distributed code that differentiates the sex of conspecifics and is strengthened with social experience. While males and females both represent sex in the dmPFC, male mice show stronger encoding of female cues, and the relative strength of these sex representations predicts sex preference behavior. Using activity-dependent optogenetic manipulations of natively active ensembles, we further show that these specific representations modulate preference behavior toward males and females. Together, these results define a functional role for native representations of sex in shaping social behavior and reveal a neural mechanism underlying male- versus female-directed sociality.

Predicting the responses of sensory neurons is a long-standing neuroscience goal. However, while there has been much progress in modeling neural responses to simple and/or artificial stimuli, predicting responses to natural stimuli remains an ongoing challenge. On the one hand, deep neural networks perform very well on certain datasets but can fail when data are limited. On the other hand, Gaussian processes (GPs) perform well on limited data but are poor at predicting responses to high-dimensional stimuli, such as natural images. Here, we show how structured priors, e.g., for local and smooth receptive fields, can be used to scale up GPs to model neural responses to highdimensional stimuli. With this addition, GPs largely outperform a deep neural network trained to predict retinal responses to natural images, with the largest differences observed when both models are trained on a small dataset. Further, since they allow us to quantify the uncertainty in their predictions, GPs are well suited to closed-loop experiments, where stimuli are chosen actively so as to collect "informative" neural data. We show how GPs can be used to actively select which stimuli to present, so as to i) efficiently learn a model of retinal responses to natural images, using few data, and ii) rapidly distinguish between competing models (e.g., a linear vs. a nonlinear model). In the future, our approach could be applied to other sensory areas, beyond the retina. [ABSTRACT FROM AUTHOR]

Cognitive control is adaptive in that it rapidly adjusts attention in response to changing contexts and shifting goals. Research provides evidence that cognitive control can rapidly adjust attention to focus on task-relevant information based on prior conflict experience. Neural encoding of goal-related information is critical for goal-directed behaviour; however, the empirical evidence on how conflict experience affects the encoding of cognitive conflict in the brain is rather weak. In the present fMRI study, a Stroop task with different proportions of incongruent trial was used to investigate the neural encoding of cognitive conflict in the environment with changing conflict experience. The results showed that the anterior cingulate cortex, dorsolateral prefrontal cortex, and intraparietal sulcus played a pivotal role in the neural encoding of cognitive conflict. The classification in anterior cingulate cortex was significantly above chance in the high-proportion, moderate-proportion, and low-proportion conflict conditions conducted separately, suggesting that neural encoding of cognitive conflict in this region was not altered based on proportion of conflict. The dorsolateral prefrontal cortex and intraparietal sulcus showed significant above-chance classification in the moderate-proportion and low-proportion conflict conditions, but not in the high-proportion conflict condition. These findings provide direct evidence that conflict experience modulates the neural encoding of cognitive conflict. [ABSTRACT FROM AUTHOR]

Episodic memory, an essential element of orderly thinking, requires the organization of serial events into narratives about the identity of cues along with their locations and temporal order (what, where, and when). The hippocampus plays a central role in the acquisition and retrieval of episodes with two of its subsystems being separately linked to what and where information. The substrates for the third element are poorly understood. Here we report that in hippocampal slices field CA3 maintains self-sustained activity for remarkable periods following a brief input and that this effect is extremely sensitive to minor network perturbations. Using behavioral tests, that do not involve training or explicit rewards, we show that partial silencing of the CA3 commissural/associational network in mice blocks acquisition of temporal order, but not the identity or location, of odors. These results suggest a solution to the question of how hippocampus adds time to episodic memories.

Microelectrode arrays (MEA) are extensively utilized in encoding studies of retinal ganglion cells (RGCs) due to their capacity for simultaneous recording of neural activity across multiple channels. However, conventional planar MEAs face limitations in studying RGCs due to poor coupling between electrodes and RGCs, resulting in low signal-to-noise ratio (SNR) and limited recording sensitivity. To overcome these challenges, we employed photolithography, electroplating, and other processes to fabricate a 3D MEA based on the planar MEA platform. The 3D MEA exhibited several improvements compared to planar MEA, including lower impedance (8.73 ± 1.66 kΩ) and phase delay (−15.11° ± 1.27°), as well as higher charge storage capacity (CSC = 10.16 ± 0.81 mC/cm2), cathodic charge storage capacity (CSCc = 7.10 ± 0.55 mC/cm2), and SNR (SNR = 8.91 ± 0.57). Leveraging the advanced 3D MEA, we investigated the encoding characteristics of RGCs under multi-modal stimulation. Optical, electrical, and chemical stimulation were applied as sensory inputs, and distinct response patterns and response times of RGCs were detected, as well as variations in rate encoding and temporal encoding. Specifically, electrical stimulation elicited more effective RGC firing, while optical stimulation enhanced RGC synchrony. These findings hold promise for advancing the field of neural encoding.

In recent years, a third-generation neural network, namely, spiking neural network, has received plethora of attention in the broad areas of Machine learning and Artificial Intelligence. In this paper, a novel differential-based encoding method is proposed and new spike-based learning rules for backpropagation is derived by constraining the addition of bias voltage in spiking neurons. The proposed differential encoding method can effectively exploit the correlation between the data and improve the performance of the proposed model, and the new learning rule can take complete advantage of the modulation properties of bias on the spike firing threshold. We experiment with the proposed model on the environmental sound dataset RWCP and the image dataset MNIST and Fashion-MNIST, respectively, and assign various conditions to test the learning ability and robustness of the proposed model. The experimental results demonstrate that the proposed model achieves near-optimal results with a smaller time step by maintaining the highest accuracy and robustness with less training data. Among them, in MNIST dataset, compared with the original spiking neural network with the same network structure, we achieved a 0.39% accuracy improvement. [ABSTRACT FROM AUTHOR]

A comprehensive analysis of everyday sound perception can be achieved using electroencephalography (EEG) with the concurrent acquisition of information about the environment. While extensive research has been dedicated to speech perception, the complexities of auditory perception within everyday environments, specifically the types of information and the key features to extract, remain less explored. Our study aims to systematically investigate the relevance of different feature categories: discrete sound-identity markers, general cognitive state information, and acoustic representations, including discrete sound onset, the envelope, and mel-spectrogram. Using continuous data analysis, we contrast different features in terms of their predictive power for unseen data and thus their distinct contributions to explaining neural data. For this, we analyze data from a complex audio-visual motor task using a naturalistic soundscape. The results demonstrated that the feature sets that explain the most neural variability were a combination of highly detailed acoustic features with a comprehensive description of specific sound onsets. Furthermore, it showed that established features can be applied to complex soundscapes. Crucially, the outcome hinged on excluding periods devoid of sound onsets in the analysis in the case of the discrete features. Our study highlights the importance to comprehensively describe the soundscape, using acoustic and non-acoustic aspects, to fully understand the dynamics of sound perception in complex situations. This approach can serve as a foundation for future studies aiming to investigate sound perception in natural settings., (Copyright © 2025 Haupt et al.)

Introduction: To reorient gaze (the eye's direction in space) towards a target is an overdetermined problem, as infinitely many combinations of eye- and head movements can specify the same gaze-displacement vector. Yet, behavioral measurements show that the primate gaze-control system selects a specific contribution of eye- and head movements to the saccade, which depends on the initial eye-in-head orientation. Single-unit recordings in the primate superior colliculus (SC) during head-unrestrained gaze shifts have further suggested that cells may encode the instantaneous trajectory of a desired straight gaze path in a feedforward way by the total cumulative number of spikes in the neural population, and that the instantaneous gaze kinematics are thus determined by the neural firing rates. The recordings also indicated that the latter is modulated by the initial eye position. We recently proposed a conceptual model that accounts for many of the observed properties of eye-head gaze shifts and on the potential role of the SC in gaze control. Methods: Here, we extend and test the model by incorporating a spiking neural network of the SC motor map, the output of which drives the eye-head motor control circuitry by linear cumulative summation of individual spike effects of each recruited SC neuron. We propose a simple neural mechanism on SC cells that explains the modulatory influence of feedback from an initial eye-in-head position signal on their spiking activity. The same signal also determines the onset delay of the head movement with respect to the eye. Moreover, the downstream eye- and head burst generators were taken to be linear, as our earlier work had indicated that much of the non-linear main-sequence kinematics of saccadic eye movements may be due to neural encoding at the collicular level, rather than at the brainstem. Results and discussion: We investigate how the spiking activity of the SC population drives gaze to the intended target location within a dynamic local gaze-velocity feedback circuit that yields realistic eye- and head-movement kinematics and dynamic SC gaze-movement fields. [ABSTRACT FROM AUTHOR]

Compiler frameworks are crucial for the widespread use of FPGA-based deep learning accelerators. They allow researchers and developers, who are not familiar with hardware engineering, to harness the performance attained by domain-specific logic. There exists a variety of frameworks for conventional artificial neural networks. However, not much research effort has been put into the creation of frameworks optimized for spiking neural networks (SNNs). This new generation of neural networks becomes increasingly interesting for the deployment of AI on edge devices, which have tight power and resource constraints. Our end-to-end framework E3NE automates the generation of efficient SNN inference logic for FPGAs. Based on a PyTorch model and user parameters, it applies various optimizations and assesses trade-offs inherent to spike-based accelerators. Multiple levels of parallelism and the use of an emerging neural encoding scheme result in an efficiency superior to previous SNN hardware implementations. For a similar model, E3NE uses less than 50% of hardware resources and 20% less power, while reducing the latency by an order of magnitude. Furthermore, scalability and generality allowed the deployment of the large-scale SNN models AlexNet and VGG. [ABSTRACT FROM AUTHOR]

IntroductionTo reorient gaze (the eye’s direction in space) towards a target is an overdetermined problem, as infinitely many combinations of eye- and head movements can specify the same gaze-displacement vector. Yet, behavioral measurements show that the primate gaze-control system selects a specific contribution of eye- and head movements to the saccade, which depends on the initial eye-in-head orientation. Single-unit recordings in the primate superior colliculus (SC) during head-unrestrained gaze shifts have further suggested that cells may encode the instantaneous trajectory of a desired straight gaze path in a feedforward way by the total cumulative number of spikes in the neural population, and that the instantaneous gaze kinematics are thus determined by the neural firing rates. The recordings also indicated that the latter is modulated by the initial eye position. We recently proposed a conceptual model that accounts for many of the observed properties of eye-head gaze shifts and on the potential role of the SC in gaze control.MethodsHere, we extend and test the model by incorporating a spiking neural network of the SC motor map, the output of which drives the eye-head motor control circuitry by linear cumulative summation of individual spike effects of each recruited SC neuron. We propose a simple neural mechanism on SC cells that explains the modulatory influence of feedback from an initial eye-in-head position signal on their spiking activity. The same signal also determines the onset delay of the head movement with respect to the eye. Moreover, the downstream eye- and head burst generators were taken to be linear, as our earlier work had indicated that much of the non-linear main-sequence kinematics of saccadic eye movements may be due to neural encoding at the collicular level, rather than at the brainstem.Results and discussionWe investigate how the spiking activity of the SC population drives gaze to the intended target location within a dynamic local gaze-velocity feedback circuit that yields realistic eye- and head-movement kinematics and dynamic SC gaze-movement fields.

To understand neural encoding of neuropathic pain, evoked and resting activity of peripheral human C-fibers are studied via microneurography experiments. Before different spiking patterns can be analyzed, spike sorting is necessary to distinguish the activity of particular fibers of a recorded bundle. Due to single-electrode measurements and high noise contamination, standard methods based on spike shapes are insufficient and need to be enhanced with additional information. Such information can be derived from the activity-dependent slowing of the fiber propagation speed, which in turn can be assessed by introducing continuous "background" 0.125-0.25Hz electrical stimulation and recording the corresponding responses from the fibers. Each fiber's speed propagation remains almost constant in the absence of spontaneous firing or additional stimulation. This way, the responses to the "background stimulation" can be sorted by fiber. In this article, we model the changes in the propagation speed resulting from the history of fiber activity with polynomial regression. This is done to assess the feasibility of using the developed models to enhance the spike shape-based sorting. In addition to human microneurography data, we use animal in-vitro recordings with a similar stimulation protocol as higher signal-to-noise ratio data example for the models. [ABSTRACT FROM AUTHOR]

To understand neural encoding of neuropathic pain, evoked and resting activity of peripheral human C-fibers are studied via microneurography experiments. Before different spiking patterns can be analyzed, spike sorting is necessary to distinguish the activity of particular fibers of a recorded bundle. Due to single-electrode measurements and high noise contamination, standard methods based on spike shapes are insufficient and need to be enhanced with additional information. Such information can be derived from the activity-dependent slowing of the fiber propagation speed, which in turn can be assessed by introducing continuous “background” 0.125–0.25 Hz electrical stimulation and recording the corresponding responses from the fibers. Each fiber's speed propagation remains almost constant in the absence of spontaneous firing or additional stimulation. This way, the responses to the “background stimulation” can be sorted by fiber. In this article, we model the changes in the propagation speed resulting from the history of fiber activity with polynomial regression. This is done to assess the feasibility of using the developed models to enhance the spike shape-based sorting. In addition to human microneurography data, we use animal in-vitro recordings with a similar stimulation protocol as higher signal-to-noise ratio data example for the models.

By integrating hierarchical feature modeling of auditory information using deep neural networks (DNNs), recent functional magnetic resonance imaging (fMRI) encoding studies have revealed the hierarchical neural auditory representation in the superior temporal gyrus (STG). Most of these studies adopted supervised DNNs (e.g., for audio classification) to derive the hierarchical feature representation of external auditory stimuli. One possible limitation is that the extracted features could be biased toward discriminative features while ignoring general attributes shared by auditory information in multiple categories. Consequently, the hierarchy of neural acoustic processing revealed by the encoding model might be biased toward classification. In this study, we explored the hierarchical neural auditory representation via an fMRI encoding framework in which an unsupervised deep convolutional auto-encoder (DCAE) model was adopted to derive the hierarchical feature representations of the stimuli (naturalistic auditory excerpts in different categories) in fMRI acquisition. The experimental results showed that the neural representation of hierarchical auditory features is not limited to previously reported STG, but also involves the bilateral insula, ventral visual cortex, and thalamus. The current study may provide complementary evidence to understand the hierarchical auditory processing in the human brain. [ABSTRACT FROM AUTHOR]

Neural codes are reflected in complex neural activation patterns. Conventional electroencephalography (EEG) decoding analyses summarize activations by averaging/down-sampling signals within the analysis window. This diminishes informative fine-grained patterns. While previous studies have proposed distinct statistical features capable of capturing variability-dependent neural codes, it has been suggested that the brain could use a combination of encoding protocols not reflected in any one mathematical feature alone. To check, we combined 30 features using state-of-the-art supervised and unsupervised feature selection procedures (n = 17). Across three datasets, we compared decoding of visual object category between these 17 sets of combined features, and between combined and individual features. Object category could be robustly decoded using the combined features from all of the 17 algorithms. However, the combination of features, which were equalized in dimension to the individual features, were outperformed across most of the time points by the multiscale feature of Wavelet coefficients. Moreover, the Wavelet coefficients also explained the behavioral performance more accurately than the combined features. These results suggest that a single but multiscale encoding protocol may capture the EEG neural codes better than any combination of protocols. Our findings put new constraints on the models of neural information encoding in EEG. [ABSTRACT FROM AUTHOR]

Neural codes are reflected in complex neural activation patterns. Conventional electroencephalography (EEG) decoding analyses summarize activations by averaging/down-sampling signals within the analysis window. This diminishes informative fine-grained patterns. While previous studies have proposed distinct statistical features capable of capturing variability-dependent neural codes, it has been suggested that the brain could use a combination of encoding protocols not reflected in any one mathematical feature alone. To check, we combined 30 features using state-of-the-art supervised and unsupervised feature selection procedures (n = 17). Across three datasets, we compared decoding of visual object category between these 17 sets of combined features, and between combined and individual features. Object category could be robustly decoded using the combined features from all of the 17 algorithms. However, the combination of features, which were equalized in dimension to the individual features, were outperformed across most of the time points by the multiscale feature of Wavelet coefficients. Moreover, the Wavelet coefficients also explained the behavioral performance more accurately than the combined features. These results suggest that a single but multiscale encoding protocol may capture the EEG neural codes better than any combination of protocols. Our findings put new constraints on the models of neural information encoding in EEG.

By integrating hierarchical feature modeling of auditory information using deep neural networks (DNNs), recent functional magnetic resonance imaging (fMRI) encoding studies have revealed the hierarchical neural auditory representation in the superior temporal gyrus (STG). Most of these studies adopted supervised DNNs (e.g., for audio classification) to derive the hierarchical feature representation of external auditory stimuli. One possible limitation is that the extracted features could be biased toward discriminative features while ignoring general attributes shared by auditory information in multiple categories. Consequently, the hierarchy of neural acoustic processing revealed by the encoding model might be biased toward classification. In this study, we explored the hierarchical neural auditory representation via an fMRI encoding framework in which an unsupervised deep convolutional auto-encoder (DCAE) model was adopted to derive the hierarchical feature representations of the stimuli (naturalistic auditory excerpts in different categories) in fMRI acquisition. The experimental results showed that the neural representation of hierarchical auditory features is not limited to previously reported STG, but also involves the bilateral insula, ventral visual cortex, and thalamus. The current study may provide complementary evidence to understand the hierarchical auditory processing in the human brain.

Neural manifolds summarize the intrinsic structure of the information encoded by a population of neurons. Advances in experimental techniques have made simultaneous recordings from multiple brain regions increasingly commonplace, raising the possibility of studying how these manifolds relate across populations. However, when the manifolds are nonlinear and possibly code for multiple unknown variables, it is challenging to extract robust and falsifiable information about their relationships. We introduce a framework, called the method of analogous cycles, for matching topological features of neural manifolds using only observed dissimilarity matrices within and between neural populations. We demonstrate via analysis of simulations and in vivo experimental data that this method can be used to correctly identify multiple shared circular coordinate systems across both stimuli and inferred neural manifolds. Conversely, the method rejects matching features that are not intrinsic to one of the systems. Further, as this method is deterministic and does not rely on dimensionality reduction or optimization methods, it is amenable to direct mathematical investigation and interpretation in terms of the underlying neural activity. We thus propose the method of analogous cycles as a suitable foundation for a theory of cross-population analysis via neural manifolds., Competing Interests: Competing interests statement:The authors declare no competing interest.

Functional MRI (fMRI) is a powerful technique that has allowed us to characterize visual cortex responses to stimuli, yet such experiments are by nature constructed based on a priori hypotheses, limited to the set of images presented to the individual while they are in the scanner, are subject to noise in the observed brain responses, and may vary widely across individuals. In this work, we propose a novel computational strategy, which we call NeuroGen, to overcome these limitations and develop a powerful tool for human vision neuroscience discovery. NeuroGen combines an fMRI-trained neural encoding model of human vision with a deep generative network to synthesize images predicted to achieve a target pattern of macro-scale brain activation. We demonstrate that the reduction of noise that the encoding model provides, coupled with the generative network’s ability to produce images of high fidelity, results in a robust discovery architecture for visual neuroscience. By using only a small number of synthetic images created by NeuroGen, we demonstrate that we can detect and amplify differences in regional and individual human brain response patterns to visual stimuli. We then verify that these discoveries are reflected in the several thousand observed image responses measured with fMRI. We further demonstrate that NeuroGen can create synthetic images predicted to achieve regional response patterns not achievable by the best-matching natural images. The NeuroGen framework extends the utility of brain encoding models and opens up a new avenue for exploring, and possibly precisely controlling, the human visual system.

Cognitive neuroscience, in particular research on speech and language, has seen an increase in the use of linear modeling techniques for studying the processing of natural, environmental stimuli. The availability of such computational tools has prompted similar investigations in many clinical domains, facilitating the study of cognitive and sensory deficits under more naturalistic conditions. However, studying clinical (and often highly heterogeneous) cohorts introduces an added layer of complexity to such modeling procedures, potentially leading to instability of such techniques and, as a result, inconsistent findings. Here, we outline some key methodological considerations for applied research, referring to a hypothetical clinical experiment involving speech processing and worked examples of simulated electrophysiological (EEG) data. In particular, we focus on experimental design, data preprocessing, stimulus feature extraction, model design, model training and evaluation, and interpretation of model weights. Throughout the paper, we demonstrate the implementation of each step in MATLAB using the mTRF-Toolbox and discuss how to address issues that could arise in applied research. In doing so, we hope to provide better intuition on these more technical points and provide a resource for applied and clinical researchers investigating sensory and cognitive processing using ecologically rich stimuli.

Cognitive neuroscience, in particular research on speech and language, has seen an increase in the use of linear modeling techniques for studying the processing of natural, environmental stimuli. The availability of such computational tools has prompted similar investigations in many clinical domains, facilitating the study of cognitive and sensory deficits under more naturalistic conditions. However, studying clinical (and often highly heterogeneous) cohorts introduces an added layer of complexity to such modeling procedures, potentially leading to instability of such techniques and, as a result, inconsistent findings. Here, we outline some key methodological considerations for applied research, referring to a hypothetical clinical experiment involving speech processing and worked examples of simulated electrophysiological (EEG) data. In particular, we focus on experimental design, data preprocessing, stimulus feature extraction, model design, model training and evaluation, and interpretation of model weights. Throughout the paper, we demonstrate the implementation of each step in MATLAB using the mTRF-Toolbox and discuss how to address issues that could arise in applied research. In doing so, we hope to provide better intuition on these more technical points and provide a resource for applied and clinical researchers investigating sensory and cognitive processing using ecologically rich stimuli. [ABSTRACT FROM AUTHOR]

Supraspinal signals play a significant role in compensatory responses to postural perturbations. Although the cortex is not necessary for basic postural tasks in intact animals, its role in responding to unexpected postural perturbations after spinal cord injury (SCI) has not been studied. To better understand how SCI impacts cortical encoding of postural perturbations, the activity of single neurons in the hindlimb sensorimotor cortex (HLSMC) was recorded in the rat during unexpected tilts before and after a complete midthoracic spinal transection. In a subset of animals, limb ground reaction forces were also collected. HLSMC activity was strongly modulated in response to different tilt profiles. As the velocity of the tilt increased, more information was conveyed by the HLSMC neurons about the perturbation due to increases in both the number of recruited neurons and the magnitude of their responses. SCI led to attenuated and delayed hindlimb ground reaction forces. However, HLSMC neurons remained responsive to tilts after injury but with increased latencies and decreased tuning to slower tilts. Information conveyed by cortical neurons about the tilts was therefore reduced after SCI, requiring more cells to convey the same amount of information as before the transection. Given that reorganization of the hindlimb sensorimotor cortex in response to therapy after complete midthoracic SCI is necessary for behavioral recovery, this sustained encoding of information after SCI could be a substrate for the reorganization that uses sensory information from above the lesion to control trunk muscles that permit weight-supported stepping and postural control. [ABSTRACT FROM AUTHOR]

Internal representation of the space is a fundamental and crucial function of the animal's brain. Grid cells in the medial entorhinal cortex are thought to provide an environment-invariant metric system for the navigation of the animal. Most experimental and theoretical studies have focused on the horizontal planar codes of grid cell, while how this metric coordinate system is configured in the actual three-dimensional space remains unclear. Evidence has implied the spatial cognition may not be fully volumetric. We proposed an oscillatory interference model with a novel gravity and body plane modulation to simulate grid cell activity in complex space for rodents. The animal can perceive the rotation of its body plane along the local surface by sensing the gravity, causing the modulation to the dendritic oscillations. The results not only reproduce the firing patterns of the grid cell recorded from known experiments, but also predict the grid codes in novel environments. It further demonstrates that the gravity signal is indispensable for the animal's navigation, and supports the hypothesis that the periodic firing of the grid cell is intrinsically not a volumetric code in three-dimensional space. This will provide new insights to understand the spatial representation of the actual world in the brain. [ABSTRACT FROM AUTHOR]

Study of the neural code for space in rodents has many insights to offer for how mammals, including humans, construct a mental representation of space. This code is centered on the hippocampal place cells, which are active in particular places in the environment. Place cells are informed by numerous other spatial cell types including grid cells, which provide a signal for distance and direction and are thought to help anchor the place cell signal. These neurons combine self-motion and environmental information to create and update their map-like representation. Study of their activity patterns in complex environments of varying structure has revealed that this "cognitive map" of space is not a fixed and rigid entity that permeates space, but rather is variably affected by the movement constraints of the environment. These findings are pointing toward a more flexible spatial code in which the map is adapted to the movement possibilities of the space. An as-yet-unanswered question is whether these different forms of representation have functional consequences, as suggested by an enactivist view of spatial cognition. [ABSTRACT FROM AUTHOR]

Experiments with brain-machine interfaces (BMIs) reveal that the estimated preferred direction (EPD) of cortical motor units may shift following the transition to brain control. However, the cause of those shifts, and in particular, whether they imply neural adaptation, is an open issue. Here we address this question in simulations and theoretical analysis. Simulations are based on the assumption that the brain implements optimal state estimation and feedback control and that cortical motor neurons encode the estimated state and control vector. Our simulations successfully reproduce apparent shifts in EPDs observed in BMI experiments with different BMI filters, including linear, Kalman and re-calibrated Kalman filters, even with no neural adaptation. Theoretical analysis identifies the conditions for reducing those shifts. We demonstrate that simulations that better satisfy those conditions result in smaller shifts in EPDs. We conclude that the observed shifts in EPDs may result from experimental conditions, and in particular correlated velocities or tuning weights, even with no adaptation. Under the above assumptions, we show that if neurons are tuned differently to the estimated velocity, estimated position and control signal, the EPD with respect to actual velocity may not capture the real PD in which the neuron encodes the estimated velocity. Our investigation provides theoretical and simulation tools for better understanding shifts in EPD and BMI experiments.

Experiments with brain-machine interfaces (BMIs) reveal that the estimated preferred direction (EPD) of cortical motor units may shift following the transition to brain control. However, the cause of those shifts, and in particular, whether they imply neural adaptation, is an open issue. Here we address this question in simulations and theoretical analysis. Simulations are based on the assumption that the brain implements optimal state estimation and feedback control and that cortical motor neurons encode the estimated state and control vector. Our simulations successfully reproduce apparent shifts in EPDs observed in BMI experiments with different BMI filters, including linear, Kalman and re-calibrated Kalman filters, even with no neural adaptation. Theoretical analysis identifies the conditions for reducing those shifts. We demonstrate that simulations that better satisfy those conditions result in smaller shifts in EPDs. We conclude that the observed shifts in EPDs may result from experimental conditions, and in particular correlated velocities or tuning weights, even with no adaptation. Under the above assumptions, we show that if neurons are tuned differently to the estimated velocity, estimated position and control signal, the EPD with respect to actual velocity may not capture the real PD in which the neuron encodes the estimated velocity. Our investigation provides theoretical and simulation tools for better understanding shifts in EPD and BMI experiments. [ABSTRACT FROM AUTHOR]

The cerebellum resembles a feedforward, three layer network of neurons in which the “hidden layer” consists of Purkinje cells (Pcells) and the output layer consists of deep cerebellar nucleus (DCN) neurons. In this analogy, the output of each DCN neuron is a prediction that is compared with the actual observation, resulting in an error signal that originates in the inferior olive. Efficient learning requires that the error signal reach the DCN neurons, as well as the P-cells that project onto them. However, this basic rule of learning is violated in the cerebellum: the olivary projections to the DCN are weak, particularly in adulthood. Instead, an extraordinarily strong signal is sent from the olive to the P-cells, producing complex spikes. Curiously, P-cells are grouped into small populations that converge onto single DCN neurons. Why are the P-cells organized in this way, and what is the membership criterion of each population? Here, I apply elementary mathematics from machine learning and consider the fact that P-cells that form a population exhibit a special property: they can synchronize their complex spikes, which in turn suppress activity of DCN neuron they project to. Thus complex spikes cannot only act as a teaching signal for a P-cell, but through complex spike synchrony, a P-cell population may act as a surrogate teacher for the DCN neuron that produced the erroneous output. It appears that grouping of P-cells into small populations that share a preference for error satisfies a critical requirement of efficient learning: providing error information to the output layer neuron (DCN) that was responsible for the error, as well as the hidden layer neurons (P-cells) that contributed to it. This population coding may account for several remarkable features of behavior during learning, including multiple timescales, protection from erasure, and spontaneous recovery of memory. [ABSTRACT FROM AUTHOR]

Both experimenter-controlled stimuli and stimulus-independent variables impact cortical neural activity. A major hurdle to understanding neural representation is distinguishing between qualitatively different causes of the fluctuating population activity. We applied an unsupervised low-rank tensor decomposition analysis to the recorded population activity in the visual cortex of awake mice in response to repeated presentations of naturalistic visual stimuli. We found that neurons covaried largely independently of individual neuron stimulus response reliability and thus encoded both stimulus-driven and stimulus-independent variables. Importantly, a neuron's response reliability and the neuronal coactivation patterns substantially reorganized for different external visual inputs. Analysis of recurrent balanced neural network models revealed that both the stimulus specificity and the mixed encoding of qualitatively different variables can arise from clustered external inputs. These results establish that coactive neurons with diverse response reliability mediate a mixed representation of stimulus-driven and stimulus-independent variables in the visual cortex. NEW & NOTEWORTHY V1 neurons covary largely independently of individual neuron's response reliability. A single neuron's response reliability imposes only a weak constraint on its encoding capabilities. Visual stimulus instructs a neuron's reliability and coactivation pattern. Network models revealed using clustered external inputs. [ABSTRACT FROM AUTHOR]

Innate behavioral biases and preferences can vary significantly among individuals of the same genotype. Though individuality is a fundamental property of behavior, it is not currently understood how individual differences in brain structure and physiology produce idiosyncratic behaviors. Here we present evidence for idiosyncrasy in olfactory behavior and neural responses in Drosophila. We show that individual female Drosophila from a highly inbred laboratory strain exhibit idiosyncratic odor preferences that persist for days. We used in vivo calcium imaging of neural responses to compare projection neuron (second-order neurons that convey odor information from the sensory periphery to the central brain) responses to the same odors across animals. We found that, while odor responses appear grossly stereotyped, upon closer inspection, many individual differences are apparent across antennal lobe (AL) glomeruli (compact microcircuits corresponding to different odor channels). Moreover, we show that neuromodulation, environmental stress in the form of altered nutrition, and activity of certain AL local interneurons affect the magnitude of interfly behavioral variability. Taken together, this work demonstrates that individual Drosophila exhibit idiosyncratic olfactory preferences and idiosyncratic neural responses to odors, and that behavioral idiosyncrasies are subject to neuromodulation and regulation by neurons in the AL. [ABSTRACT FROM AUTHOR]

Visual neuroprosthesis, that provide electrical stimulation along several sites of the human visual system, constitute a potential tool for vision restoration for the blind. Scientific and technological progress in the fields of neural engineering and artificial vision comes with new theories and tools that, along with the dawn of modern artificial intelligence, constitute a promising framework for the further development of neurotechnology. In the framework of the development of a Cortical Visual Neuroprosthesis for the blind (CORTIVIS), we are now facing the challenge of developing not only computationally powerful tools and flexible approaches that will allow us to provide some degree of functional vision to individuals who are profoundly blind. In this work, we propose a general neuroprosthesis framework composed of several task-oriented and visual encoding modules. We address the development and implementation of computational models of the firing rates of retinal ganglion cells and design a tool — Neurolight — that allows these models to be interfaced with intracortical microelectrodes in order to create electrical stimulation patterns that can evoke useful perceptions. In addition, the developed framework allows the deployment of a diverse array of state-of-the-art deep-learning techniques for task-oriented and general image pre-processing, such as semantic segmentation and object detection in our system's pipeline. To the best of our knowledge, this constitutes the first deep-learning-based system designed to directly interface with the visual brain through an intracortical microelectrode array. We implement the complete pipeline, from obtaining a video stream to developing and deploying task-oriented deep-learning models and predictive models of retinal ganglion cells' encoding of visual inputs under the control of a neurostimulation device able to send electrical train pulses to a microelectrode array implanted at the visual cortex. [ABSTRACT FROM AUTHOR]

Neuronal synchronization plays important roles in information encoding and transmission in the brain. Mathematical models of neurons have been widely used to simulate synchronization behavior and analyze its mechanisms. Common stochastic inputs are considered to be effective in facilitating synchronization. However, the mechanisms of how partial reset affects neuronal synchronization are still not well understood. In this paper, the synchronization of Stein's model neurons with partial reset is studied. The differences in synchronization mechanisms between neurons with full reset and those with partial reset are analyzed, and the findings lead to the novel concept of transient synchronization. Furthermore, it is proven analytically that due to common stochastic inputs, Stein's model neurons with different initial membrane potentials and partial reset achieve transient synchronization with probability 1. Additionally, a systematic numerical analysis is performed to explore the similarities and differences between full reset and partial reset regarding model parameters, synchronization time, and desynchronization behavior. Thus, partial reset is a powerful and flexible tool that facilitates neuronal synchronization while reserving the possibility of desynchronization. Our analysis also provides an alternative approach to analyze neurons of the integrate-and-fire family and a theoretical complement implying possible information encoding mechanisms in the brain. [ABSTRACT FROM AUTHOR]

Studies have suggested that consciousness is encoded discretely in time and synchronously in space of the brain. The present study was to model the alpha EEG as a brain clock to carry out the functions and to test whether the quality and rate of the oscillation could predict behavioral timing. Results showed that the alpha peak frequency was correlated with the conflict reaction time, and the selectivity was associated with the simple reaction time. These findings are consistent with previous reports and support the hypothesis that alpha EEG represents excitability cycles and may serves as a brain clock for spatial synchronization.

The auditory front-end is an integral part of a spiking neural network (SNN) when performing auditory cognitive tasks. It encodes the temporal dynamic stimulus, such as speech and audio, into an efficient, effective and reconstructable spike pattern to facilitate the subsequent processing. However, most of the auditory front-ends in current studies have not made use of recent findings in psychoacoustics and physiology concerning human listening. In this paper, we propose a neural encoding and decoding scheme that is optimized for audio processing. The neural encoding scheme, that we call Biologically plausible Auditory Encoding (BAE), emulates the functions of the perceptual components of the human auditory system, that include the cochlear filter bank, the inner hair cells, auditory masking effects from psychoacoustic models, and the spike neural encoding by the auditory nerve. We evaluate the perceptual quality of the BAE scheme using PESQ; the performance of the BAE based on sound classification and speech recognition experiments. Finally, we also built and published two spike-version of speech datasets: the Spike-TIDIGITS and the Spike-TIMIT, for researchers to use and benchmarking of future SNN research.