Search

For stimuli near perceptual threshold, the trial-by-trial activity of single neurons in many sensory areas is correlated with the animal’s perceptual report. This phenomenon has often been attributed to feedforward readout of the neural activity by the downstream decision-making circuits. The interpretation of choice-correlated activity is quite ambiguous, but its meaning can be better understood in the light of population-wide correlations among sensory neurons. Using a statistical nonlinear dimensionality reduction technique on single-trial ensemble recordings from the middle temporal (MT) area during perceptual-decision-making, we extracted low-dimensional latent factors that captured the population-wide fluctuations. We dissected the particular contributions of sensory-driven versus choice-correlated activity in the low-dimensional population code. We found that the latent factors strongly encoded the direction of the stimulus in single dimension with a temporal signature similar to that of single MT neurons. If the downstream circuit were optimally utilizing this information, choice-correlated signals should be aligned with this stimulus encoding dimension. Surprisingly, we found that a large component of the choice information resides in the subspace orthogonal to the stimulus representation inconsistent with the optimal readout view. This misaligned choice information allows the feedforward sensory information to coexist with the decision-making process. The time course of these signals suggest that this misaligned contribution likely is feedback from the downstream areas. We hypothesize that this non-corrupting choice-correlated feedback might be related to learning or reinforcing sensory-motor relations in the sensory population., Author summary In sensorimotor decision-making, internal representation of sensory stimuli is utilized for the generation of appropriate behavior for the context. Therefore, the correlation between variability in sensory neurons and perceptual decisions is sometimes explained by a causal, feedforward role of sensory noise in behavior. However, this correlation could also originate via feedback from decision-making mechanisms downstream of the sensory representation. This cannot be resolved by analyzing single unit responses, but requires a population level analysis. Area MT contains both sensory and choice information and is known to be the key sensory area for visual motion perception. Thus the decision-making process may be corrupting the sensory representation. However, we find that the sensory stimuli and choice variables are separate at the population level, contradicting the previous interpretations based on single unit recordings. This new insight postulates how neural systems can maintain a mixed representation while allows learning and adaptation.

Recent advances in T cell repertoire (TCR) sequencing allow for the characterization of repertoire properties, as well as the frequency and sharing of specific TCR. However, there is no efficient measure for the local density of a given TCR. TCRs are often described either through their Complementary Determining region 3 (CDR3) sequences, or theirV/J usage, or their clone size. We here show that the local repertoire density can be estimated using a combined representation of these components through distance conserving autoencoders and Kernel Density Estimates (KDE). We present ELATE–an Encoder-based LocAl Tcr dEnsity and show that the resulting density of a sample can be used as a novel measure to study repertoire properties. The cross-density between two samples can be used as a similarity matrix to fully characterize samples from the same host. Finally, the same projection in combination with machine learning algorithms can be used to predict TCR-peptide binding through the local density of known TCRs binding a specific target., Author summary T cell repertoires contain a vast amount of information on the donors, and can be used to characterize the donor, and apply machine learning algorithms on such repertoires. A limiting factor in the analysis of such repertoire is the lack of a good representation of the T cell receptors. We here propose an autoencoder, named ELATE to present receptors as real vectors. We show that this encoder can be used to characterize both full donors and specific receptors using either supervised or unsupervised methods.

Closed-loop neurotechnologies often need to adaptively learn an encoding model that relates the neural activity to the brain state, and is used for brain state decoding. The speed and accuracy of adaptive learning algorithms are critically affected by the learning rate, which dictates how fast model parameters are updated based on new observations. Despite the importance of the learning rate, currently an analytical approach for its selection is largely lacking and existing signal processing methods vastly tune it empirically or heuristically. Here, we develop a novel analytical calibration algorithm for optimal selection of the learning rate in adaptive Bayesian filters. We formulate the problem through a fundamental trade-off that learning rate introduces between the steady-state error and the convergence time of the estimated model parameters. We derive explicit functions that predict the effect of learning rate on error and convergence time. Using these functions, our calibration algorithm can keep the steady-state parameter error covariance smaller than a desired upper-bound while minimizing the convergence time, or keep the convergence time faster than a desired value while minimizing the error. We derive the algorithm both for discrete-valued spikes modeled as point processes nonlinearly dependent on the brain state, and for continuous-valued neural recordings modeled as Gaussian processes linearly dependent on the brain state. Using extensive closed-loop simulations, we show that the analytical solution of the calibration algorithm accurately predicts the effect of learning rate on parameter error and convergence time. Moreover, the calibration algorithm allows for fast and accurate learning of the encoding model and for fast convergence of decoding to accurate performance. Finally, larger learning rates result in inaccurate encoding models and decoders, and smaller learning rates delay their convergence. The calibration algorithm provides a novel analytical approach to predictably achieve a desired level of error and convergence time in adaptive learning, with application to closed-loop neurotechnologies and other signal processing domains., Author summary Closed-loop neurotechnologies for treatment of neurological disorders often require adaptively learning an encoding model to relate the neural activity to the brain state and decode this state. Fast and accurate adaptive learning is critically affected by the learning rate, a key variable in any adaptive algorithm. However, existing signal processing algorithms select the learning rate empirically or heuristically due to the lack of a principled approach for learning rate calibration. Here, we develop a novel analytical calibration algorithm to optimally select the learning rate. The learning rate introduces a trade-off between the steady-state error and the convergence time of the estimated model parameters. Our calibration algorithm can keep the steady-state parameter error smaller than a desired value while minimizing the convergence time, or keep the convergence time faster than a desired value while minimizing the error. Using extensive closed-loop simulations, we show that the calibration algorithm allows for fast learning of accurate encoding models, and consequently for fast convergence of decoder performance to high values for both discrete-valued spike recordings and continuous-valued recordings such as local field potentials. The calibration algorithm can achieve a predictable level of speed and accuracy in adaptive learning, with significant implications for neurotechnologies.

General-purpose computers have become ubiquitous and important for everyday life, but they are difficult for people with paralysis to use. Specialized software and personalized input devices can improve access, but often provide only limited functionality. In this study, three research participants with tetraplegia who had multielectrode arrays implanted in motor cortex as part of the BrainGate2 clinical trial used an intracortical brain-computer interface (iBCI) to control an unmodified commercial tablet computer. Neural activity was decoded in real time as a point-and-click wireless Bluetooth mouse, allowing participants to use common and recreational applications (web browsing, email, chatting, playing music on a piano application, sending text messages, etc.). Two of the participants also used the iBCI to "chat" with each other in real time. This study demonstrates, for the first time, high-performance iBCI control of an unmodified, commercially available, general-purpose mobile computing device by people with tetraplegia.

In vision science, cascades of Linear+Nonlinear transforms are very successful in modeling a number of perceptual experiences. However, the conventional literature is usually too focused on only describing the forward input-output transform. Instead, in this work we present the mathematics of such cascades beyond the forward transform, namely the Jacobian matrices and the inverse. The fundamental reason for this analytical treatment is that it offers useful analytical insight into the psychophysics, the physiology, and the function of the visual system. For instance, we show how the trends of the sensitivity (volume of the discrimination regions) and the adaptation of the receptive fields can be identified in the expression of the Jacobian w.r.t. the stimulus. This matrix also tells us which regions of the stimulus space are encoded more efficiently in multi-information terms. The Jacobian w.r.t. the parameters shows which aspects of the model have bigger impact in the response, and hence their relative relevance. The analytic inverse implies conditions for the response and model parameters to ensure appropriate decoding. From the experimental and applied perspective, (a) the Jacobian w.r.t. the stimulus is necessary in new experimental methods based on the synthesis of visual stimuli with interesting geometrical properties, (b) the Jacobian matrices w.r.t. the parameters are convenient to learn the model from classical experiments or alternative goal optimization, and (c) the inverse is a promising model-based alternative to blind machine-learning methods for neural decoding that do not include meaningful biological information. The theory is checked by building and testing a vision model that actually follows a modular Linear+Nonlinear program. Our illustrative derivable and invertible model consists of a cascade of modules that account for brightness, contrast, energy masking, and wavelet masking. To stress the generality of this modular setting we show examples where some of the canonical Divisive Normalization modules are substituted by equivalent modules such as the Wilson-Cowan interaction model (at the V1 cortex) or a tone-mapping model (at the retina)., This work was partially funded by the Spanish Ministerio de Economia y Competitividad projects CICYT TEC2013-50520-EXP and CICYT BFU2014-59776-R, by the European Research Council, Starting Grant ref. 306337, by the Spanish government and FEDER Fund, grant ref. TIN2015-71537-P(MINECO/FEDER,UE), 1021, and by the ICREA Academia Award.

Studies of neuron-behaviour correlation and causal manipulation have long been used separately to understand the neural basis of perception. Yet these approaches sometimes lead to drastically conflicting conclusions about the functional role of brain areas. Theories that focus only on choice-related neuronal activity cannot reconcile those findings without additional experiments involving large-scale recordings to measure interneuronal correlations. By expanding current theories of neural coding and incorporating results from inactivation experiments, we demonstrate here that it is possible to infer decoding weights of different brain areas at a coarse scale without precise knowledge of the correlation structure. We apply this technique to neural data collected from two different cortical areas in macaque monkeys trained to perform a heading discrimination task. We identify two opposing decoding schemes, each consistent with data depending on the nature of correlated noise. Our theory makes specific testable predictions to distinguish these scenarios experimentally without requiring measurement of the underlying noise correlations., Author summary The neocortex is structurally organized into distinct brain areas. The role of specific brain areas in sensory perception is typically studied using two kinds of laboratory experiments: those that measure correlations between neural activity and reported percepts, and those that inactivate a brain region and measure the resulting changes in percepts. The two types of experiments have generally been interpreted in isolation, in part because no theory has been able combine their outcomes. Here, we describe a mathematical framework that synthesizes both kinds of results, giving us a new way to assess how different brain areas contribute to perception. When we apply our framework to experiments on behaving monkeys, we discover two models that can explain the perplexing finding that one brain area can predict an animal’s reported percepts, even though the percepts are not affected when that brain area is inactivated. The two models ascribe dramatically different efficiencies to brain computation. We show that these two models could be distinguished by a proposed experiment that measures correlations while inactivating different brain areas.

In human studies, how averaged activation in a brain region relates to human behavior has been extensively investigated. This approach has led to the finding that positive and negative facial preferences are represented by different brain regions. However, using a functional magnetic resonance imaging (fMRI) decoded neurofeedback (DecNef) method, we found that different patterns of neural activations within the cingulate cortex (CC) play roles in representing opposite directions of facial preference. In the present study, while neutrally preferred faces were presented, multi-voxel activation patterns in the CC that corresponded to higher (or lower) preference were repeatedly induced by fMRI DecNef. As a result, previously neutrally preferred faces became more (or less) preferred. We conclude that a different activation pattern in the CC, rather than averaged activation in a different area, represents and suffices to determine positive or negative facial preference. This new approach may reveal the importance of an activation pattern within a brain region in many cognitive functions., A newly developed fMRI method, decoded neurofeedback (DecNef), reveals that specific activation patterns in the cingulate cortex are largely responsible for determining human facial preferences., Author Summary Although it is well studied how averaged activation of a brain region relates to behavior, it is still unclear if specific patterns of activation within regions also relate to cognitive function. In recent years, several methods have been developed for manipulating brain activity in humans. Real-time functional magnetic resonance imaging decoded neurofeedback (fMRI DecNef) is a method that allows the induction of specific patterns of brain activity by measuring the current pattern, comparing this to the pattern to be induced, and giving the subjects feedback on how close the two patterns of neuronal activity are. Using fMRI DecNef, we manipulated the pattern of activation in the cingulate cortex—a part of the cerebral cortex that plays a role in preference to different categories including faces and daily items—and tested whether we could change these preferences. In the experiment, a specific activation pattern in the cingulate cortex corresponding to higher (or lower) preference was induced by fMRI DecNef while subjects were seeing a neutrally preferred face. As a result, these neutrally preferred faces became more (or less) preferred. Our finding suggests that different patterns of activation in the cingulate cortex represent, and are sufficient to determine, different emotional states. Our new approach using fMRI DecNef may reveal the importance of activation patterns within a brain region, rather than activation in a whole region, in many cognitive functions.

Individuals with tetraplegia lack independent mobility, making them highly dependent on others to move from one place to another. Here, we describe how two macaques were able to use a wireless integrated system to control a robotic platform, over which they were sitting, to achieve independent mobility using the neuronal activity in their motor cortices. The activity of populations of single neurons was recorded using multiple electrode arrays implanted in the arm region of primary motor cortex, and decoded to achieve brain control of the platform. We found that free-running brain control of the platform (which was not equipped with any machine intelligence) was fast and accurate, resembling the performance achieved using joystick control. The decoding algorithms can be trained in the absence of joystick movements, as would be required for use by tetraplegic individuals, demonstrating that the non-human primate model is a good pre-clinical model for developing such a cortically-controlled movement prosthetic. Interestingly, we found that the response properties of some neurons differed greatly depending on the mode of control (joystick or brain control), suggesting different roles for these neurons in encoding movement intention and movement execution. These results demonstrate that independent mobility can be achieved without first training on prescribed motor movements, opening the door for the implementation of this technology in persons with tetraplegia.

It is well known that signals encoded by mechanoreceptors facilitate precise object manipulation in humans. It is therefore of interest to study signals encoded by the mechanoreceptors because this will contribute further towards the understanding of fundamental sensory mechanisms that are responsible for coordinating force components during object manipulation. From a practical point of view, this may suggest strategies for designing sensory-controlled biomedical devices and robotic manipulators. We use a two-stage nonlinear decoding paradigm to reconstruct the force stimulus given signals from slowly adapting type one (SA-I) tactile afferents. First, we describe a nonhomogeneous Poisson encoding model which is a function of the force stimulus and the force's rate of change. In the decoding phase, we use a recursive nonlinear Bayesian filter to reconstruct the force profile, given the SA-I spike patterns and parameters described by the encoding model. Under the current encoding model, the mode ratio of force to its derivative is: 1.26 to 1.02. This indicates that the force derivative contributes significantly to the rate of change to the SA-I afferent spike modulation. Furthermore, using recursive Bayesian decoding algorithms is advantageous because it can incorporate past and current information in order to make predictions--consistent with neural systems--with little computational resources. This makes it suitable for interfacing with prostheses.

Neuroprosthetic brain-computer interfaces function via an algorithm which decodes neural activity of the user into movements of an end effector, such as a cursor or robotic arm. In practice, the decoder is often learned by updating its parameters while the user performs a task. When the user’s intention is not directly observable, recent methods have demonstrated value in training the decoder against a surrogate for the user’s intended movement. Here we show that training a decoder in this way is a novel variant of an imitation learning problem, where an oracle or expert is employed for supervised training in lieu of direct observations, which are not available. Specifically, we describe how a generic imitation learning meta-algorithm, dataset aggregation (DAgger), can be adapted to train a generic brain-computer interface. By deriving existing learning algorithms for brain-computer interfaces in this framework, we provide a novel analysis of regret (an important metric of learning efficacy) for brain-computer interfaces. This analysis allows us to characterize the space of algorithmic variants and bounds on their regret rates. Existing approaches for decoder learning have been performed in the cursor control setting, but the available design principles for these decoders are such that it has been impossible to scale them to naturalistic settings. Leveraging our findings, we then offer an algorithm that combines imitation learning with optimal control, which should allow for training of arbitrary effectors for which optimal control can generate goal-oriented control. We demonstrate this novel and general BCI algorithm with simulated neuroprosthetic control of a 26 degree-of-freedom model of an arm, a sophisticated and realistic end effector., Author Summary There are various existing methods for rapidly learning a decoder during closed-loop brain computer interface (BCI) tasks. While many of these methods work well in practice, there is no clear theoretical foundation for parameter learning. We offer a unification of closed-loop decoder learning setting as an imitation learning problem. This has two major consequences: first, our approach clarifies how to derive “intention-based” algorithms for any BCI setting, most notably more complex settings like control of an arm; and second, this framework allows us to provide theoretical results, building from an existing literature on the regret of related algorithms. After first demonstrating algorithmic performance in simulation on the well-studied setting of a user trying to reach targets by controlling a cursor on a screen, we then simulate a user controlling an arm with many degrees of freedom in order to grasp a wand. Finally, we describe how extensions in the online-imitation learning literature can improve BCI in additional settings.