Search

Animals make decisions under the principle of reward value maximization and surprise minimization. It is still unclear how these principles are represented in the brain and are reflected in behavior. We addressed this question using a closed-loop virtual reality system to train adult zebrafish for active avoidance. Analysis of the neural activity of the dorsal pallium during training revealed neural ensembles assigning rules to the colors of the surrounding walls. Additionally, one third of fish generated another ensemble that becomes activated only when the real perceived scenery shows discrepancy from the predicted favorable scenery. The fish with the latter ensemble escape more efficiently than the fish with the former ensembles alone, even though both fish have successfully learned to escape, consistent with the hypothesis that the latter ensemble guides zebrafish to take action to minimize this prediction error. Our results suggest that zebrafish can use both principles of goal-directed behavior, but with different behavioral consequences depending on the repertoire of the adopted principles., Using a closed-loop virtual reality system for fish, the authors show that zebrafish are capable of assigning rules to the scenery they see, and of generating a state prediction error by comparing reality with a prediction derived from an internal model.

Empirical applications of the free-energy principle are not straightforward because they entail a commitment to a particular process theory, especially at the cellular and synaptic levels. Using a recently established reverse engineering technique, we confirm the quantitative predictions of the free-energy principle using in vitro networks of rat cortical neurons that perform causal inference. Upon receiving electrical stimuli—generated by mixing two hidden sources—neurons self-organised to selectively encode the two sources. Pharmacological up- and downregulation of network excitability disrupted the ensuing inference, consistent with changes in prior beliefs about hidden sources. As predicted, changes in effective synaptic connectivity reduced variational free energy, where the connection strengths encoded parameters of the generative model. In short, we show that variational free energy minimisation can quantitatively predict the self-organisation of neuronal networks, in terms of their responses and plasticity. These results demonstrate the applicability of the free-energy principle to in vitro neural networks and establish its predictive validity in this setting.

The neuronal substrates that implement the free-energy principle and ensuing active inference at the neuron and synapse level have not been fully elucidated. This Review considers possible neuronal substrates underlying the principle. First, the foundations of the free-energy principle are introduced, and then its ability to empirically explain various brain functions and psychological and biological phenomena in terms of Bayesian inference is described. Mathematically, the dynamics of neural activity and plasticity that minimise a cost function can be cast as performing Bayesian inference that minimises variational free energy. This equivalence licenses the adoption of the free-energy principle as a universal characterisation of neural networks. Further, the neural network structure itself represents a generative model under which an agent operates. A virtue of this perspective is that it enables the formal association of neural network properties with prior beliefs that regulate inference and learning. The possible neuronal substrates that implement prior and posterior beliefs and how to empirically examine the theory are discussed. This perspective renders brain activity explainable, leading to a deeper understanding of the neuronal mechanisms underlying basic psychology and psychiatric disorders in terms of an implicit generative model.

This paper considers the emergence of a generalised synchrony in ensembles of coupled self-organising systems, such as neurons. We start from the premise that any self-organising system complies with the free energy principle, in virtue of placing an upper bound on its entropy. Crucially, the free energy principle allows one to interpret biological systems as inferring the state of their environment or external milieu. An emergent property of this inference is synchronisation among an ensemble of systems that infer each other. Here, we investigate the implications of neuronal dynamics by simulating neuronal networks, where each neuron minimises its free energy. We cast the ensuing ensemble dynamics in terms of inference and show that cardinal behaviours of neuronal networks – both in vivo and in vitro – can be explained by this framework. In particular, we test the hypotheses that (i) generalised synchrony is an emergent property of free energy minimisation; thereby explaining synchronisation in the resting brain: (ii) desynchronisation is induced by exogenous input; thereby explaining event-related desynchronisation and (iii) structure learning emerges in response to causal structure in exogenous input; thereby explaining functional segregation in real neuronal systems.

This work considers a class of canonical neural networks comprising rate coding models, wherein neural activity and plasticity minimise a common cost function—and plasticity is modulated with a certain delay. We show that such neural networks implicitly perform active inference and learning to minimise the risk associated with future outcomes. Mathematical analyses demonstrate that this biological optimisation can be cast as maximisation of model evidence, or equivalently minimisation of variational free energy, under the well-known form of a partially observed Markov decision process model. This equivalence indicates that the delayed modulation of Hebbian plasticity—accompanied with adaptation of firing thresholds—is a sufficient neuronal substrate to attain Bayes optimal inference and control. We corroborated this proposition using numerical analyses of maze tasks. This theory offers a universal characterisation of canonical neural networks in terms of Bayesian belief updating and provides insight into the neuronal mechanisms underlying planning and adaptive behavioural control., Takuya Isomura, Hideaki Shimazaki and Karl Friston perform mathematical analysis to show that neural networks implicitly perform active inference and learning to minimise the risk associated with future outcomes. Their work provides insight into the neuronal mechanisms underlying planning and adaptive behavioural control.

For many years, a combination of principal component analysis (PCA) and independent component analysis (ICA) has been used for blind source separation (BSS). However, it remains unclear why these linear methods work well with real-world data that involve nonlinear source mixtures. This work theoretically validates that a cascade of linear PCA and ICA can solve a nonlinear BSS problem accurately -- when the sensory inputs are generated from hidden sources via nonlinear mappings with sufficient dimensionality. Our proposed theorem, termed the asymptotic linearization theorem, theoretically guarantees that applying linear PCA to the inputs can reliably extract a subspace spanned by the linear projections from every hidden source as the major components -- and thus projecting the inputs onto their major eigenspace can effectively recover a linear transformation of the hidden sources. Then, subsequent application of linear ICA can separate all the true independent hidden sources accurately. Zero-element-wise-error nonlinear BSS is asymptotically attained when the source dimensionality is large and the input dimensionality is sufficiently larger than the source dimensionality. Our proposed theorem is validated analytically and numerically. Moreover, the same computation can be performed by using Hebbian-like plasticity rules, implying the biological plausibility of this nonlinear BSS strategy. Our results highlight the utility of linear PCA and ICA for accurately and reliably recovering nonlinearly mixed sources -- and further suggest the importance of employing sensors with sufficient dimensionality to identify true hidden sources of real-world data.

Generalization of time series prediction remains an important open issue in machine learning; earlier methods have either large generalization errors or local minima. Here, we develop an analytically solvable, unsupervised learning scheme that extracts the most informative components for predicting future inputs, which we call predictive principal component analysis (PredPCA). Our scheme can effectively remove unpredictable noise and minimize test prediction error through convex optimization. Mathematical analyses demonstrate that, provided with sufficient training samples and sufficiently high-dimensional observations, PredPCA can asymptotically identify hidden states, system parameters and dimensionalities of canonical nonlinear generative processes, with a global convergence guarantee. We demonstrate the performance of PredPCA using sequential visual inputs comprising handwritten digits, rotating three-dimensional objects and natural scenes. It reliably estimates distinct hidden states and predicts future outcomes of previously unseen test input data, based exclusively on noisy observations. The simple architecture and low computational cost of PredPCA are highly desirable for neuromorphic hardware. Predicting what comes next in a previously unseen time series of input data is a challenging task for machine learning. A novel unsupervised learning scheme termed predictive principal component analysis can extract the most informative components for predicting future inputs with low computational cost.

To exhibit social intelligence, animals have to recognize whom they are communicating with. One way to make this inference is to select among internal generative models of each conspecific who may be encountered. However, these models also have to be learned via some form of Bayesian belief updating. This induces an interesting problem: When receiving sensory input generated by a particular conspecific, how does an animal know which internal model to update? We consider a theoretical and neurobiologically plausible solution that enables inference and learning of the processes that generate sensory inputs (e.g., listening and understanding) and reproduction of those inputs (e.g., talking or singing), under multiple generative models. This is based on recent advances in theoretical neurobiology—namely, active inference and post hoc (online) Bayesian model selection. In brief, this scheme fits sensory inputs under each generative model. Model parameters are then updated in proportion to the probability that each model could have generated the input (i.e., model evidence). The proposed scheme is demonstrated using a series of (real zebra finch) birdsongs, where each song is generated by several different birds. The scheme is implemented using physiologically plausible models of birdsong production. We show that generalized Bayesian filtering, combined with model selection, leads to successful learning across generative models, each possessing different parameters. These results highlight the utility of having multiple internal models when making inferences in social environments with multiple sources of sensory information.

This work considers a class of biologically plausible cost functions for neural networks, where the same cost function is minimised by both neural activity and plasticity. We show that such cost functions can be cast as a variational bound on model evidence under an implicit generative model. Using generative models based on Markov decision processes (MDP), we show, analytically, that neural activity and plasticity perform Bayesian inference and learning, respectively, by maximising model evidence. Using mathematical and numerical analyses, we then confirm that biologically plausible cost functions—used in neural networks—correspond to variational free energy under some prior beliefs about the prevalence of latent states that generate inputs. These prior beliefs are determined by particular constants (i.e., thresholds) that define the cost function. This means that the Bayes optimal encoding of latent or hidden states is achieved when, and only when, the network’s implicit priors match the process that generates the inputs. Our results suggest that when a neural network minimises its cost function, it is implicitly minimising variational free energy under optimal or sub-optimal prior beliefs. This insight is potentially important because it suggests that any free parameter of a neural network’s cost function can itself be optimised—by minimisation with respect to variational free energy.

We consider suppression of macroscopic synchronized oscillations in mixed populations of active and inactive oscillators with local diffusive coupling, described by a lattice complex Ginzburg-Landau model with discrete Laplacian in general dimensions. Approximate expression for the stability of the non-oscillatory stationary state is derived on the basis of the generalized free energy of the system. We show that an effective wavenumber of the system determined by the spatial arrangement of the active and inactive oscillators is an decisive factor in the suppression, in addition to the ratio of active population to inactive population and relative intensity of each population. The effectiveness of the proposed theory is illustrated with a cortico-thalamic model of epileptic seizures, where active and inactive oscillators correspond to epileptic foci and healthy cerebral cortex tissue, respectively., Comment: 20 pages, 5 figures

SummaryHuman predicts future. To ask if fish also has this capacity, we established the virtual reality training system with live imaging of the telencephalic neurons of adult zebrafish in the active avoidance and found that, at the onset of the trial, learned fish conceives two future conditions as the favorable status on its way to the safe goal, i.e. one with the backwardly moving landscape and the other with the color of the safe goal. And the two different neural ensembles monitor the discrepancy between these predictions and the perceived real external status. Once fish reaches the goal, another ensemble is set to work to monitor whether fish keeps staying in the safe goal. The manipulation to artificially enhance these prediction errors elevated the activities of these ensembles and induced fish to behave to correct errors, revealing that fish sets behavioral strategy to actively realize these predictions.

The free-energy principle is a candidate unified theory for learning and memory in the brain that predicts that neurons, synapses, and neuromodulators work in a manner that minimizes free energy. However, electrophysiological data elucidating the neural and synaptic bases for this theory are lacking. Here, we propose a novel theory bridging the information-theoretical principle with the biological phenomenon of spike-timing dependent plasticity (STDP) regulated by neuromodulators, which we term mSTDP. We propose that by integrating an mSTDP equation, we can obtain a form of Friston’s free energy (an information-theoretical function). Then we analytically and numerically show that dopamine (DA) and noradrenaline (NA) influence the accuracy of a principal component analysis (PCA) performed using the mSTDP algorithm. From the perspective of free-energy minimization, these neuromodulatory changes alter the relative weighting or precision of accuracy and prior terms, which induces a switch from pattern completion to separation. These results are consistent with electrophysiological findings and validate the free-energy principle and mSTDP. Moreover, our scheme can potentially be applied in computational psychiatry to build models of the faulty neural networks that underlie the positive symptoms of schizophrenia, which involve abnormal DA levels, as well as models of the NA contribution to memory triage and posttraumatic stress disorder.

We developed a biologically plausible unsupervised learning algorithm, error-gated Hebbian rule (EGHR)-β, that performs principal component analysis (PCA) and independent component analysis (ICA) in a single-layer feedforward neural network. If parameter β = 1, it can extract the subspace that major principal components span similarly to Oja’s subspace rule for PCA. If β = 0, it can separate independent sources similarly to Bell-Sejnowski’s ICA rule but without requiring the same number of input and output neurons. Unlike these engineering rules, the EGHR-β can be easily implemented in a biological or neuromorphic circuit because it only uses local information available at each synapse. We analytically and numerically demonstrate the reliability of the EGHR-β in extracting and separating major sources given high-dimensional input. By adjusting β, the EGHR-β can extract sources that are missed by the conventional engineering approach that first applies PCA and then ICA. Namely, the proposed rule can successfully extract hidden natural images even in the presence of dominant or non-Gaussian noise components. The results highlight the reliability and utility of the EGHR-β for large-scale parallel computation of PCA and ICA and its future implementation in a neuromorphic hardware.

We have flexible control over our cognition depending on the context or surrounding environments. The prefrontal cortex (PFC) controls this cognitive flexibility; however, the detailed underlying mechanisms remain unclear. Recent developments in machine learning techniques have allowed simple recurrent neural network PFC models to perform human- or animal-like behavioral tasks. These systems allow us to acquire parameters, which we could not in biological experiments, for performing the tasks. We compared four models, in which a flexible cognition task, called context-dependent integration task, was performed; subsequently, we searched for common features. In all the models, we observed that high plastic synapses were concentrated in the small neuronal population and the more concentrated neuronal units contributed further to the performance. However, there were no common properties in the constructed structures. These results suggest that plastic changes can be more general and important to accomplish cognitive tasks than features of the constructed structures.

Background Synchrony within neuronal networks is thought to be a fundamental feature of neuronal networks. In order to quantify synchrony between spike trains, various synchrony measures were developed. Most of them are time scale dependent and thus require the setting of an appropriate time scale. Recently, alternative methods have been developed, such as the time scale independent SPIKE-distance by Kreuz et al. New method In this study, a novel time-scale independent spike train synchrony measure called Spike-contrast is proposed. The algorithm is based on the temporal “contrast” (activity vs. non-activity in certain temporal bins) and not only provides a single synchrony value, but also a synchrony curve as a function of the bin size. Results For most test data sets synchrony values obtained with Spike-contrast are highly correlated with those of the SPIKE-distance (Spearman correlation value of 0.99). Correlation was lower for data containing multiple time scales (Spearman correlation value of 0.89). When analyzing large sets of data, Spike-contrast performed faster. Comparison of existing method Spike-contrast is compared to the SPIKE-distance algorithm. The test data consisted of artificial spike trains with various levels of synchrony, including Poisson spike trains and bursts, spike trains from simulated neuronal Izhikevich networks, and bursts made of smaller bursts (sub-bursts). Conclusions The high correlation of Spike-contrast with the established SPIKE-distance for most test data, suggests the suitability of the proposed measure. Both measures are complementary as SPIKE-distance provides a synchrony profile over time, whereas Spike-contrast provides a synchrony curve over bin size.

Synapse elimination and neurite pruning are essential processes for the formation of neuronal circuits. These regressive events depend on neural activity and occur in the early postnatal days known as the critical period, but what makes this temporal specificity is not well understood. One possibility is that the neural activities during the developmentally regulated shift of action of GABA inhibitory transmission lead to the critical period. Moreover, it has been reported that the shifting action of the inhibitory transmission on immature neurons overlaps with synapse elimination and neurite pruning and that increased inhibitory transmission by drug treatment could induce temporal shift of the critical period. However, the relationship among these phenomena remains unclear because it is difficult to experimentally show how the developmental shift of inhibitory transmission influences neural activities and whether the activities promote synapse elimination and neurite pruning. In this study, we modeled synapse elimination in neuronal circuits using the modified Izhikevich's model with functional shifting of GABAergic transmission. The simulation results show that synaptic pruning within a specified period like the critical period is spontaneously generated as a function of the developmentally shifting inhibitory transmission and that the specific firing rate and increasing synchronization of neural circuits are seen at the initial stage of the critical period. This temporal relationship was experimentally supported by an in vitro primary culture of rat cortical neurons in a microchannel on a multi-electrode array (MEA). The firing rate decreased remarkably between the 18–25 days in vitro (DIV), and following these changes in the firing rate, the neurite density was slightly reduced. Our simulation and experimental results suggest that decreasing neural activity due to developing inhibitory synaptic transmission could induce synapse elimination and neurite pruning at particular time such as the critical period. Additionally, these findings indicate that we can estimate the maturity level of inhibitory transmission and the critical period by measuring the firing rate and the degree of synchronization in engineered neural networks.

Synaptic plasticity is a central theme in neuroscience. A framework of three-factor learning rules provides a powerful abstraction, helping to navigate through the abundance of models of synaptic plasticity. It is well-known that the dopamine modulation of learning is related to reward, but theoretical models predict other functional roles of the modulatory third factor; it may encode errors for supervised learning, summary statistics of the population activity for unsupervised learning or attentional feedback. Specialized structures may be needed in order to generate and propagate third factors in the neural network.

SUMMARY Recently, there has been abundant research using multineuron recording, but there are many problems with extracting the features from the obtained spike time series, which are huge in volume and complex. Here we introduce a new method of estimating synaptic connection strengths between neurons by fitting to the Izhikevich model by maximum likelihood estimation. We demonstrate that our method can estimate connection strengths from spike time series given by a simulated neural ensemble and can estimate nonconnectivity between two independent cultured neuronal networks. These results suggest that our method is applicable to network and plasticity analysis of neuronal networks.

Humans can separately recognize independent sources when they sense their superposition. This decomposition is mathematically formulated as independent component analysis (ICA). While a few biologically plausible learning rules, so-called local learning rules, have been proposed to achieve ICA, their performance varies depending on the parameters characterizing the mixed signals. Here, we propose a new learning rule that is both easy to implement and reliable. Both mathematical and numerical analyses confirm that the proposed rule outperforms other local learning rules over a wide range of parameters. Notably, unlike other rules, the proposed rule can separate independent sources without any preprocessing, even if the number of sources is unknown. The successful performance of the proposed rule is then demonstrated using natural images and movies. We discuss the implications of this finding for our understanding of neuronal information processing and its promising applications to neuromorphic engineering.

Blind source separation is the computation underlying the cocktail party effect––a partygoer can distinguish a particular talker’s voice from the ambient noise. Early studies indicated that the brain might use blind source separation as a signal processing strategy for sensory perception and numerous mathematical models have been proposed; however, it remains unclear how the neural networks extract particular sources from a complex mixture of inputs. We discovered that neurons in cultures of dissociated rat cortical cells could learn to represent particular sources while filtering out other signals. Specifically, the distinct classes of neurons in the culture learned to respond to the distinct sources after repeating training stimulation. Moreover, the neural network structures changed to reduce free energy, as predicted by the free-energy principle, a candidate unified theory of learning and memory, and by Jaynes’ principle of maximum entropy. This implicit learning can only be explained by some form of Hebbian plasticity. These results are the first in vitro (as opposed to in silico) demonstration of neural networks performing blind source separation, and the first formal demonstration of neuronal self-organization under the free energy principle., Author Summary The ‘cocktail party’ effect is a phenomenon by which one is able to pick out and listen to a single person’s speech in a noisy room. In information engineering, this is termed blind source separation. Numerous computational studies demonstrate that simulated neural networks can perform blind source separation. However, if or how a living neural network learns to perform blind source separation remains unknown. Using a microelectrode array (MEA) system that allowed us to apply composite inputs and record responses from neurons throughout a cultured neural network, we discovered that even neurons in cultures of dissociated rat cortical cells can separate individual sources from a complex mixture of inputs in the absence of teacher signals. Given these findings, we then determined that the neural networks adapted to reduce free energy, as predicted by the free energy principle and Jaynes’ principle of maximum entropy. These results provide evidence that cultured neural networks can perform blind source separation and that they are governed by the free-energy principle, providing a compelling framework for understanding how the brain identifies and processes signals hidden in complex multivariate information.

The authors report the preparation of a free-standing film of the cholesteric blue phase II by photopolymerization, and investigate its fluorescence characteristics upon optically pumping the sample. Before polymerization, laser emission was observed from the long band-edge wavelength of the selective reflection band of the blue phase. After polymerization, laser emission was not observed from the band-edge wavelength, but appeared to have red-shifted possibly due to a distortion of the helix.

Connection strength estimation is widely used in detecting the topology of neuronal networks and assessing their synaptic plasticity. A recently proposed model-based method using the leaky integrate-and-fire model neuron estimates membrane potential from spike trains by calculating the maximum a posteriori (MAP) path. We further enhance the MAP path method using variational Bayes and dynamic causal modeling. Several simulations demonstrate that the proposed method can accurately estimate connection strengths with an error ratio of less than 20%. The results suggest that the proposed method can be an effective tool for detecting network structure and synaptic plasticity.