Search

Cortical networks are complex systems of a great many interconnected neurons that operate from collective dynamical states. To understand how cortical neural networks function, it is important to identify their common dynamical operating states from the probabilistic viewpoint. Probabilistic characteristics of these operating states often underlie network functions. Here, using multi-electrode data from three separate experiments, we identify and characterize a cortical operating state (the "probability polling" or "p-polling" state), common across mouse and monkey with different behaviors. If the interaction among neurons is weak, the p-polling state provides a quantitative understanding of how the high dimensional probability distribution of firing patterns can be obtained by the low-order maximum entropy formulation, effectively utilizing a low dimensional stimulus-coding structure. These results show evidence for generality of the p-polling state and in certain situations its advantage of providing a mathematical validation for the low-order maximum entropy principle as a coding strategy.

Why heavily parameterized neural networks (NNs) do not overfit the data is an important long standing open question. We propose a phenomenological model of the NN training to explain this non-overfitting puzzle. Our linear frequency principle (LFP) model accounts for a key dynamical feature of NNs: they learn low frequencies first, irrespective of microscopic details. Theory based on our LFP model shows that low frequency dominance of target functions is the key condition for the non-overfitting of NNs and is verified by experiments. Furthermore, through an ideal two-layer NN, we unravel how detailed microscopic NN training dynamics statistically gives rise to a LFP model with quantitative prediction power., Comment: to appear in Chinese Physics Letters

Along with fruitful applications of Deep Neural Networks (DNNs) to realistic problems, recently, some empirical studies of DNNs reported a universal phenomenon of Frequency Principle (F-Principle): a DNN tends to learn a target function from low to high frequencies during the training. The F-Principle has been very useful in providing both qualitative and quantitative understandings of DNNs. In this paper, we rigorously investigate the F-Principle for the training dynamics of a general DNN at three stages: initial stage, intermediate stage, and final stage. For each stage, a theorem is provided in terms of proper quantities characterizing the F-Principle. Our results are general in the sense that they work for multilayer networks with general activation functions, population densities of data, and a large class of loss functions. Our work lays a theoretical foundation of the F-Principle for a better understanding of the training process of DNNs., under review

In many realistic systems, maximum entropy principle (MEP) analysis provides an effective characterization of the probability distribution of network states. However, to implement the MEP analysis, a sufficiently long-time data recording in general is often required, e.g., hours of spiking recordings of neurons in neuronal networks. The issue of whether the MEP analysis can be successfully applied to network systems with data from short recordings has yet to be fully addressed. In this work, we investigate relationships underlying the probability distributions, moments, and effective interactions in the MEP analysis and then show that, with short recordings of network dynamics, the MEP analysis can be applied to reconstructing probability distributions of network states under the condition of asynchronous activity of nodes in the network. Using spike trains obtained from both Hodgkin-Huxley neuronal networks and electrophysiological experiments, we verify our results and demonstrate that MEP analysis provides a tool to investigate the neuronal population coding properties, even for short recordings., Comment: 10 pages, 5 figures