Search

There is now substantial evidence for traveling waves and other structured spatiotemporal recurrent neural dynamics in cortical structures; but these observations have typically been difficult to reconcile with notions of topographically organized selectivity and feedforward receptive fields. We introduce a new 'spacetime' perspective on neural computation in which structured selectivity and dynamics are not contradictory but instead are complimentary. We show that spatiotemporal dynamics may be a mechanism by which natural neural systems encode approximate visual, temporal, and abstract symmetries of the world as conserved quantities, thereby enabling improved generalization and long-term working memory.

The capabilities of transformer networks such as ChatGPT and other Large Language Models (LLMs) have captured the world's attention. The crucial computational mechanism underlying their performance relies on transforming a complete input sequence - for example, all the words in a sentence - into a long "encoding vector" that allows transformers to learn long-range temporal dependencies in naturalistic sequences. Specifically, "self-attention" applied to this encoding vector enhances temporal context in transformers by computing associations between pairs of words in the input sequence. We suggest that waves of neural activity traveling across single cortical areas or multiple regions at the whole-brain scale could implement a similar encoding principle. By encapsulating recent input history into a single spatial pattern at each moment in time, cortical waves may enable temporal context to be extracted from sequences of sensory inputs, the same computational principle used in transformers., Comment: 27 pages, 5 figures

Chimera states are a phenomenon in which order and disorder can co-exist within a network that is fully homogeneous. Precisely how transient chimeras emerge in finite networks of Kuramoto oscillators with phase-lag remains unclear. Utilizing an operator-based framework to study nonlinear oscillator networks at finite scale, we reveal the spatiotemporal impact of the adjacency matrix eigenvectors on the Sakaguchi-Kuramoto dynamics. We identify a specific condition for the emergence of transient chimeras in these finite networks: the eigenvectors of the network adjacency matrix create a combination of a zero phase-offset mode and low spatial frequency waves traveling in opposite directions. This combination of eigenvectors leads directly to the coherent and incoherent clusters in the chimera. This approach provides two specific analytical predictions: (1) a precise formula predicting the combination of connectivity and phase-lag that creates transient chimeras, (2) a mathematical procedure for rewiring arbitrary networks to produce transient chimeras.

Traveling waves of neural activity have been observed throughout the brain at a diversity of regions and scales; however, their precise computational role is still debated. One physically inspired hypothesis suggests that the cortical sheet may act like a wave-propagating system capable of invertibly storing a short-term memory of sequential stimuli through induced waves traveling across the cortical surface, and indeed many experimental results from neuroscience correlate wave activity with memory tasks. To date, however, the computational implications of this idea have remained hypothetical due to the lack of a simple recurrent neural network architecture capable of exhibiting such waves. In this work, we introduce a model to fill this gap, which we denote the Wave-RNN (wRNN), and demonstrate how such an architecture indeed efficiently encodes the recent past through a suite of synthetic memory tasks where wRNNs learn faster and reach significantly lower error than wave-free counterparts. We further explore the implications of this memory storage system on more complex sequence modeling tasks such as sequential image classification and find that wave-based models not only again outperform comparable wave-free RNNs while using significantly fewer parameters, but additionally perform comparably to more complex gated architectures such as LSTMs and GRUs.

Networks with different levels of interactions, including multilayer and multiplex networks, can display a rich diversity of dynamical behaviors and can be used to model and study a wide range of systems. Despite numerous efforts to investigate these networks, obtaining mathematical descriptions for the dynamics of multilayer and multiplex systems is still an open problem. Here, we combine ideas and concepts from linear algebra and graph theory with nonlinear dynamics to offer a novel approach to study multiplex networks of Kuramoto oscillators. Our approach allows us to study the dynamics of a large, multiplex network by decomposing it into two smaller systems: one representing the connection scheme within layers (intra-layer), and the other representing the connections between layers (inter-layer). Particularly, we use this approach to compose solutions for multiplex networks of Kuramoto oscillators. These solutions are given by a combination of solutions for the smaller systems given by the intra and inter-layer system and, in addition, our approach allows us to study the linear stability of these solutions.

Metastability, characterized by a variability of regimes in time, is a ubiquitous type of neural dynamics. It has been formulated in many different ways in the neuroscience literature, however, which may cause some confusion. In this Perspective, we discuss metastability from the point of view of dynamical systems theory. We extract from the literature a very simple but general definition through the concept of metastable regimes as long-lived but transient epochs of activity with unique dynamical properties. This definition serves as an umbrella term that encompasses formulations from other works, and readily connects to concepts from dynamical systems theory. This allows us to examine general dynamical properties of metastable regimes, propose in a didactic manner several dynamics-based mechanisms that generate them, and discuss a theoretical tool to characterize them quantitatively. This perspective leads to insights that help to address issues debated in the literature and also suggest pathways for future research., Comment: 4 figures

Modeling the dynamics of the formation and evolution of protostellar disks as well as the history of stellar mass accretion typically involve the numerical solution of complex systems of coupled differential equations. The resulting mass accretion history of protostars is known to be highly episodic due to recurrent instabilities and also exhibits short timescale flickering. By leveraging the strong predictive abilities of neural networks, we extract some of the critical temporal dynamics experienced during the mass accretion including periods of instability. Particularly, we utilize a novel form of the Echo-State Neural Network (ESN), which has been shown to efficiently deal with data having inherent nonlinearity. We introduce the use of Optimized-ESN (Opt-ESN) to make model-independent time series forecasting of mass accretion rate in the evolution of protostellar disks. We apply the network to multiple hydrodynamic simulations with different initial conditions and exhibiting a variety of temporal dynamics to demonstrate the predictability of the Opt-ESN model. The model is trained on simulation data of $\sim 1-2$ Myr, and achieves predictions with a low normalized mean square error ($\sim 10^{-5}$ to $10^{-3}$) for forecasts ranging between 100 and 3800 yr. This result shows the promise of the application of machine learning based models to time-domain astronomy., Comment: 16 pages, 13 figures

Paley graphs form a nice link between the distribution of quadratic residues and graph theory. These graphs possess remarkable properties which make them useful in several branches of mathematics. Classically, for each prime number $p$ we can construct the corresponding Paley graph using quadratic and non-quadratic residues modulo $p$. Therefore, Paley graphs are naturally associated with the Legendre symbol at $p$ which is a quadratic Dirichlet character of conductor $p$. In this article, we introduce the generalized Paley graphs. These are graphs that are associated with a general quadratic Dirichlet character. We will then provide some of their basic properties. In particular, we describe their spectrum explicitly. We then use those generalized Paley graphs to construct some new families of Ramanujan graphs. Finally, using special values of $L$-functions, we provide an effective upper bound for their Cheeger number., Comment: To appear in Mathematica Slovaca

Although temporal coding through spike-time patterns has long been of interest in neuroscience, the specific structures that could be useful for spike-time codes remain highly unclear. Here, we introduce a new analytical approach, using techniques from discrete mathematics, to study spike-time codes. We focus on the phenomenon of ``phase precession'' in the rodent hippocampus. During navigation and learning on a physical track, specific cells in a rodent's brain form a highly structured pattern relative to the oscillation of local population activity. Studies of phase precession largely focus on its well established role in synaptic plasticity and memory formation. Comparatively less attention has been paid to the fact that phase precession represents one of the best candidates for a spike-time neural code. The precise nature of this code remains an open question. Here, we derive an analytical expression for an operator mapping points in physical space, through individual spike times, to complex numbers. The properties of this operator highlight a specific relationship between past and future in hippocampal spike patterns. Importantly, this approach generalizes beyond the specific phenomenon studied here, providing a new technique to study the neural codes within spike-time sequences found during sensory coding and motor behavior. We then introduce a novel spike-based decoding algorithm, based on this operator, that successfully decodes a simulated animal's trajectory using only the animal's initial position and a pattern of spike times. This decoder is robust to noise in spike times and works on a timescale almost an order of magnitude shorter than typically used with decoders that work on average firing rate. These results illustrate the utility of a discrete approach, based on the symmetries in spike patterns, to provide insight into the structure and function of neural systems., Comment: 11 pages, 5 figures

Networked systems have been used to model and investigate the dynamical behavior of a variety of systems. For these systems, different levels of complexity can be considered in the modeling procedure. On one hand, this can offer a more realistic and rich modeling option. On the other hand, it can lead to intrinsic difficulty in analyzing the system. Here, we present an approach to investigate the dynamics of Kuramoto oscillators on networks with different levels of connections: a network of networks. To do so, we utilize a construction in network theory known as the join of networks, which represents ``intra-area" and ``inter-area" connections. This approach provides a reduced representation of the original, multi-level system, where both systems have equivalent dynamics. Then, we can find solutions for the reduced system and broadcast them to the original network of networks. Moreover, using the same idea we can investigate the stability of these states, where we can obtain information on the Jacobian of the multi-level system by analyzing the reduced one. This approach is general for arbitrary connection schemes between nodes within the same area. Finally, our work opens the possibility of studying the dynamics of networked systems using a simpler representation, thus leading to a better understanding of the dynamical behavior of these systems.

Given a collection $\{ G_i\}_{i=1}^d$ of finite groups and a ring $R$, we define a subring of the ring $M_n(R)$ ($n = \sum_{i=1}^d|G_i|)$ that encompasses all the individual group rings $R[G_i]$ along the diagonal blocks as $G_i$-circulant matrices. The precise definition of this ring was inspired by a construction in graph theory known as the joined union of graphs. We call this ring the join of group rings and denote it by $\mathcal{J}_{G_1,\dots, G_d}(R)$. In this paper, we present a systematic study of the algebraic structure of $\mathcal{J}_{G_1,\dots, G_d}(R)$. We show that it has a ring structure and characterize its center, group of units, and Jacobson radical. When $R=k$ is an algebraically closed field, we derive a formula for the number of irreducible modules over $\mathcal{J}_{G_1,\dots, G_d}(k)$. We also show how a blockwise extension of the Fourier transform provides both a generalization of the Circulant Diagonalization Theorem to joins of circulant matrices and an explicit isomorphism between the join algebra and its Wedderburn components., Comment: 33 pages, Accepted for publication in the Journal of Pure and Applied Algebra

Understanding the sensitivity of a system's behavior with respect to parameter changes is essential for many applications. This sensitivity may be desired - for instance in the brain, where a large repertoire of different dynamics, particularly different synchronization patterns, is crucial - or may be undesired - for instance in power grids, where disruptions to synchronization may lead to blackouts. In this work, we show that the dynamics of networks of phase oscillators can acquire a very large and complex sensitivity to changes made in either their units' parameters or in their connections - even modifications made to a parameter of a single unit can radically alter the global dynamics of the network in an unpredictable manner. As a consequence, each modification leads to a different path to phase synchronization manifested as large fluctuations along that path. This dynamical malleability occurs over a wide parameter region, around the network's two transitions to phase synchronization. One transition is induced by increasing the coupling strength between the units, and another is induced by increasing the prevalence of long-range connections. Specifically, we study Kuramoto phase oscillators connected under either Watts-Strogatz or distance-dependent topologies to analyze the statistical properties of the fluctuations along the paths to phase synchrony. We argue that this increase in the dynamical malleability is a general phenomenon, as suggested by both previous studies and the theory of phase transitions., Comment: 14 pages, 8 figures

We introduce an analytical approach that allows predictions and mechanistic insights into the dynamics of nonlinear oscillator networks with heterogeneous time delays. We demonstrate that time delays shape the spectrum of a matrix associated to the system, leading to the emergence of waves with a preferred direction. We then create analytical predictions for the specific spatiotemporal patterns observed in individual simulations of time-delayed Kuramoto networks. This approach generalizes to systems with heterogeneous time delays at finite scales, which permits the study of spatiotemporal dynamics in a broad range of applications.

We study the spectrum of the join of several normal matrices with constant row sums. We apply our results to compute the characteristic polynomial of the join of several regular graphs. We then use this calculation to study several problems in spectral graph theory. In particular, we provide some simple constructions of Ramanujan graphs and give new proofs for some theorems in [4]., Comment: Comments are welcome

Let $A$ be a matrix of size $n \times n$ over an algebraically closed field $F$ and $q(t)$ a monic polynomial of degree $n$. In this article, we describe the necessary and sufficient conditions of $q(t)$ so that there exists a rank one matrix $B$ such that the characteristic polynomial of $A+B$ is $q(t)$., Comment: Comments are welcome!

We study the spectrum of the join of several circulant matrices. We apply our results to compute explicitly the spectrum of certain graphs obtained by joining several circulant graphs., Comment: To appear in Linear Algebra and Its Applications

Kuramoto networks constitute a paradigmatic model for the investigation of collective behavior in networked systems. Despite many advances in recent years, many open questions remain on the solutions for systems composed of coupled Kuramoto oscillators on complex networks. In this article, we describe an algebraic method to find equilibrium points for this kind of system without using standard approximations in the limit of infinite system size or the continuum limit. To do this, we use a recently introduced algebraic approach to the Kuramoto dynamics, which results in an explicitly solvable complex-valued equation that captures the dynamics of the original Kuramoto model. Using this new approach, we obtain equilibria for both the nonlinear original Kuramoto and complex-valued systems. We then completely classify all equilibria in the case of complete graphs originally studied by Kuramoto. Finally, we go on to study equilibria in networks of coupled oscillators with phase lag, in generalized circulant networks, multi-layer networks, and also random networks., Comment: To appear in SIAM Journal on Applied Dynamical Systems (SIADS)

One of the simplest mathematical models in the study of nonlinear systems is the Kuramoto model, which describes synchronization in systems from swarms of insects to superconductors. We have recently found a connection between the original, real-valued nonlinear Kuramoto model and a corresponding complex-valued system that permits describing the system in terms of a linear operator and iterative update rule. We now use this description to investigate three major synchronization phenomena in Kuramoto networks (phase synchronization, chimera states, and traveling waves), not only in terms of steady state solutions but also in terms of transient dynamics and individual simulations. These results provide new mathematical insight into how sophisticated behaviors arise from connection patterns in nonlinear networked systems., Comment: Final version, published in Chaos: An Interdisciplinary Journal of Nonlinear Science

We study the Kuramoto model with attractive sine coupling. We introduce a complex-valued matrix formulation whose argument coincides with the original Kuramoto dynamics. We derive an exact solution for the complex-valued model, which permits analytical insight into individual realizations of the Kuramoto model. The existence of a complex-valued form of the Kuramoto model provides a key demonstration that, in some cases, re-formulations of nonlinear dynamics in higher-order number fields may provide tractable analytical approaches., Comment: Published in Physical Review E, Volume. 104, No. 2

We propose an approach to calculate the critical percolation threshold for finite-sized Erdos-Renyi digraphs using minimal Hamiltonian cycles. We obtain an analytically exact result, valid non-asymptotically for all graph sizes, which scales in accordance with results obtained for infinite random graphs using the emergence of a giant connected component as marking the percolation transition. Our approach is general and can be applied to all graph models for which an algebraic formulation of the adjacency matrix is available., Comment: 4 pages, 3 figures

Advancing the size and complexity of neural network models leads to an ever increasing demand for computational resources for their simulation. Neuromorphic devices offer a number of advantages over conventional computing architectures, such as high emulation speed or low power consumption, but this usually comes at the price of reduced configurability and precision. In this article, we investigate the consequences of several such factors that are common to neuromorphic devices, more specifically limited hardware resources, limited parameter configurability and parameter variations. Our final aim is to provide an array of methods for coping with such inevitable distortion mechanisms. As a platform for testing our proposed strategies, we use an executable system specification (ESS) of the BrainScaleS neuromorphic system, which has been designed as a universal emulation back-end for neuroscientific modeling. We address the most essential limitations of this device in detail and study their effects on three prototypical benchmark network models within a well-defined, systematic workflow. For each network model, we start by defining quantifiable functionality measures by which we then assess the effects of typical hardware-specific distortion mechanisms, both in idealized software simulations and on the ESS. For those effects that cause unacceptable deviations from the original network dynamics, we suggest generic compensation mechanisms and demonstrate their effectiveness. Both the suggested workflow and the investigated compensation mechanisms are largely back-end independent and do not require additional hardware configurability beyond the one required to emulate the benchmark networks in the first place. We hereby provide a generic methodological environment for configurable neuromorphic devices that are targeted at emulating large-scale, functional neural networks.

We propose an explicit representation of central $(2k+1)$-nomial coefficients in terms of finite sums over trigonometric constructs. The approach utilizes the diagonalization of circulant boolean matrices and is generalizable to all $(2k+1)$-nomial coefficients, thus yielding a new family of combinatorical identities., Comment: 5 pages

In the past two decades, significant advances have been made in understanding the structural and functional properties of biological networks, via graph-theoretic analysis. In general, most graph-theoretic studies are conducted in the presence of serious uncertainties, such as major undersampling of the experimental data. In the specific case of neural systems, however, a few moderately robust experimental reconstructions do exist, and these have long served as fundamental prototypes for studying connectivity patterns in the nervous system. In this paper, we provide a comparative analysis of these "historical" graphs, both in (unmodified) directed and (often symmetrized) undirected forms, and focus on simple structural characterizations of their connectivity. We find that in most measures the networks studied are captured by simple random graph models; in a few key measures, however, we observe a marked departure from the random graph prediction. Our results suggest that the mechanism of graph formation in the networks studied is not well-captured by existing abstract graph models, such as the small-world or scale-free graph., Comment: 19 pages, 7 figures

The number of connected components and the size of the largest connected component are studied under node and edge removal in the connectivity graph of the C. elegans nervous system. By studying the two subgraphs - the directed graph of chemical synapses and the undirected graph of electrical junctions - we observe that adding a small number of undirected edges dramatically reduces the number of components in the complete graph. Under random node and edge removal, the C. elegans graph displays a remarkable structural robustness. We then compare these results with the vulnerability of a number of canonical graph models., Comment: 4 pages, 4 figures

In this paper we present a methodological framework that meets novel requirements emerging from upcoming types of accelerated and highly configurable neuromorphic hardware systems. We describe in detail a device with 45 million programmable and dynamic synapses that is currently under development, and we sketch the conceptual challenges that arise from taking this platform into operation. More specifically, we aim at the establishment of this neuromorphic system as a flexible and neuroscientifically valuable modeling tool that can be used by non-hardware-experts. We consider various functional aspects to be crucial for this purpose, and we introduce a consistent workflow with detailed descriptions of all involved modules that implement the suggested steps: The integration of the hardware interface into the simulator-independent model description language PyNN; a fully automated translation between the PyNN domain and appropriate hardware configurations; an executable specification of the future neuromorphic system that can be seamlessly integrated into this biology-to-hardware mapping process as a test bench for all software layers and possible hardware design modifications; an evaluation scheme that deploys models from a dedicated benchmark library, compares the results generated by virtual or prototype hardware devices with reference software simulations and analyzes the differences. The integration of these components into one hardware-software workflow provides an ecosystem for ongoing preparative studies that support the hardware design process and represents the basis for the maturity of the model-to-hardware mapping software. The functionality and flexibility of the latter is proven with a variety of experimental results.