Search

Tremendous neuroscientific progress has recently been made by mapping brain connectivity, complementing extensive knowledge of task-evoked brain activation patterns. However, despite evidence that they are related, these connectivity and activity lines of research have mostly progressed separately. Here I review the notable productivity and future promise of combining connectivity and task-evoked activity estimates into activity flow models. These data-driven computational models simulate the generation of task-evoked activations (including those linked to behavior), producing empirically-supported explanations of the origin of neurocognitive functions based on the flow of task-evoked activity over empirical brain connections. Critically, by incorporating causal principles and extensive empirical constraints from brain data, this approach can provide more mechanistic accounts of neurocognitive phenomena than purely predictive (as opposed to explanatory) models or models optimized primarily for task performance (e.g., standard artificial neural networks). The variety of activity-flow-based explanations reported so far are covered here along with important methodological and theoretical considerations when discovering new activity-flow-based explanations. Together, these considerations illustrate the promise of activity flow modeling for the future of neuroscience and ultimately for the development of novel clinical treatments (e.g., using brain stimulation) for brain disorders.

Humans have a remarkable ability to rapidly generalize to new tasks that is difficult to reproduce in artificial learning systems. Compositionality has been proposed as a key mechanism supporting generalization in humans, but evidence of its neural implementation and impact on behavior is still scarce. Here we study the computational properties associated with compositional generalization in both humans and artificial neural networks (ANNs) on a highly compositional task. First, we identified behavioral signatures of compositional generalization in humans, along with their neural correlates using whole-cortex functional magnetic resonance imaging (fMRI) data. Next, we designed pretraining paradigms aided by a procedure we term {\em primitives pretraining} to endow compositional task elements into ANNs. We found that ANNs with this prior knowledge had greater correspondence with human behavior and neural compositional signatures. Importantly, primitives pretraining induced abstract internal representations, excellent zero-shot generalization, and sample-efficient learning. Moreover, it gave rise to a hierarchy of abstract representations that matched human fMRI data, where sensory rule abstractions emerged in early sensory areas, and motor rule abstractions emerged in later motor areas. Our findings give empirical support to the role of compositional generalization in human behavior, implicate abstract representations as its neural implementation, and illustrate that these representations can be embedded into ANNs by designing simple and efficient pretraining procedures., Comment: NeurIPS 2022. 24 pages (10 pages main text), 17 figures (8 main text figures)

Computational Medical Extended Reality (CMXR), brings together life sciences and neuroscience with mathematics, engineering and computer science. It unifies computational science (scientific computing) with intelligent extended reality and spatial computing for the medical field. It significantly differs from previous "Clinical XR" or "Medical XR" terms, as it is focusing on how to integrate computational methods from neural simulation to computational geometry, computational vision and computer graphics with deep learning models to solve specific hard problems in medicine and neuroscience: from low/no-code/genAI authoring platforms to deep learning XR systems for training, planning, operative navigation, therapy and rehabilitation.

System identification poses a significant bottleneck to characterizing and controlling complex systems. This challenge is greatest when both the system states and parameters are not directly accessible leading to a dual-estimation problem. Current approaches to such problems are limited in their ability to scale with many-parameter systems as often occurs in networks. In the current work, we present a new, computationally efficient approach to treat large dual-estimation problems. Our approach consists of directly integrating pseudo-optimal state estimation (the Extended Kalman Filter) into a dual-optimization objective, leaving a differentiable cost/error function of only in terms of the unknown system parameters which we solve using numerical gradient/Hessian methods. Intuitively, our approach consists of solving for the parameters that generate the most accurate state estimator (Extended Kalman Filter). We demonstrate that our approach is at least as accurate in state and parameter estimation as joint Kalman Filters (Extended/Unscented), despite lower complexity. We demonstrate the utility of our approach by inverting anatomically-detailed individualized brain models from human magnetoencephalography (MEG) data., Comment: Conference

In this work, a neural network based terramechanics model and terrain estimator are presented with an outlook for optimal control applications such as model predictive control. Recognizing the limitations of the state-of-the-art terramechanics models in terms of operating conditions, computational cost, and continuous differentiability for gradient-based optimization, an efficient and twice continuously differentiable terramechanics model is developed using neural networks for dynamic operations on deformable terrains. It is demonstrated that the neural network terramechanics model is able to predict the lateral tire forces accurately and efficiently compared to the Soil Contact Model as a state-of-the-art model. Furthermore, the neural network terramechanics model is implemented within a terrain estimator and it is shown that using this model the estimator converges within around 2% of the true terrain parameter. Finally, with model predictive control applications in mind, which typically rely on bicycle models for their predictions, it is demonstrated that utilizing the estimated terrain parameter can reduce prediction errors of a bicycle model by orders of magnitude. The result is an efficient, dynamic, twice continuously differentiable terramechanics model and estimator that has inherent advantages for implementation in model predictive control as compared to previously established models.