Search

While one can characterize mental health using questionnaires, such tools do not provide direct insight into the underlying biology. By linking approaches that visualize brain activity to questionnaires in the context of individualized prediction, we can gain new insights into the biology and behavioral aspects of brain health. Resting-state fMRI (rs-fMRI) can be used to identify biomarkers of these conditions and study patterns of abnormal connectivity. In this work, we estimate mental health quality for individual participants using static functional network connectivity (sFNC) data from rs-fMRI. The deep learning model uses the sFNC data as input to predict four categories of mental health quality and visualize the neural patterns indicative of each group. We used guided gradient class activation maps (guided Grad-CAM) to identify the most discriminative sFNC patterns. The effectiveness of this model was validated using the UK Biobank dataset, in which we showed that our approach outperformed four alternative models by 4-18% accuracy. The proposed model's performance evaluation yielded a classification accuracy of 76%, 78%, 88%, and 98% for the excellent, good, fair, and poor mental health categories, with poor mental health accuracy being the highest. The findings show distinct sFNC patterns across each group. The patterns associated with excellent mental health consist of the cerebellar-subcortical regions, whereas the most prominent areas in the poor mental health category are in the sensorimotor and visual domains. Thus the combination of rs-fMRI and deep learning opens a promising path for developing a comprehensive framework to evaluate and measure mental health. Moreover, this approach had the potential to guide the development of personalized interventions and enable the monitoring of treatment response. Overall this highlights the crucial role of advanced imaging modalities and deep learning algorithms in advancing our understanding and management of mental health., (© 2024. The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.)

Structural neuroimaging studies have identified a combination of shared and disorder-specific patterns of gray matter (GM) deficits across psychiatric disorders. Pooling large data allows for examination of a possible common neuroanatomical basis that may identify a certain vulnerability for mental illness. Large-scale collaborative research is already facilitated by data repositories, institutionally supported databases, and data archives. However, these data-sharing methodologies can suffer from significant barriers. Federated approaches augment these approaches by enabling access or more sophisticated, shareable and scaled-up analyses of large-scale data. We examined GM alterations using Collaborative Informatics and Neuroimaging Suite Toolkit for Anonymous Computation, an open-source, decentralized analysis application. Through federated analysis of eight sites, we identified significant overlap in the GM patterns ( n  = 4,102) of individuals with schizophrenia, major depressive disorder, and autism spectrum disorder. These results show cortical and subcortical regions that may indicate a shared vulnerability to psychiatric disorders., Competing Interests: The authors declare no competing interests.

In recent years, deep learning approaches have gained significant attention in predicting brain disorders using neuroimaging data. However, conventional methods often rely on single-modality data and supervised models, which provide only a limited perspective of the intricacies of the highly complex brain. Moreover, the scarcity of accurate diagnostic labels in clinical settings hinders the applicability of the supervised models. To address these limitations, we propose a novel self-supervised framework for extracting multiple representations from multimodal neuroimaging data to enhance group inferences and enable analysis without resorting to labeled data during pre-training. Our approach leverages Deep InfoMax (DIM), a self-supervised methodology renowned for its efficacy in learning representations by estimating mutual information without the need for explicit labels. While DIM has shown promise in predicting brain disorders from single-modality MRI data, its potential for multimodal data remains untapped. This work extends DIM to multimodal neuroimaging data, allowing us to identify disorder-relevant brain regions and explore multimodal links. We present compelling evidence of the efficacy of our multimodal DIM analysis in uncovering disorder-relevant brain regions, including the hippocampus, caudate, insula, - and multimodal links with the thalamus, precuneus, and subthalamus hypothalamus. Our self-supervised representations demonstrate promising capabilities in predicting the presence of brain disorders across a spectrum of Alzheimer's phenotypes. Comparative evaluations against state-of-the-art unsupervised methods based on autoencoders, canonical correlation analysis, and supervised models highlight the superiority of our proposed method in achieving improved classification performance, capturing joint information, and interpretability capabilities. The computational efficiency of the decoder-free strategy enhances its practical utility, as it saves compute resources without compromising performance. This work offers a significant step forward in addressing the challenge of understanding multimodal links in complex brain disorders, with potential applications in neuroimaging research and clinical diagnosis., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 The Author(s). Published by Elsevier Inc. All rights reserved.)

Purpose of Review: Over the last decade, evidence suggests that a combination of behavioral and neuroimaging findings can help illuminate changes in functional dysconnectivity in schizophrenia. We review the recent connectivity literature considering several vital models, considering connectivity findings, and relationships with clinical symptoms. We reviewed resting state fMRI studies from 2017 to 2023. We summarized the role of two sets of brain networks (cerebello-thalamo-cortical (CTCC) and the triple network set) across three hypothesized models of schizophrenia etiology (neurodevelopmental, vulnerability-stress, and neurotransmitter hypotheses)., Recent Findings: The neurotransmitter and neurodevelopmental models best explained CTCC-subcortical dysfunction, which was consistently connected to symptom severity and motor symptoms. Triple network dysconnectivity was linked to deficits in executive functioning, and the salience network (SN)-default mode network dysconnectivity was tied to disordered thought and attentional deficits. This paper links behavioral symptoms of schizophrenia (symptom severity, motor, executive functioning, and attentional deficits) to various hypothesized mechanisms., (© 2023. The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.)

This work proposes a novel generative multimodal approach to jointly analyze multimodal data while linking the multimodal information to colors. We apply our proposed framework, which disentangles multimodal data into private and shared sets of features from pairs of structural (sMRI), functional (sFNC and ICA), and diffusion MRI data (FA maps). With our approach, we find that heterogeneity in schizophrenia is potentially a function of modality pairs. Results show (1) schizophrenia is highly multimodal and includes changes in specific networks, (2) non-linear relationships with schizophrenia are observed when interpolating among shared latent dimensions, and (3) we observe a decrease in the modularity of functional connectivity and decreased visual-sensorimotor connectivity for schizophrenia patients for the FA-sFNC and sMRI-sFNC modality pairs, respectively. Additionally, our results generally indicate decreased fractional corpus callosum anisotropy, and decreased spatial ICA map and voxel-based morphometry strength in the superior frontal lobe as found in the FA-sFNC, sMRI-FA, and sMRI-ICA modality pair clusters. In sum, we introduce a powerful new multimodal neuroimaging framework designed to provide a rich and intuitive understanding of the data which we hope challenges the reader to think differently about how modalities interact., (© 2023 The Authors. Human Brain Mapping published by Wiley Periodicals LLC.)

With the growth of decentralized/federated analysis approaches in neuroimaging, the opportunities to study brain disorders using data from multiple sites has grown multi-fold. One such initiative is the Neuromark, a fully automated spatially constrained independent component analysis (ICA) that is used to link brain network abnormalities among different datasets, studies, and disorders while leveraging subject-specific networks. In this study, we implement the neuromark pipeline in COINSTAC, an open-source neuroimaging framework for collaborative/decentralized analysis. Decentralized exploratory analysis of nearly 2000 resting-state functional magnetic resonance imaging datasets collected at different sites across two cohorts and co-located in different countries was performed to study the resting brain functional network connectivity changes in adolescents who smoke and consume alcohol. Results showed hypoconnectivity across the majority of networks including sensory, default mode, and subcortical domains, more for alcohol than smoking, and decreased low frequency power. These findings suggest that global reduced synchronization is associated with both tobacco and alcohol use. This proof-of-concept work demonstrates the utility and incentives associated with large-scale decentralized collaborations spanning multiple sites., (© 2022. The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.)

Brain dynamics are highly complex and yet hold the key to understanding brain function and dysfunction. The dynamics captured by resting-state functional magnetic resonance imaging data are noisy, high-dimensional, and not readily interpretable. The typical approach of reducing this data to low-dimensional features and focusing on the most predictive features comes with strong assumptions and can miss essential aspects of the underlying dynamics. In contrast, introspection of discriminatively trained deep learning models may uncover disorder-relevant elements of the signal at the level of individual time points and spatial locations. Yet, the difficulty of reliable training on high-dimensional low sample size datasets and the unclear relevance of the resulting predictive markers prevent the widespread use of deep learning in functional neuroimaging. In this work, we introduce a deep learning framework to learn from high-dimensional dynamical data while maintaining stable, ecologically valid interpretations. Results successfully demonstrate that the proposed framework enables learning the dynamics of resting-state fMRI directly from small data and capturing compact, stable interpretations of features predictive of function and dysfunction., (© 2022. The Author(s).)

Dynamic functional network connectivity (dFNC) analysis is a widely used approach for capturing brain activation patterns, connectivity states, and network organization. However, a typical sliding window plus clustering (SWC) approach for analyzing dFNC models the system through a fixed sequence of connectivity states. SWC assumes connectivity patterns span throughout the brain, but they are relatively spatially constrained and temporally short-lived in practice. Thus, SWC is neither designed to capture transient dynamic changes nor heterogeneity across subjects/time. We propose a state-space time series summarization framework called "statelets" to address these shortcomings. It models functional connectivity dynamics at fine-grained timescales, adapting time series motifs to changes in connectivity strength, and constructs a concise yet informative representation of the original data that conveys easily comprehensible information about the phenotypes. We leverage the earth mover distance in a nonstandard way to handle scale differences and utilize kernel density estimation to build a probability density profile for local motifs. We apply the framework to study dFNC of patients with schizophrenia (SZ) and healthy control (HC). Results demonstrate SZ subjects exhibit reduced modularity in their brain network organization relative to HC. Statelets in the HC group show an increased recurrence across the dFNC time-course compared to the SZ. Analyzing the consistency of the connections across time reveals significant differences within visual, sensorimotor, and default mode regions where HC subjects show higher consistency than SZ. The introduced approach also enables handling dynamic information in cross-modal and multimodal applications to study healthy and disordered brains., (© 2022 The Authors. Human Brain Mapping published by Wiley Periodicals LLC.)

Privacy concerns for rare disease data, institutional or IRB policies, access to local computational or storage resources or download capabilities are among the reasons that may preclude analyses that pool data to a single site. A growing number of multisite projects and consortia were formed to function in the federated environment to conduct productive research under constraints of this kind. In this scenario, a quality control tool that visualizes decentralized data in its entirety via global aggregation of local computations is especially important, as it would allow the screening of samples that cannot be jointly evaluated otherwise. To solve this issue, we present two algorithms: decentralized data stochastic neighbor embedding, dSNE, and its differentially private counterpart, DP-dSNE. We leverage publicly available datasets to simultaneously map data samples located at different sites according to their similarities. Even though the data never leaves the individual sites, dSNE does not provide any formal privacy guarantees. To overcome that, we rely on differential privacy: a formal mathematical guarantee that protects individuals from being identified as contributors to a dataset. We implement DP-dSNE with AdaCliP, a method recently proposed to add less noise to the gradients per iteration. We introduce metrics for measuring the embedding quality and validate our algorithms on these metrics against their centralized counterpart on two toy datasets. Our validation on six multisite neuroimaging datasets shows promising results for the quality control tasks of visualization and outlier detection, highlighting the potential of our private, decentralized visualization approach., (© 2022 The Authors. Human Brain Mapping published by Wiley Periodicals LLC.)

Advances in imaging acquisition techniques allow multiple imaging modalities to be collected from the same subject. Each individual modality offers limited yet unique views of the functional, structural, or dynamic temporal features of the brain. Multimodal fusion provides effective ways to leverage these complementary perspectives from multiple modalities. However, the majority of current multimodal fusion approaches involving functional magnetic resonance imaging (fMRI) are limited to 3D feature summaries that do not incorporate its rich temporal information. Thus, we propose a novel three-way parallel group independent component analysis (pGICA) fusion method that incorporates the first-level 4D fMRI data (temporal information included) by parallelizing group ICA into parallel ICA via a unified optimization framework. A new variability matrix was defined to capture subject-wise functional variability and then link it to the mixing matrices of the other two modalities. Simulation results show that the three-way pGICA provides highly accurate cross-modality linkage estimation under both weakly and strongly correlated conditions, as well as comparable source estimation under different noise levels. Results using real brain imaging data identified one linked functional-structural-diffusion component associated to differences between schizophrenia and controls. This was replicated in an independent cohort, and the identified components were also correlated with major cognitive domains. Functional network connectivity revealed visual-subcortical and default mode-cerebellum pairs that discriminate between schizophrenia and controls. Overall, both simulation and real data results support the use of three-way pGICA to identify multimodal spatiotemporal links and to pursue the study of brain disorders under a single unifying multimodal framework., (© 2021 The Authors. Human Brain Mapping published by Wiley Periodicals LLC.)

The field of neuroimaging can greatly benefit from building machine learning models to detect and predict diseases, and discover novel biomarkers, but much of the data collected at various organizations and research centers is unable to be shared due to privacy or regulatory concerns (especially for clinical data or rare disorders). In addition, aggregating data across multiple large studies results in a huge amount of duplicated technical debt and the resources required can be challenging or impossible for an individual site to build. Training on the data distributed across organizations can result in models that generalize much better than models trained on data from any of organizations alone. While there are approaches for decentralized sharing, these often do not provide the highest possible guarantees of sample privacy that only cryptography can provide. In addition, such approaches are often focused on probabilistic solutions. In this paper, we propose an approach that leverages the potential of datasets spread among a number of data collecting organizations by performing joint analyses in a secure and deterministic manner when only encrypted data is shared and manipulated. The approach is based on secure multiparty computation which refers to cryptographic protocols that enable distributed computation of a function over distributed inputs without revealing additional information about the inputs. It enables multiple organizations to train machine learning models on their joint data and apply the trained models to encrypted data without revealing their sensitive data to the other parties. In our proposed approach, organizations (or sites) securely collaborate to build a machine learning model as it would have been trained on the aggregated data of all the organizations combined. Importantly, the approach does not require a trusted party (i.e. aggregator), each contributing site plays an equal role in the process, and no site can learn individual data of any other site. We demonstrate effectiveness of the proposed approach, in a range of empirical evaluations using different machine learning algorithms including logistic regression and convolutional neural network models on human structural and functional magnetic resonance imaging datasets., (© 2021. The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.)

There has been an upward trend in developing frameworks that enable neuroimaging researchers to address challenging questions by leveraging data across multiple sites all over the world. One such open-source framework is the Collaborative Informatics and Neuroimaging Suite Toolkit for Anonymous Computation (COINSTAC) that works on Windows, macOS, and Linux operating systems and leverages containerized analysis pipelines to analyze neuroimaging data stored locally across multiple physical locations without the need for pooling the data at any point during the analysis. In this paper, the COINSTAC team partnered with a data collection consortium to implement the first-ever decentralized voxelwise analysis of brain imaging data performed outside the COINSTAC development group. Decentralized voxel-based morphometry analysis of over 2000 structural magnetic resonance imaging data sets collected at 14 different sites across two cohorts and co-located in different countries was performed to study the structural changes in brain gray matter which linked to age, body mass index (BMI), and smoking. Results produced by the decentralized analysis were consistent with and extended previous findings in the literature. In particular, a widespread cortical gray matter reduction (resembling a 'default mode network' pattern) and hippocampal increase with age, bilateral increases in the hypothalamus and basal ganglia with BMI, and cingulate and thalamic decreases with smoking. This work provides a critical real-world test of the COINSTAC framework in a "Large-N" study. It showcases the potential benefits of performing multivoxel and multivariate analyses of large-scale neuroimaging data located at multiple sites., (© 2021. Springer Science+Business Media, LLC, part of Springer Nature.)

Aim: To determine the optimal number of connectivity states in dynamic functional connectivity analysis. Introduction: Recent work has focused on the study of dynamic (vs. static) brain connectivity in resting functional magnetic resonance imaging data. In this work, we focus on temporal correlation between time courses extracted from coherent networks called functional network connectivity (FNC). Dynamic FNC is most commonly estimated using a sliding window-based approach to capture short periods of FNC change. These data are then clustered to estimate transient connectivity patterns or states. Determining the number of states is a challenging problem. The elbow criterion is one of the widely used approaches to determine the connectivity states. Materials and Methods: In our work, we present an alternative approach that evaluates classification (e.g., healthy controls [HCs] vs. patients) as a measure to select the optimal number of states (clusters). We apply different classification strategies to perform classification between HCs and patients with schizophrenia for different numbers of states (i.e., varying the model order in the clustering algorithm). We compute cross-validated accuracy for different model orders to evaluate the classification performance. Results: Our results are consistent with our earlier work which shows that overall accuracy improves when dynamic connectivity measures are used separately or in combination with static connectivity measures. Results also show that the optimal model order for classification is different from that using the standard k -means model selection method, and that such optimization improves cross-validated accuracy. The optimal model order obtained from the proposed approach also gives significantly improved classification performance over the traditional model selection method. Conclusion: The observed results suggest that if one's goal is to perform classification, using the proposed approach as a criterion for selecting the optimal number of states in dynamic connectivity analysis leads to improved accuracy in hold-out data.

Unsupervised latent variable models-blind source separation (BSS) especially-enjoy a strong reputation for their interpretability. But they seldom combine the rich diversity of information available in multiple datasets, even though multidatasets yield insightful joint solutions otherwise unavailable in isolation. We present a direct, principled approach to multidataset combination that takes advantage of multidimensional subspace structures. In turn, we extend BSS models to capture the underlying modes of shared and unique variability across and within datasets. Our approach leverages joint information from heterogeneous datasets in a flexible and synergistic fashion. We call this method multidataset independent subspace analysis (MISA). Methodological innovations exploiting the Kotz distribution for subspace modeling, in conjunction with a novel combinatorial optimization for evasion of local minima, enable MISA to produce a robust generalization of independent component analysis (ICA), independent vector analysis (IVA), and independent subspace analysis (ISA) in a single unified model. We highlight the utility of MISA for multimodal information fusion, including sample-poor regimes ( N = 600 ) and low signal-to-noise ratio, promoting novel applications in both unimodal and multimodal brain imaging data.

Blind source separation algorithms such as independent component analysis (ICA) are widely used in the analysis of neuroimaging data. To leverage larger sample sizes, different data holders/sites may wish to collaboratively learn feature representations. However, such datasets are often privacy-sensitive, precluding centralized analyses that pool the data at one site. In this work, we propose a differentially private algorithm for performing ICA in a decentralized data setting. Due to the high dimension and small sample size, conventional approaches to decentralized differentially private algorithms suffer in terms of utility. When centralizing the data is not possible, we investigate the benefit of enabling limited collaboration in the form of generating jointly distributed random noise. We show that such (anti) correlated noise improves the privacy-utility trade-off, and can reach the same level of utility as the corresponding non-private algorithm for certain parameter choices. We validate this benefit using synthetic and real neuroimaging datasets. We conclude that it is possible to achieve meaningful utility while preserving privacy, even in complex signal processing systems.

As neuroimaging data increase in complexity and related analytical problems follow suite, more researchers are drawn to collaborative frameworks that leverage data sets from multiple data-collection sites to balance out the complexity with an increased sample size. Although centralized data-collection approaches have dominated the collaborative scene, a number of decentralized approaches-those that avoid gathering data at a shared central store-have grown in popularity. We expect the prevalence of decentralized approaches to continue as privacy risks and communication overhead become increasingly important for researchers. In this article, we develop, implement and evaluate a decentralized version of one such widely used tool: dynamic functional network connectivity. Our resulting algorithm, decentralized dynamic functional network connectivity (ddFNC), synthesizes a new, decentralized group independent component analysis algorithm (dgICA) with algorithms for decentralized k-means clustering. We compare both individual decentralized components and the full resulting decentralized analysis pipeline against centralized counterparts on the same data, and show that both provide comparable performance. Additionally, we perform several experiments which evaluate the communication overhead and convergence behavior of various decentralization strategies and decentralized clustering algorithms. Our analysis indicates that ddFNC is a fine candidate for facilitating decentralized collaboration between neuroimaging researchers, and stands ready for the inclusion of privacy-enabling modifications, such as differential privacy., (© 2020 The Authors. Human Brain Mapping published by Wiley Periodicals, Inc.)

Background: In this age of big data, certain models require very large data stores in order to be informative and accurate. In many cases however, the data are stored in separate locations requiring data transfer between local sites which can cause various practical hurdles, such as privacy concerns or heavy network load. This is especially true for medical imaging data, which can be constrained due to the health insurance portability and accountability act (HIPAA) which provides security protocols for medical data. Medical imaging datasets can also contain many thousands or millions of features, requiring heavy network load., New Method: Our research expands upon current decentralized classification research by implementing a new singleshot method for both neural networks and support vector machines. Our approach is to estimate the statistical distribution of the data at each local site and pass this information to the other local sites where each site resamples from the individual distributions and trains a model on both locally available data and the resampled data. The model for each local site produces its own accuracy value which are then averaged together to produce the global average accuracy., Results: We show applications of our approach to handwritten digit classification as well as to multi-subject classification of brain imaging data collected from patients with schizophrenia and healthy controls. Overall, the results showed comparable classification accuracy to the centralized model with lower network load than multishot methods., Comparison With Existing Methods: Many decentralized classifiers are multishot, requiring heavy network traffic. Our model attempts to alleviate this load while preserving prediction accuracy., Conclusions: We show that our proposed approach performs comparably to a centralized approach while minimizing network traffic compared to multishot methods., (Copyright © 2019. Published by Elsevier B.V.)

The field of neuroimaging has recently witnessed a strong shift towards data sharing; however, current collaborative research projects may be unable to leverage institutional architectures that collect and store data in local, centralized data centers. Additionally, though research groups are willing to grant access for collaborations, they often wish to maintain control of their data locally. These concerns may stem from research culture as well as privacy and accountability concerns. In order to leverage the potential of these aggregated larger data sets, we require tools that perform joint analyses without transmitting the data. Ideally, these tools would have similar performance and ease of use as their current centralized counterparts. In this paper, we propose and evaluate a new Algorithm, decentralized joint independent component analysis (djICA), which meets these technical requirements. djICA shares only intermediate statistics about the data, plausibly retaining privacy of the raw information to local sites, thus making it amenable to further privacy protections, for example via differential privacy. We validate our method on real functional magnetic resonance imaging (fMRI) data and show that it enables collaborative large-scale temporal ICA of fMRI, a rich vein of analysis as of yet largely unexplored, and which can benefit from the larger-N studies enabled by a decentralized approach. We show that djICA is robust to different distributions of data over sites, and that the temporal components estimated with djICA show activations similar to the temporal functional modes analyzed in previous work, thus solidifying djICA as a new, decentralized method oriented toward the frontiers of temporal independent component analysis., (Copyright © 2018 Elsevier Inc. All rights reserved.)

This work presents a novel approach to finding linkage/association between multimodal brain imaging data, such as structural MRI (sMRI) and functional MRI (fMRI). Motivated by the machine translation domain, we employ a deep learning model, and consider two different imaging views of the same brain like two different languages conveying some common facts. That analogy enables finding linkages between two modalities. The proposed translation-based fusion model contains a computing layer that learns "alignments" (or links) between dynamic connectivity features from fMRI data and static gray matter patterns from sMRI data. The approach is evaluated on a multi-site dataset consisting of eyes-closed resting state imaging data collected from 298 subjects (age- and gender matched 154 healthy controls and 144 patients with schizophrenia). Results are further confirmed on an independent dataset consisting of eyes-open resting state imaging data from 189 subjects (age- and gender matched 91 healthy controls and 98 patients with schizophrenia). We used dynamic functional connectivity (dFNC) states as the functional features and ICA-based sources from gray matter densities as the structural features. The dFNC states characterized by weakly correlated intrinsic connectivity networks (ICNs) were found to have stronger association with putamen and insular gray matter pattern, while the dFNC states of profuse strongly correlated ICNs exhibited stronger links with the gray matter pattern in precuneus, posterior cingulate cortex (PCC), and temporal cortex. Further investigation with the estimated link strength (or alignment score) showed significant group differences between healthy controls and patients with schizophrenia in several key regions including temporal lobe, and linked these to connectivity states showing less occupancy in healthy controls. Moreover, this novel approach revealed significant correlation between a cognitive score (attention/vigilance) and the function/structure alignment score that was not detected when data modalities were considered separately., (Copyright © 2018. Published by Elsevier Inc.)

We introduce a novel recurrent neural network (RNN) approach to account for temporal dynamics and dependencies in brain networks observed via functional magnetic resonance imaging (fMRI). Our approach directly parameterizes temporal dynamics through recurrent connections, which can be used to formulate blind source separation with a conditional (rather than marginal) independence assumption, which we call RNN-ICA. This formulation enables us to visualize the temporal dynamics of both first order (activity) and second order (directed connectivity) information in brain networks that are widely studied in a static sense, but not well-characterized dynamically. RNN-ICA predicts dynamics directly from the recurrent states of the RNN in both task and resting state fMRI. Our results show both task-related and group-differentiating directed connectivity.

In the field of neuroimaging, there is a growing interest in developing collaborative frameworks that enable researchers to address challenging questions about the human brain by leveraging data across multiple sites all over the world. Additionally, efforts are also being directed at developing algorithms that enable collaborative analysis and feature learning from multiple sites without requiring the often large data to be centrally located. In this paper, we propose two new decentralized algorithms: (1) A decentralized regression algorithm for performing a voxel-based morphometry analysis on structural magnetic resonance imaging (MRI) data and, (2) A decentralized dynamic functional network connectivity algorithm which includes decentralized group ICA and sliding-window analysis of functional MRI data. We compare results against those obtained from their pooled (or centralized) counterparts on the same data i.e., as if they are at one site. Results produced by the decentralized algorithms are similar to the pooled-case and showcase the potential of performing multi-voxel and multivariate analyses of data located at multiple sites. Such approaches enable many more collaborative and comparative analysis in the context of large-scale neuroimaging studies.

Music is ubiquitous throughout recent human culture, and many individual's have an innate ability to appreciate and understand music. Our appreciation of music likely emerges from the brain's ability to process a series of repeated complex acoustic patterns. In order to understand these processes further, cortical responses were measured to a series of guitar notes presented with a musical pattern or without a pattern. ERP responses to individual notes were measured using a 24 electrode Bluetooth mobile EEG system (Smarting mBrainTrain) while 13 healthy non-musicians listened to structured (i.e., within musical keys and with repetition) or random sequences of guitar notes for 10 min each. We demonstrate an increased amplitude to the ERP that appears ~200 ms to notes presented within the musical sequence. This amplitude difference between random notes and patterned notes likely reflects individual's cortical sensitivity to guitar note patterns. These amplitudes were compared to ERP responses to a rare note embedded within a stream of frequent notes to determine whether the sensitivity to complex musical structure overlaps with the sensitivity to simple irregularities reflected in traditional auditory oddball experiments. Response amplitudes to the negative peak at ~175 ms are statistically correlated with the mismatch negativity (MMN) response measured to a rare note presented among a series of frequent notes (i.e., in a traditional oddball sequence), but responses to the subsequent positive peak at ~200 do not show a statistical relationship with the P300 response. Thus, the sensitivity to musical structure identified to 4 Hz note patterns appears somewhat distinct from the sensitivity to statistical regularities reflected in the traditional "auditory oddball" sequence. Overall, we suggest that this is a promising approach to examine individual's sensitivity to complex acoustic patterns, which may overlap with higher level cognitive processes, including language.

Feedforward deep neural networks (DNNs), artificial neural networks with multiple hidden layers, have recently demonstrated a record-breaking performance in multiple areas of applications in computer vision and speech processing. Following the success, DNNs have been applied to neuroimaging modalities including functional/structural magnetic resonance imaging (MRI) and positron-emission tomography data. However, no study has explicitly applied DNNs to 3D whole-brain fMRI volumes and thereby extracted hidden volumetric representations of fMRI that are discriminative for a task performed as the fMRI volume was acquired. Our study applied fully connected feedforward DNN to fMRI volumes collected in four sensorimotor tasks (i.e., left-hand clenching, right-hand clenching, auditory attention, and visual stimulus) undertaken by 12 healthy participants. Using a leave-one-subject-out cross-validation scheme, a restricted Boltzmann machine-based deep belief network was pretrained and used to initialize weights of the DNN. The pretrained DNN was fine-tuned while systematically controlling weight-sparsity levels across hidden layers. Optimal weight-sparsity levels were determined from a minimum validation error rate of fMRI volume classification. Minimum error rates (mean±standard deviation; %) of 6.9 (±3.8) were obtained from the three-layer DNN with the sparsest condition of weights across the three hidden layers. These error rates were even lower than the error rates from the single-layer network (9.4±4.6) and the two-layer network (7.4±4.1). The estimated DNN weights showed spatial patterns that are remarkably task-specific, particularly in the higher layers. The output values of the third hidden layer represented distinct patterns/codes of the 3D whole-brain fMRI volume and encoded the information of the tasks as evaluated from representational similarity analysis. Our reported findings show the ability of the DNN to classify a single fMRI volume based on the extraction of hidden representations of fMRI volumes associated with tasks across multiple hidden layers. Our study may be beneficial to the automatic classification/diagnosis of neuropsychiatric and neurological diseases and prediction of disease severity and recovery in (pre-) clinical settings using fMRI volumes without requiring an estimation of activation patterns or ad hoc statistical evaluation., Competing Interests: The authors have no conflicts of interest regarding this study, including financial, consultant, institutional, or other relationships., (Copyright © 2016 Elsevier Inc. All rights reserved.)

In the past decade, numerous advances in the study of the human brain were fostered by successful applications of blind source separation (BSS) methods to a wide range of imaging modalities. The main focus has been on extracting "networks" represented as the underlying latent sources. While the broad success in learning latent representations from multiple datasets has promoted the wide presence of BSS in modern neuroscience, it also introduced a wide variety of objective functions, underlying graphical structures, and parameter constraints for each method. Such diversity, combined with a host of datatype-specific know-how, can cause a sense of disorder and confusion, hampering a practitioner's judgment and impeding further development. We organize the diverse landscape of BSS models by exposing its key features and combining them to establish a novel unifying view of the area. In the process, we unveil important connections among models according to their properties and subspace structures. Consequently, a high-level descriptive structure is exposed, ultimately helping practitioners select the right model for their applications. Equipped with that knowledge, we review the current state of BSS applications to neuroimaging. The gained insight into model connections elicits a broader sense of generalization, highlighting several directions for model development. In light of that, we discuss emerging multi-dataset multidimensional (MDM) models and summarize their benefits for the study of the healthy brain and disease-related changes.

Huntington disease (HD) is caused by an abnormally expanded cytosine-adenine-guanine (CAG) trinucleotide repeat in the HTT gene. Age and CAG-expansion number are related to age at diagnosis and can be used to index disease progression. However, observed onset-age variability suggests that other factors also modulate progression. Indexing prodromal (pre-diagnosis) progression may highlight therapeutic targets by isolating the earliest-affected factors. We present the largest prodromal HD application of the univariate method voxel-based morphometry (VBM) and the first application of the multivariate method source-based morphometry (SBM) to, respectively, compare gray matter concentration (GMC) and capture co-occurring GMC patterns in control and prodromal participants. Using structural MRI data from 1050 (831 prodromal, 219 control) participants, we characterize control-prodromal, whole-brain GMC differences at various prodromal stages. Our results provide evidence for (1) regional co-occurrence and differential patterns of decline across the prodrome, with parietal and occipital differences commonly co-occurring, and frontal and temporal differences being relatively independent from one another, (2) fronto-striatal circuits being among the earliest and most consistently affected in the prodrome, (3) delayed degradation in some movement-related regions, with increasing subcortical and occipital differences with later progression, (4) an overall superior-to-inferior gradient of GMC reduction in frontal, parietal, and temporal lobes, and (5) the appropriateness of SBM for studying the prodromal HD population and its enhanced sensitivity to early prodromal and regionally concurrent differences.

The field of neuroimaging has embraced the need for sharing and collaboration. Data sharing mandates from public funding agencies and major journal publishers have spurred the development of data repositories and neuroinformatics consortia. However, efficient and effective data sharing still faces several hurdles. For example, open data sharing is on the rise but is not suitable for sensitive data that are not easily shared, such as genetics. Current approaches can be cumbersome (such as negotiating multiple data sharing agreements). There are also significant data transfer, organization and computational challenges. Centralized repositories only partially address the issues. We propose a dynamic, decentralized platform for large scale analyses called the Collaborative Informatics and Neuroimaging Suite Toolkit for Anonymous Computation (COINSTAC). The COINSTAC solution can include data missing from central repositories, allows pooling of both open and "closed" repositories by developing privacy-preserving versions of widely-used algorithms, and incorporates the tools within an easy-to-use platform enabling distributed computation. We present an initial prototype system which we demonstrate on two multi-site data sets, without aggregating the data. In addition, by iterating across sites, the COINSTAC model enables meta-analytic solutions to converge to "pooled-data" solutions (i.e., as if the entire data were in hand). More advanced approaches such as feature generation, matrix factorization models, and preprocessing can be incorporated into such a model. In sum, COINSTAC enables access to the many currently unavailable data sets, a user friendly privacy enabled interface for decentralized analysis, and a powerful solution that complements existing data sharing solutions.

Linear independent component analysis (ICA) is a standard signal processing technique that has been extensively used on neuroimaging data to detect brain networks with coherent brain activity (functional MRI) or covarying structural patterns (structural MRI). However, its formulation assumes that the measured brain signals are generated by a linear mixture of the underlying brain networks and this assumption limits its ability to detect the inherent nonlinear nature of brain interactions. In this paper, we introduce nonlinear independent component estimation (NICE) to structural MRI data to detect abnormal patterns of gray matter concentration in schizophrenia patients. For this biomedical application, we further addressed the issue of model regularization of nonlinear ICA by performing dimensionality reduction prior to NICE, together with an appropriate control of the complexity of the model and the usage of a proper approximation of the probability distribution functions of the estimated components. We show that our results are consistent with previous findings in the literature, but we also demonstrate that the incorporation of nonlinear associations in the data enables the detection of spatial patterns that are not identified by linear ICA. Specifically, we show networks including basal ganglia, cerebellum and thalamus that show significant differences in patients versus controls, some of which show distinct nonlinear patterns.

In this paper we propose a web-based approach for quick visualization of big data from brain magnetic resonance imaging (MRI) scans using a combination of an automated image capture and processing system, nonlinear embedding, and interactive data visualization tools. We draw upon thousands of MRI scans captured via the COllaborative Imaging and Neuroinformatics Suite (COINS). We then interface the output of several analysis pipelines based on structural and functional data to a t-distributed stochastic neighbor embedding (t-SNE) algorithm which reduces the number of dimensions for each scan in the input data set to two dimensions while preserving the local structure of data sets. Finally, we interactively display the output of this approach via a web-page, based on data driven documents (D3) JavaScript library. Two distinct approaches were used to visualize the data. In the first approach, we computed multiple quality control (QC) values from pre-processed data, which were used as inputs to the t-SNE algorithm. This approach helps in assessing the quality of each data set relative to others. In the second case, computed variables of interest (e.g., brain volume or voxel values from segmented gray matter images) were used as inputs to the t-SNE algorithm. This approach helps in identifying interesting patterns in the data sets. We demonstrate these approaches using multiple examples from over 10,000 data sets including (1) quality control measures calculated from phantom data over time, (2) quality control data from human functional MRI data across various studies, scanners, sites, (3) volumetric and density measures from human structural MRI data across various studies, scanners and sites. Results from (1) and (2) show the potential of our approach to combine t-SNE data reduction with interactive color coding of variables of interest to quickly identify visually unique clusters of data (i.e., data sets with poor QC, clustering of data by site) quickly. Results from (3) demonstrate interesting patterns of gray matter and volume, and evaluate how they map onto variables including scanners, age, and gender. In sum, the proposed approach allows researchers to rapidly identify and extract meaningful information from big data sets. Such tools are becoming increasingly important as datasets grow larger.

Multi-subject or group-level component analysis provides a data-driven approach to study properties of brain networks. Algorithms for group-level data decomposition of functional magnetic resonance imaging data have been brought forward more than a decade ago and have significantly matured since. Similar applications for electroencephalographic data are at a comparatively early stage of development though, and their sensitivity to topographic variability of the electroencephalogram or loose time-locking of neuronal responses has not yet been assessed. This study investigates the performance of independent component analysis (ICA) and second order blind source identification (SOBI) for data decomposition, and their combination with either temporal or spatial concatenation of data sets, for multi-subject analyses of electroencephalographic data. Analyses of simulated sources with different spatial, frequency, and time-locking profiles, revealed that temporal concatenation of data sets with either ICA or SOBI served well to reconstruct sources with both strict and loose time-locking, whereas performance decreased in the presence of topographical variability. The opposite pattern was found with a spatial concatenation of subject-specific data sets. This study proofs that procedures for group-level decomposition of electroencephalographic data can be considered valid and promising approaches to infer the latent structure of multi-subject data sets. Yet, specific implementations need further adaptations to optimally address sources of inter-subject and inter-trial variance commonly found in EEG recordings.

Identifying the complex activity relationships present in rich, modern neuroimaging data sets remains a key challenge for neuroscience. The problem is hard because (a) the underlying spatial and temporal networks may be nonlinear and multivariate and (b) the observed data may be driven by numerous latent factors. Further, modern experiments often produce data sets containing multiple stimulus contexts or tasks processed by the same subjects. Fusing such multi-session data sets may reveal additional structure, but raises further statistical challenges. We present a novel analysis method for extracting complex activity networks from such multifaceted imaging data sets. Compared to previous methods, we choose a new point in the trade-off space, sacrificing detailed generative probability models and explicit latent variable inference in order to achieve robust estimation of multivariate, nonlinear group factors ("network clusters"). We apply our method to identify relationships of task-specific intrinsic networks in schizophrenia patients and control subjects from a large fMRI study. After identifying network-clusters characterized by within- and between-task interactions, we find significant differences between patient and control groups in interaction strength among networks. Our results are consistent with known findings of brain regions exhibiting deviations in schizophrenic patients. However, we also find high-order, nonlinear interactions that discriminate groups but that are not detected by linear, pairwise methods. We additionally identify high-order relationships that provide new insights into schizophrenia but that have not been found by traditional univariate or second-order methods. Overall, our approach can identify key relationships that are missed by existing analysis methods, without losing the ability to find relationships that are known to be important., (© 2013.)

Multimodal fusion is becoming more common as it proves to be a powerful approach to identify complementary information from multimodal datasets. However, simulation of joint information is not straightforward. Published approaches mostly employ limited, provisional designs that often break the link between the model assumptions and the data for the sake of demonstrating properties of fusion techniques. This work introduces a new approach to synthetic data generation which allows full-compliance between data and model while still representing realistic spatiotemporal features in accordance with the current neuroimaging literature. The focus is on the simulation of joint information for the verification of stochastic linear models, particularly those used in multimodal data fusion of brain imaging data. Our first goal is to obtain a benchmark ground-truth in which estimation errors due to model mismatch are minimal or none. Then we move on to assess how estimation is affected by gradually increasing model discrepancies toward a more realistic dataset. The key aspect of our approach is that it permits complete control over the type and level of model mismatch, allowing for more educated inferences about the limitations and caveats of select stochastic linear models. As a result, impartial comparison of models is possible based on their performance in multiple different scenarios. Our proposed method uses the commonly overlooked theory of copulas to enable full control of the type and level of dependence/association between modalities, with no occurrence of spurious multimodal associations. Moreover, our approach allows for arbitrary single-modality marginal distributions for any fixed choice of dependence/association between multimodal features. Using our simulation framework, we can rigorously challenge the assumptions of several existing multimodal fusion approaches. Our study brings a new perspective to the problem of simulating multimodal data that can be used for ground-truth verification of various stochastic multimodal models available in the literature, and reveals some important aspects that are not captured or are overlooked by ad hoc simulations that lack a firm statistical motivation., (Copyright © 2014 Elsevier Inc. All rights reserved.)

Deep learning methods have recently made notable advances in the tasks of classification and representation learning. These tasks are important for brain imaging and neuroscience discovery, making the methods attractive for porting to a neuroimager's toolbox. Success of these methods is, in part, explained by the flexibility of deep learning models. However, this flexibility makes the process of porting to new areas a difficult parameter optimization problem. In this work we demonstrate our results (and feasible parameter ranges) in application of deep learning methods to structural and functional brain imaging data. These methods include deep belief networks and their building block the restricted Boltzmann machine. We also describe a novel constraint-based approach to visualizing high dimensional data. We use it to analyze the effect of parameter choices on data transformations. Our results show that deep learning methods are able to learn physiologically important representations and detect latent relations in neuroimaging data.

Matrix factorization models are the current dominant approach for resolving meaningful data-driven features in neuroimaging data. Among them, independent component analysis (ICA) is arguably the most widely used for identifying functional networks, and its success has led to a number of versatile extensions to group and multimodal data. However there are indications that ICA may have reached a limit in flexibility and representational capacity, as the majority of such extensions are case-driven, custom-made solutions that are still contained within the class of mixture models. In this work, we seek out a principled and naturally extensible approach and consider a probabilistic model known as a restricted Boltzmann machine (RBM). An RBM separates linear factors from functional brain imaging data by fitting a probability distribution model to the data. Importantly, the solution can be used as a building block for more complex (deep) models, making it naturally suitable for hierarchical and multimodal extensions that are not easily captured when using linear factorizations alone. We investigate the capability of RBMs to identify intrinsic networks and compare its performance to that of well-known linear mixture models, in particular ICA. Using synthetic and real task fMRI data, we show that RBMs can be used to identify networks and their temporal activations with accuracy that is equal or greater than that of factorization models. The demonstrated effectiveness of RBMs supports its use as a building block for deeper models, a significant prospect for future neuroimaging research., (Copyright © 2014 Elsevier Inc. All rights reserved.)

Behavioral inhibition often is studied by comparing the electroencephalographic responses to stop and to go signals. Most studies simply assess amplitude differences of the N200 and P300 event-related potentials, which seem to best correspond to increased activity in the theta and delta frequency bands, respectively. However, neither have reliable indicators for successful behavioral inhibition been identified nor have the causal dependencies of stop-related neurocognitive processes been addressed yet. By studying functional and effective connectivity underlying stopping behavior, this study opens new directions for the investigation of behavioral inhibition. Group independent component analysis was used to infer functionally coherent networks from electroencephalographic data, which were recorded from healthy human participants during processing of a stop signal task. Then, the temporal dynamics of causal dependencies between independent components were identified by means of Bayesian network estimations. The mean clustering coefficient and the characteristic path length measure indicated time windows between 130 and 180 ms and between 420 and 500 ms to express significantly different connectivity profiles between conditions. Three components showed significant correlations between 120 and 260 ms with stop signal reaction times and the number of failed stops. Two of these components acted as sources of causal flow, one capturing P300/delta characteristics while the other was characterized by alpha power depletion putatively representing the evaluation or processing of stimulus features. Although results suggest that the P300 and associated delta activity seem to be statistically dependent on earlier processes associated with behavioral inhibition, the time window critical for inhibition coincides with early changes in causal patterns and largely precedes peak amplitude differences between go and stop trials. Altogether, utilizing the analysis of stopping-related connectivity, previously undetected patterns emerged that warrant further investigation., (Copyright © 2014 Elsevier Inc. All rights reserved.)

The growth of data sharing initiatives for neuroimaging and genomics represents an exciting opportunity to confront the "small N" problem that plagues contemporary neuroimaging studies while further understanding the role genetic markers play in the function of the brain. When it is possible, open data sharing provides the most benefits. However, some data cannot be shared at all due to privacy concerns and/or risk of re-identification. Sharing other data sets is hampered by the proliferation of complex data use agreements (DUAs) which preclude truly automated data mining. These DUAs arise because of concerns about the privacy and confidentiality for subjects; though many do permit direct access to data, they often require a cumbersome approval process that can take months. An alternative approach is to only share data derivatives such as statistical summaries-the challenges here are to reformulate computational methods to quantify the privacy risks associated with sharing the results of those computations. For example, a derived map of gray matter is often as identifiable as a fingerprint. Thus alternative approaches to accessing data are needed. This paper reviews the relevant literature on differential privacy, a framework for measuring and tracking privacy loss in these settings, and demonstrates the feasibility of using this framework to calculate statistics on data distributed at many sites while still providing privacy.

Spontaneous fluctuations are a hallmark of recordings of neural signals, emergent over time scales spanning milliseconds and tens of minutes. However, investigations of intrinsic brain organization based on resting-state functional magnetic resonance imaging have largely not taken into account the presence and potential of temporal variability, as most current approaches to examine functional connectivity (FC) implicitly assume that relationships are constant throughout the length of the recording. In this work, we describe an approach to assess whole-brain FC dynamics based on spatial independent component analysis, sliding time window correlation, and k-means clustering of windowed correlation matrices. The method is applied to resting-state data from a large sample (n = 405) of young adults. Our analysis of FC variability highlights particularly flexible connections between regions in lateral parietal and cingulate cortex, and argues against a labeling scheme where such regions are treated as separate and antagonistic entities. Additionally, clustering analysis reveals unanticipated FC states that in part diverge strongly from stationary connectivity patterns and challenge current descriptions of interactions between large-scale networks. Temporal trends in the occurrence of different FC states motivate theories regarding their functional roles and relationships with vigilance/arousal. Overall, we suggest that the study of time-varying aspects of FC can unveil flexibility in the functional coordination between different neural systems, and that the exploitation of these dynamics in further investigations may improve our understanding of behavioral shifts and adaptive processes.

A recent paper by Daubechies et al. claims that two independent component analysis (ICA) algorithms, Infomax and FastICA, which are widely used for functional magnetic resonance imaging (fMRI) analysis, select for sparsity rather than independence. The argument was supported by a series of experiments on synthetic data. We show that these experiments fall short of proving this claim and that the ICA algorithms are indeed doing what they are designed to do: identify maximally independent sources.

Information must integrate from multiple brain areas in healthy cognition and perception. The present study examined the extent to which cortical responses within one sensory modality are modulated by a complex task conducted within another sensory modality. Electroencephalographic (EEG) responses were measured to a 40 Hz auditory stimulus while individuals attended to modulations in the amplitude of the 40 Hz stimulus, and as a function of the difficulty of the popular computer game Tetris. The steady-state response to the 40 Hz stimulus was isolated by Fourier analysis of the EEG. The response at the stimulus frequency was normalized by the response within the surrounding frequencies, generating the signal-to-noise ratio (SNR). Seven out of eight individuals demonstrate a monotonic increase in the log SNR of the 40 Hz responses going from the difficult visuospatial task to the easy visuospatial task to attending to the auditory stimuli. This pattern is represented statistically by a One-Way ANOVA, indicating significant differences in log SNR across the three tasks. The sensitivity of 40 Hz auditory responses to the visuospatial load was further demonstrated by a significant correlation between log SNR and the difficulty (i.e., speed) of the Tetris task. Thus, the results demonstrate that 40 Hz auditory cortical responses are influenced by an individual's goal-directed attention to the stimulus, and by the degree of difficulty of a complex visuospatial task.

Altered brain connectivity has emerged as a central feature of schizophrenia. Low frequency oscillations and connectivity strength (CS) of resting state brain networks are altered in patients with schizophrenia (SZs). However, the relationship between these two measures has not yet been studied. Such work may be helpful in understanding the so-called "rich club" organization (i.e. high-CS nodes are more densely connected among themselves than are nodes of a lower CS in the human brain) in healthy controls (HCs) and SZs. Here we present a study of HCs and SZs examining low frequency oscillations and CS by first decomposing resting state fMRI (R-fMRI) data into independent components (ICs) using group independent component analysis (ICA) and computing the low frequency power ratio (LFPR) of each ICA time course. Weighted brain graphs consisting of ICs were built based on correlations between ICA time courses. Positive CS and negative CS of each node in the brain graphs were then examined. The correlations between LFPR and CSs as well as "rich club" coefficients of group mean brain graphs were assessed. Results demonstrate that the LFPR of some ICs were lower in SZs compared to HCs. In addition, LFPR was correlated with positive CS in HCs, but to a lesser extent in SZs. HCs showed higher normalized rich club parameter than SZs. The findings provide new insight into disordered intrinsic brain graphs in schizophrenia., (Copyright © 2012 Elsevier B.V. All rights reserved.)

Neuroimaging studies have shown that functional brain networks composed from select regions of interest have a modular community structure. However, the organization of functional network connectivity (FNC), comprising a purely data-driven network built from spatially independent brain components, is not yet clear. The aim of this study is to explore the modular organization of FNC in both healthy controls (HCs) and patients with schizophrenia (SZs). Resting state functional magnetic resonance imaging data of HCs and SZs were decomposed into independent components (ICs) by group independent component analysis (ICA). Then weighted brain networks (in which nodes are brain components) were built based on correlations between ICA time courses. Clustering coefficients and connectivity strength of the networks were computed. A dynamic branch cutting algorithm was used to identify modules of the FNC in HCs and SZs. Results show stronger connectivity strength and higher clustering coefficient in HCs with more and smaller modules in SZs. In addition, HCs and SZs had some different hubs. Our findings demonstrate altered modular architecture of the FNC in schizophrenia and provide insights into abnormal topological organization of intrinsic brain networks in this mental illness.

Estimation of effective connectivity, a measure of the influence among brain regions, can potentially reveal valuable information about organization of brain networks. Effective connectivity is usually evaluated from the functional data of a single modality. In this paper we show why that may lead to incorrect conclusions about effective connectivity. In this paper we use Bayesian networks to estimate connectivity on two different modalities. We analyze structures of estimated effective connectivity networks using aggregate statistics from the field of complex networks. Our study is conducted on functional MRI and magnetoencephalography data collected from the same subjects under identical paradigms. Results showed some similarities but also revealed some striking differences in the conclusions one would make on the fMRI data compared with the MEG data and are strongly supportive of the use of multiple modalities in order to gain a more complete picture of how the brain is organized given the limited information one modality is able to provide., (2011 Elsevier Ltd. All rights reserved.)

Non-negative matrix factorization (NMF) is a problem of decomposing multivariate data into a set of features and their corresponding activations. When applied to experimental data, NMF has to cope with noise, which is often highly correlated. We show that correlated noise can break the Donoho and Stodden separability conditions of a dataset and a regular NMF algorithm will fail to decompose it, even when given freedom to be able to represent the noise as a separate feature. To cope with this issue, we present an algorithm for NMF with a generalized least squares objective function (glsNMF) and derive multiplicative updates for the method together with proving their convergence. The new algorithm successfully recovers the true representation from the noisy data. Robust performance can make glsNMF a valuable tool for analyzing empirical data.

The combined analysis of magnetoencephalography (MEG)/electroencephalography and functional magnetic resonance imaging (fMRI) measurements can lead to improvement in the description of the dynamical and spatial properties of brain activity. In this paper we empirically demonstrate this improvement using simulated and recorded task related MEG and fMRI activity. Neural activity estimates were derived using a dynamic Bayesian network with continuous real valued parameters by means of a sequential Monte Carlo technique. In synthetic data, we show that MEG and fMRI fusion improves estimation of the indirectly observed neural activity and smooths tracking of the blood oxygenation level dependent (BOLD) response. In recordings of task related neural activity the combination of MEG and fMRI produces a result with greater signal-to-noise ratio, that confirms the expectation arising from the nature of the experiment. The highly non-linear model of the BOLD response poses a difficult inference problem for neural activity estimation; computational requirements are also high due to the time and space complexity. We show that joint analysis of the data improves the system's behavior by stabilizing the differential equations system and by requiring fewer computational resources.

Source localization by electroencephalography (EEG) requires an accurate model of head geometry and tissue conductivity. The estimation of source time courses from EEG or from EEG in conjunction with magnetoencephalography (MEG) requires a forward model consistent with true activity for the best outcome. Although MRI provides an excellent description of soft tissue anatomy, a high resolution model of the skull (the dominant resistive component of the head) requires CT, which is not justified for routine physiological studies. Although a number of techniques have been employed to estimate tissue conductivity, no present techniques provide the noninvasive 3D tomographic mapping of conductivity that would be desirable. We introduce a formalism for probabilistic forward modeling that allows the propagation of uncertainties in model parameters into possible errors in source localization. We consider uncertainties in the conductivity profile of the skull, but the approach is general and can be extended to other kinds of uncertainties in the forward model. We and others have previously suggested the possibility of extracting conductivity of the skull from measured electroencephalography data by simultaneously optimizing over dipole parameters and the conductivity values required by the forward model. Using Cramer-Rao bounds, we demonstrate that this approach does not improve localization results nor does it produce reliable conductivity estimates. We conclude that the conductivity of the skull has to be either accurately measured by an independent technique, or that the uncertainties in the conductivity values should be reflected in uncertainty in the source location estimates.

We propose a new model to approximate spatiotemporal noise covariance for use in neural electromagnetic source analysis, which better captures temporal variability in background activity. As with other existing formalisms, our model employs a Kronecker product of matrices representing temporal and spatial covariance. In our model, spatial components are allowed to have differing temporal covariances. Variability is represented as a series of Kronecker products of spatial component covariances and corresponding temporal covariances. Unlike previous attempts to model covariance through a sum of Kronecker products, our model is designed to have a computationally manageable inverse. Despite increased descriptive power, inversion of the model is fast, making it useful in source analysis. We have explored two versions of the model. One is estimated based on the assumption that spatial components of background noise have uncorrelated time courses. Another version, which gives closer approximation, is based on the assumption that time courses are statistically independent. The accuracy of the structural approximation is compared to an existing model, based on a single Kronecker product, using both Frobenius norm of the difference between spatiotemporal sample covariance and a model, and scatter plots. Performance of ours and previous models is compared in source analysis of a large number of single dipole problems with simulated time courses and with background from authentic magnetoencephalography data.

The performance of parametric magnetoencephalography (MEG) and electroencephalography (EEG) source localization approaches can be degraded by the use of poor background noise covariance estimates. In general, estimation of the noise covariance for spatiotemporal analysis is difficult mainly due to the limited noise information available. Furthermore, its estimation requires a large amount of storage and a one-time but very large (and sometimes intractable) calculation or its inverse. To overcome these difficulties, noise covariance models consisting of one pair or a sum of multi-pairs of Kronecker products of spatial covariance and temporal covariance have been proposed. However, these approaches cannot be applied when the noise information is very limited, i.e., the amount of noise information is less than the degrees of freedom of the noise covariance models. A common example of this is when only averaged noise data are available for a limited prestimulus region (typically at most a few hundred milliseconds duration). For such cases, a diagonal spatiotemporal noise covariance model consisting of sensor variances with no spatial or temporal correlation has been the common choice for spatiotemporal analysis. In this work, we propose a different noise covariance model which consists of diagonal spatial noise covariance and Toeplitz temporal noise covariance. It can easily be estimated from limited noise information, and no time-consuming optimization and data-processing are required. Thus, it can be used as an alternative choice when one-pair or multi-pair noise covariance models cannot be estimated due to lack of noise information. To verify its capability we used Bayesian inference dipole analysis and a number of simulated and empirical datasets. We compared this covariance model with other existing covariance models such as conventional diagonal covariance, one-pair and multi-pair noise covariance models, when noise information is sufficient to estimate them. We found that our proposed noise covariance model yields better localization performance than a diagonal noise covariance, while it performs slightly worse than one-pair or multi-pair noise covariance models - although these require much more noise information. Finally, we present some localization results on median nerve stimulus empirical MEG data for our proposed noise covariance model.

Most existing spatiotemporal multi-dipole approaches for MEG/EEG source localization assume that the dipoles are active for the full time range being analysed. If the actual time range of activity of sources is significantly shorter than the time range being analysed, the detectability, localization and time-course determination of such sources may be adversely affected, especially for weak sources. In order to improve detectability and reconstruction of such sources, it is natural to add active time range information (starting time point and ending time point of source activation) for each candidate source as unknown parameters in the analysis. However, this adds additional nonlinear free parameters that could burden the analysis and could be unfeasible for some methods. Recently, we described a spatiotemporal Bayesian inference multi-dipole analysis for the MEG/EEG inverse problem. This approach treated the number of dipoles as a free parameter, produced realistic uncertainty estimates using a Markov chain Monte Carlo numerical sampling of the posterior distribution and included a method to reduce the unwanted effects of local minima. In this paper, our spatiotemporal Bayesian inference multi-dipole analysis is extended to incorporate active time range parameters of starting and stopping time points. The properties of this analysis in comparison to the previous one without active time range parameters are demonstrated through extensive studies using both simulated and empirical MEG data.

Using a noise covariance model based on a single Kronecker product of spatial and temporal covariance in the spatiotemporal analysis of MEG data was demonstrated to provide improvement in the results over that of the commonly used diagonal noise covariance model. In this paper we present a model that is a generalization of all of the above models. It describes models based on a single Kronecker product of spatial and temporal covariance as well as more complicated multi-pair models together with any intermediate form expressed as a sum of Kronecker products of spatial component matrices of reduced rank and their corresponding temporal covariance matrices. The model provides a framework for controlling the tradeoff between the described complexity of the background and computational demand for the analysis using this model. Ways to estimate the value of the parameter controlling this tradeoff are also discussed.

Recently, we described a Bayesian inference approach to the MEG/EEG inverse problem that used numerical techniques to estimate the full posterior probability distributions of likely solutions upon which all inferences were based [Schmidt, D.M., George, J.S., Wood, C.C., 1999. Bayesian inference applied to the electromagnetic inverse problem. Human Brain Mapping 7, 195; Schmidt, D.M., George, J.S., Ranken, D.M., Wood, C.C., 2001. Spatial-temporal bayesian inference for MEG/EEG. In: Nenonen, J., Ilmoniemi, R. J., Katila, T. (Eds.), Biomag 2000: 12th International Conference on Biomagnetism. Espoo, Norway, p. 671]. Schmidt et al. (1999) focused on the analysis of data at a single point in time employing an extended region source model. They subsequently extended their work to a spatiotemporal Bayesian inference analysis of the full spatiotemporal MEG/EEG data set. Here, we formulate spatiotemporal Bayesian inference analysis using a multi-dipole model of neural activity. This approach is faster than the extended region model, does not require use of the subject's anatomical information, does not require prior determination of the number of dipoles, and yields quantitative probabilistic inferences. In addition, we have incorporated the ability to handle much more complex and realistic estimates of the background noise, which may be represented as a sum of Kronecker products of temporal and spatial noise covariance components. This reduces the effects of undermodeling noise. In order to reduce the rigidity of the multi-dipole formulation which commonly causes problems due to multiple local minima, we treat the given covariance of the background as uncertain and marginalize over it in the analysis. Markov Chain Monte Carlo (MCMC) was used to sample the many possible likely solutions. The spatiotemporal Bayesian dipole analysis is demonstrated using simulated and empirical whole-head MEG data.