Search

Determining and decoding emotional brain processes under ecologically valid conditions remains a key challenge in affective neuroscience. The current functional Magnetic Resonance Imaging (fMRI) based emotion decoding studies are mainly based on brief and isolated episodes of emotion induction, while sustained emotional experience in naturalistic environments that mirror daily life experiences are scarce. Here we used 12 different 10-minute movie clips as ecologically valid emotion-evoking procedures in n = 52 individuals to explore emotion-specific fMRI functional connectivity (FC) profiles on the whole-brain level at high spatial resolution (432 parcellations including cortical and subcortical structures). Employing machine-learning based decoding and cross validation procedures allowed to investigate FC profiles contributing to classification that can accurately distinguish sustained happiness and sadness and that generalize across subjects, movie clips, and parcellations. Both functional brain network-based and subnetwork-based emotion classification results suggested that emotion manifests as distributed representation of multiple networks, rather than a single functional network or subnetwork. Further, the results showed that the Visual Network (VN) and Default Mode Network (DMN) associated functional networks, especially VN-DMN, exhibited a strong contribution to emotion classification. To further estimate the temporal accumulative effect of naturalistic long-term movie-based video-evoking emotions, we divided the 10-min episode into three stages: early stimulation (1∼200 s), middle stimulation (201∼400 s), and late stimulation (401∼600 s) and examined the emotion classification performance at different stimulation stages. We found that the late stimulation contributes most to the classification (accuracy=85.32%, F1-score=85.62%) compared to early and middle stimulation stages, implying that continuous exposure to emotional stimulation can lead to more intense emotions and further enhance emotion-specific distinguishable representations. The present work demonstrated that sustained happiness and sadness under naturalistic conditions are presented in emotion-specific network profiles and these expressions may play different roles in the generation and modulation of emotions. These findings elucidated the importance of network level adaptations for sustained emotional experiences during naturalistic contexts and open new venues for imaging network level contributions under naturalistic conditions.

Everyday experiences are dynamic, driving fluctuations across simultaneous cognitive processes. A key challenge in the study of naturalistic cognition is to disentangle the complexity of these dynamic processes, without altering the natural experience itself. Retrospective behavioral sampling (RBS) is a novel approach to model the cognitive fluctuations corresponding to the time-course of naturalistic stimulation, across a variety of cognitive dimensions. We tested the effectiveness and reliability of RBS in a webbased experiment, in which 53 participants viewed short movies and listened to a story, followed by retrospective reporting. Participants recalled their experience of 55 discrete events from the stimuli, rating their quality of memory, magnitude of surprise, intensity of negative and positive emotions, perceived importance, reflectivity state, and mental time travel. In addition, a subset of the original cohort re-rated their memory of events in a followup questionnaire. Results show highly replicable fluctuation patterns across distinct cognitive dimensions, thereby revealing a stimulus-driven experience that is substantially shared among individuals. Remarkably, memory ratings more than a week after stimulation resulted in an almost identical timecourse of memorability as measured immediately following stimulation. In addition, idiosyncratic response patterns were preserved across different stimuli, indicating that RBS characterizes individual differences that are stimulus invariant. The current findings highlight the potential of RBS as a powerful tool for measuring dynamic processes of naturalistic cognition. We discuss the promising approach of matching RBS fluctuations with dynamic processes measured via other testing modalities, such as neuroimaging, to study the neural manifestations of naturalistic cognitive processing. [ABSTRACT FROM AUTHOR]

Everyday experiences are dynamic, driving fluctuations across simultaneous cognitive processes. A key challenge in the study of naturalistic cognition is to disentangle the complexity of these dynamic processes, without altering the natural experience itself. Retrospective behavioral sampling (RBS) is a novel approach to model the cognitive fluctuations corresponding to the time-course of naturalistic stimulation, across a variety of cognitive dimensions. We tested the effectiveness and reliability of RBS in a web-based experiment, in which 53 participants viewed short movies and listened to a story, followed by retrospective reporting. Participants recalled their experience of 55 discrete events from the stimuli, rating their quality of memory, magnitude of surprise, intensity of negative and positive emotions, perceived importance, reflectivity state, and mental time travel. In addition, a subset of the original cohort re-rated their memory of events in a follow-up questionnaire. Results show highly replicable fluctuation patterns across distinct cognitive dimensions, thereby revealing a stimulus-driven experience that is substantially shared among individuals. Remarkably, memory ratings more than a week after stimulation resulted in an almost identical time-course of memorability as measured immediately following stimulation. In addition, idiosyncratic response patterns were preserved across different stimuli, indicating that RBS characterizes individual differences that are stimulus invariant. The current findings highlight the potential of RBS as a powerful tool for measuring dynamic processes of naturalistic cognition. We discuss the promising approach of matching RBS fluctuations with dynamic processes measured via other testing modalities, such as neuroimaging, to study the neural manifestations of naturalistic cognitive processing.

In everyday life, the stream of affect results from the interaction between past experiences, expectations and the unfolding of events. How the brain represents the relationship between time and affect has been hardly explored, as it requires modeling the complexity of everyday life in the laboratory setting. Movies condense into hours a multitude of emotional responses, synchronized across subjects and characterized by temporal dynamics alike real-world experiences. Here, we use time-varying intersubject brain synchronization and real-time behavioral reports to test whether connectivity dynamics track changes in affect during movie watching. The results show that polarity and intensity of experiences relate to the connectivity of the default mode and control networks and converge in the right temporoparietal cortex. We validate these results in two experiments including four independent samples, two movies and alternative analysis workflows. Finally, we reveal chronotopic connectivity maps within the temporoparietal and prefrontal cortex, where adjacent areas preferentially encode affect at specific timescales. [ABSTRACT FROM AUTHOR]

Schizophrenia is marked by aberrant processing of complex speech and gesture, which may contribute functionally to its impaired social communication. To date, extant neuroscientific studies of schizophrenia have largely investigated dysfunctional speech and gesture in isolation, and no prior research has examined how the two communicative channels may interact in more natural contexts. Here, we tested if patients with schizophrenia show aberrant neural processing of semantically complex story segments, and if speech-associated gestures (co-speech gestures) might modulate this effect. In a functional MRI study, we presented to 34 participants (16 patients and 18 matched-controls) an ecologically-valid retelling of a continuous story, performed via speech and spontaneous gestures. We split the entire story into ten-word segments, and measured the semantic complexity for each segment with idea density, a linguistic measure that is commonly used clinically to evaluate aberrant language dysfunction at the semantic level. Per segment, the presence of numbers of gestures varied (n = 0, 1, +2). Our results suggest that, in comparison to controls, patients showed reduced activation for more complex segments in the bilateral middle frontal and inferior parietal regions. Importantly, this neural aberrance was normalized in segments presented with gestures. Thus, for the first time with a naturalistic multimodal stimulation paradigm, we show that gestures reduced group differences when processing a natural story, probably by facilitating the processing of semantically complex segments of the story in schizophrenia. [ABSTRACT FROM AUTHOR]

Introduction Affective experiences vary as function of context, motivations and the unfolding of events. This temporal fundamental aspect of emotional processes is often disrupted in psychiatric conditions. Objectives To investigate how the brain represents the association between affect and time, we combined fMRI and behavioral ratings during movie watching. Methods Participants watched ‘Forrest Gump’ in the fMRI scanner (n=14, 6F). Data were preprocessed (see 10.1101/2020.06.06.137851v1) and average brain activity from 1000 regions was extracted. Independent subjects (n=12, 5F) provided continuous ratings of the intensity of their affective state while watching the same movie. Using PCA, we derived the first 3 affective dimensions (polarity, complexity, intensity; 10.1038/s41467-019-13599-z) and computed their time-varying correlation in windows from 5-1000tps. We identified the window size with the maximum between-subjects accordance and computed the inter-subject functional connectivity (10.1038/ncomms12141). For each region, we obtained connectivity strength and its association in time with changes in affective dimensions (pBonf

Traditional neuroimaging methods have identified alterations in brain activity patterns following mild traumatic brain injury (mTBI), particularly during rest, complex tasks, and normal vision. However, studies using graph theory to examine brain network changes in mTBI have produced varied results, influenced by the specific networks and task demands analyzed. In our study, we employed functional MRI to observe 17 mTBI patients and 54 healthy individuals as they viewed a simple, non-narrative underwater film, simulating everyday visual tasks. This approach revealed significant mTBI-related changes in network connectivity, efficiency, and organization. Specifically, the mTBI group exhibited higher overall connectivity and local network specialization, suggesting enhanced information integration without overwhelming the brain's processing capabilities. Conversely, these patients showed reduced network segregation, indicating a less compartmentalized brain function compared to healthy controls. These patterns were consistent across various visual cortex subnetworks, except in primary visual areas. Our findings highlight the potential of using naturalistic stimuli in graph-based neuroimaging to understand brain network alterations in mTBI and possibly other conditions affecting brain integration.

A fundamental step to predicting brain activity in healthy and diseased populations is characterizing the common spatio-temporal response to a shared experience. Multivoxel pattern analysis allows us to investigate information encoding through these patterns; however, we have yet to explore local, stimulus-driven, patterns of cortical activity during naturalistic stimulation. We sought to examine these patterns with minimum interpolation--excluding functional alignment--to characterize the most basic degree of shared response between subjects. We used an unbiased analytic approach, combined with rich, naturalistic, and nonsemantic stimulation to estimate shared spatial patterns in functional magnetic resonance imaging responses across a large group. We found that meso-scale spatial patterns were shared nonuniformly across the visual cortex and represent information distinct from the shared temporal response. Shared spatial patterns were stimulus-driven, modulated by pattern size, and more sensitive to the contrast of 3D versus 2D stimulus differences than the temporal signals. Although the grand functional structure of the brain is understood to be common, these results suggest that even at a meso-scale, we share common spatial structures with anatomical alignment alone. The strength of this similarity varies across the cortex, suggesting some spatial structures are innately organized, whereas others are shaped by factors such as learning and plasticity. [ABSTRACT FROM AUTHOR]

It is well established that some patients, who are deemed to have disorders of consciousness, remain entirely behaviorally non-responsive and are diagnosed as being in a vegetative state, yet can nevertheless demonstrate covert awareness of their external environment by modulating their brain activity, a phenomenon known as cognitive-motor dissociation. However, the extent to which these patients retain internal awareness remains unknown. To investigate the potential for internal and external awareness in patients with chronic disorders of consciousness (DoC), we asked whether the pattern of juxtaposition between the functional time-courses of the default mode (DMN) and fronto-parietal networks, shown in healthy individuals to mediate the naturally occurring dominance switching between internal and external aspects of consciousness, was present in these patients. We used a highly engaging movie by Alfred Hitchcock to drive the recruitment of the fronto-parietal networks, including the dorsal attention (DAN) and executive control (ECN) networks, and their maximal juxtaposition to the DMN in response to the complex stimulus, relative to rest and a scrambled, meaningless movie baseline condition. We tested a control group of healthy participants (N = 13/12) and two groups of patients with disorders of consciousness, one comprised of patients who demonstrated independent, neuroimaging-based evidence of covert external awareness (N = 8), and the other of those who did not (N = 8). Similarly to the healthy controls, only the group of patients with overt and, critically, covert external awareness showed significantly heightened differentiation between the DMN and the DAN in response to movie viewing relative to their resting state time-courses, which was driven by the movie's narrative. This result suggested the presence of functional integrity in the DMN and fronto-parietal networks and their relationship to one another in patients with covert external awareness. Similar to the effect in healthy controls, these networks became more strongly juxtaposed to one another in response to movie viewing relative to the baseline conditions, suggesting the potential for internal and external awareness during complex stimulus processing. Furthermore, our results suggest that naturalistic paradigms can dissociate between groups of DoC patients with and without covert awareness based on the functional integrity of brain networks.

It is well established that some patients, who are deemed to have disorders of consciousness, remain entirely behaviorally non-responsive and are diagnosed as being in a vegetative state, yet can nevertheless demonstrate covert awareness of their external environment by modulating their brain activity, a phenomenon known as cognitive-motor dissociation. However, the extent to which these patients retain internal awareness remains unknown. To investigate the potential for internal and external awareness in patients with chronic disorders of consciousness (DoC), we asked whether the pattern of juxtaposition between the functional time-courses of the default mode (DMN) and fronto-parietal networks, shown in healthy individuals to mediate the naturally occurring dominance switching between internal and external aspects of consciousness, was present in these patients. We used a highly engaging movie by Alfred Hitchcock to drive the recruitment of the fronto-parietal networks, including the dorsal attention (DAN) and executive control (ECN) networks, and their maximal juxtaposition to the DMN in response to the complex stimulus, relative to rest and a scrambled, meaningless movie baseline condition. We tested a control group of healthy participants (N = 13/12) and two groups of patients with disorders of consciousness, one comprised of patients who demonstrated independent, neuroimaging-based evidence of covert external awareness (N = 8), and the other of those who did not (N = 8). Similarly to the healthy controls, only the group of patients with overt and, critically, covert external awareness showed significantly heightened differentiation between the DMN and the DAN in response to movie viewing relative to their resting state time-courses, which was driven by the movie's narrative. This result suggested the presence of functional integrity in the DMN and fronto-parietal networks and their relationship to one another in patients with covert external awareness. Similar to the effect in healthy controls, these networks became more strongly juxtaposed to one another in response to movie viewing relative to the baseline conditions, suggesting the potential for internal and external awareness during complex stimulus processing. Furthermore, our results suggest that naturalistic paradigms can dissociate between groups of DoC patients with and without covert awareness based on the functional integrity of brain networks. [ABSTRACT FROM AUTHOR]

Movie viewing allows human perception and cognition to be studied in complex, real-life-like situations in a brain-imaging laboratory. Previous studies with functional magnetic resonance imaging (fMRI) and with magneto- and electroencephalography (MEG and EEG) have demonstrated consistent temporal dynamics of brain activity across movie viewers. However, little is known about the similarities and differences of fMRI and MEG or EEG dynamics during such naturalistic situations. We thus compared MEG and fMRI responses to the same 15-min black-and-white movie in the same eight subjects who watched the movie twice during both MEG and fMRI recordings. We analyzed intra- and intersubject voxel-wise correlations within each imaging modality as well as the correlation of the MEG envelopes and fMRI signals. The fMRI signals showed voxel-wise within- and between-subjects correlations up to r  = 0.66 and r  = 0.37, respectively, whereas these correlations were clearly weaker for the envelopes of band-pass filtered (7 frequency bands below 100 Hz) MEG signals (within-subjects correlation r  < 0.14 and between-subjects r  < 0.05). Direct MEG–fMRI voxel-wise correlations were unreliable. Notably, applying a spatial-filtering approach to the MEG data uncovered consistent canonical variates that showed considerably stronger (up to r  = 0.25) between-subjects correlations than the univariate voxel-wise analysis. Furthermore, the envelopes of the time courses of these variates up to about 10 Hz showed association with fMRI signals in a general linear model. Similarities between envelopes of MEG canonical variates and fMRI voxel time-courses were seen mostly in occipital, but also in temporal and frontal brain regions, whereas intra- and intersubject correlations for MEG and fMRI separately were strongest only in the occipital areas. In contrast to the conventional univariate analysis, the spatial-filtering approach was able to uncover associations between the MEG envelopes and fMRI time courses, shedding light on the similarities of hemodynamic and electromagnetic brain activities during movie viewing. [ABSTRACT FROM AUTHOR]

The human brain continuously processes massive amounts of rich sensory information. To better understand such highly complex brain processes, modern neuroimaging studies are increasingly utilizing experimental setups that better mimic daily-life situations. A new exploratory data-analysis approach, functional segmentation inter-subject correlation analysis (FuSeISC), was proposed to facilitate the analysis of functional magnetic resonance (fMRI) data sets collected in these experiments. The method provides a new type of functional segmentation of brain areas, not only characterizing areas that display similar processing across subjects but also areas in which processing across subjects is highly variable. FuSeISC was tested using fMRI data sets collected during traditional block-design stimuli (37 subjects) as well as naturalistic auditory narratives (19 subjects). The method identified spatially local and/or bilaterally symmetric clusters in several cortical areas, many of which are known to be processing the types of stimuli used in the experiments. The method is not only useful for spatial exploration of large fMRI data sets obtained using naturalistic stimuli, but also has other potential applications, such as generation of a functional brain atlases including both lower- and higher-order processing areas. Finally, as a part of FuSeISC, a criterion-based sparsification of the shared nearest-neighbor graph was proposed for detecting clusters in noisy data. In the tests with synthetic data, this technique was superior to well-known clustering methods, such as Ward's method, affinity propagation, and K-means [ABSTRACT FROM AUTHOR]

We present a simple but effective correlation-based method (maxCorr) for extracting subject-specific components from group- fMRI data. The method finds signal components that correlate maximally with the data set of one subject and minimally with the data sets of the other subjects. We show that such subject-specific components are often related to movement and physiological noise (e.g. cardiac cycle, respiration). We further demonstrate that removing the most subject-specific components for each subject reduces the overall data variance and improves the statistical identification of true fMRI activations. We compare the performance of maxCorr with CompCor, a commonly used artifact-finding method in fMRI analysis. We show that maxCorr is less likely than CompCor to remove actual stimulus-related activity, especially when no information about the stimulus is available. MaxCorr operates without stimulus information and is therefore well suitable for analyses of fMRI experiments employing naturalistic stimuli, such as movies, where stimulus regressors are difficult to construct, and for brain decoding techniques benefiting from reduced subject-specific variance in each subject's data. [ABSTRACT FROM AUTHOR]

Independent component analysis (ICA) of electroencephalographic (EEG) and magnetoencephalographic (MEG) data is usually performed over the temporal dimension: each channel is one row of the data matrix, and a linear transformation maximizing the independence of component time courses is sought. In functional magnetic resonance imaging (fMRI), by contrast, most studies use spatial ICA: each time point constitutes a row of the data matrix, and independence of the spatial patterns is maximized. Here, we show the utility of spatial ICA in characterizing oscillatory neuromagnetic signals. We project the sensor data into cortical space using a standard minimum-norm estimate and apply a sparsifying transform to focus on oscillatory signals. The resulting method, spatial Fourier-ICA, provides a concise summary of the spatiotemporal and spectral content of spontaneous neuromagnetic oscillations in cortical source space over time scales of minutes. Spatial Fourier-ICA applied to resting-state and naturalistic stimulation MEG data from nine healthy subjects revealed consistent components covering the early visual, somatosensory and motor cortices with spectral peaks at ∼10 and ∼20 Hz. The proposed method seems valuable for inferring functional connectivity, stimulus-related modulation of rhythmic activity, and their commonalities across subjects from nonaveraged MEG data. Hum Brain Mapp, 2011. © 2011 Wiley-Liss, Inc. [ABSTRACT FROM AUTHOR]

Sensing the movement of fast objects within our visual environments is essential for controlling actions. It requires online estimation of motion direction and speed. We probed human speed representation using ocular tracking of stimuli of different statistics. First, we compared ocular responses to single drifting gratings (DGs) with a given set of spatiotemporal frequencies to broadband motion clouds (MCs) of matched mean frequencies. Motion energy distributions of gratings and clouds are point-like, and ellipses oriented along the constant speed axis, respectively. Sampling frequency space, MCs elicited stronger, less variable, and speed-tuned responses. DGs yielded weaker and more frequency-tuned responses. Second, we measured responses to patterns made of two or three components covering a range of orientations within Fourier space. Early tracking initiation of the patterns was best predicted by a linear combination of components before nonlinear interactions emerged to shape later dynamics. Inputs are supralinearly integrated along an iso-velocity line and sublinearly integrated away from it. A dynamical probabilistic model characterizes these interactions as an excitatory pooling along the iso-velocity line and inhibition along the orthogonal "scale" axis. Such crossed patterns of interaction would appropriately integrate or segment moving objects. This study supports the novel idea that speed estimation is better framed as a dynamic channel interaction organized along speed and scale axes., (Copyright © 2022 Meso et al.)

Schizophrenia is marked by aberrant processing of complex speech and gesture, which may contribute functionally to its impaired social communication. To date, extant neuroscientific studies of schizophrenia have largely investigated dysfunctional speech and gesture in isolation, and no prior research has examined how the two communicative channels may interact in more natural contexts. Here, we tested if patients with schizophrenia show aberrant neural processing of semantically complex story segments, and if speech-associated gestures (co-speech gestures) might modulate this effect. In a functional MRI study, we presented to 34 participants (16 patients and 18 matched-controls) an ecologically-valid retelling of a continuous story, performed via speech and spontaneous gestures. We split the entire story into ten-word segments, and measured the semantic complexity for each segment with idea density, a linguistic measure that is commonly used clinically to evaluate aberrant language dysfunction at the semantic level. Per segment, the presence of numbers of gestures varied (n = 0, 1, +2). Our results suggest that, in comparison to controls, patients showed reduced activation for more complex segments in the bilateral middle frontal and inferior parietal regions. Importantly, this neural aberrance was normalized in segments presented with gestures. Thus, for the first time with a naturalistic multimodal stimulation paradigm, we show that gestures reduced group differences when processing a natural story, probably by facilitating the processing of semantically complex segments of the story in schizophrenia., (© The Author(s) 2022. Published by Oxford University Press on behalf of the University of Maryland's school of medicine, Maryland Psychiatric Research Center.)

Introduction: Affective experiences vary as function of context, motivations and the unfolding of events. This temporal fundamental aspect of emotional processes is often disrupted in psychiatric conditions. Objectives: To investigate how the brain represents the association between affect and time, we combined fMRI and behavioral ratings during movie watching. Methods: Participants watched 'Forrest Gump' in the fMRI scanner (n=14, 6F). Data were preprocessed (see 10.1101/2020.06.06.137851v1) and average brain activity from 1000 regions was extracted. Independent subjects (n=12, 5F) provided continuous ratings of the intensity of their affective state while watching the same movie. Using PCA, we derived the first 3 affective dimensions (polarity, complexity, intensity; 10.1038/s41467-019-13599-z) and computed their time-varying correlation in windows from 5-1000tps. We identified the window size with the maximum between-subjects accordance and computed the inter-subject functional connectivity (10.1038/ncomms12141). For each region, we obtained connectivity strength and its association in time with changes in affective dimensions (pBonf<0.05). Results: Fluctuations in connectivity strength of the right rMFG, precuneus, pSTS/TPJ, dmPFC, aINS and left pMTG were associated to polarity. Also, connectivity of the right IPS/SPL, SFG, dpreCS, IFGpOrb, OFC, precuneus, vpreCS and pSTS/TPJ followed the timecourse of perceived intensity of affect. Conclusions: Connectivity strength of default mode represents the pleasantness of the experience, whereas attention and control networks encode its intensity. Emotional descriptions converge in right temporoparietal and fronto-polar cortex, where the stream of affect is encoded in a chronotopic manner. These results expand our understanding of the neural correlates of emotional processing, a function severely affected by mental disorders. [ABSTRACT FROM AUTHOR]