Search

Working memory contents are represented in neural activity patterns across multiple regions of the cortical hierarchy. A division of labor has been proposed where more anterior regions harbor increasingly abstract and categorical representations while the most detailed representations are held in primary sensory cortices. Here, using fMRI and multivariate encoding modeling, we demonstrate that for color stimuli categorical codes are already present at the level of extrastriate visual cortex (V4 and VO1), even when subjects are neither implicitly nor explicitly encouraged to categorize the stimuli. Importantly, this categorical coding was observed during working memory, but not during perception. Thus, visual working memory is likely to rely at least in part on categorical representations. SIGNIFICANCE STATEMENT: Working memory is the representational basis for human cognition. Recent work has demonstrated that numerous regions across the human brain can represent the contents of working memory. We use fMRI brain scanning and machine learning methods to demonstrate that different regions can represent the same content differently during working memory. Reading out the neural codes used to store working memory contents, we show that already in sensory cortex, areas V4 and VO1 represent color in a categorical format rather than a purely sensory fashion. Thereby, we provide a better understanding of how different regions of the brain might serve working memory and cognition., Competing Interests: Declaration of Competing Interest The authors have no interests to declare., (Copyright © 2023. Published by Elsevier Inc.)

Representations of sensory working memory can be found across the entire neocortex. But how are verbal working memory (VWM) contents retained in the human brain? Here we used fMRI and multi-voxel pattern analyses to study Chinese native speakers (15 males, 13 females) memorizing Chinese characters. Chinese characters are uniquely suitable to study VWM because verbal encoding is encouraged by their complex visual appearance and monosyllabic pronunciation. We found that activity patterns in Broca's area and left premotor cortex carried information about the memorized characters. These language-related areas carried (1) significantly more information about cued characters than those not cued for memorization, (2) significantly more information on the left than the right hemisphere and (3) significantly more information about Chinese symbols than complex visual patterns which are hard to verbalize. In contrast, early visual cortex carries a comparable amount of information about cued and uncued stimuli and is thus unlikely to be involved in memory retention. This study provides evidence for verbal working memory maintenance in a distributed network of language-related brain regions, consistent with distributed accounts of WM. The results also suggest that Broca's area and left premotor cortex form the articulatory network which serves articulatory rehearsal in the retention of verbal working memory contents., Competing Interests: Declaration of Competing Interest None., (Copyright © 2020. Published by Elsevier Inc.)

Visual working memory (VWM) allows for keeping visual information available for upcoming goal-directed behavior, while new visual input is processed concurrently. Interactions between the mnemonic and perceptual systems cause VWM to affect the processing of visual input in a content-specific manner: visual input that is initially suppressed from consciousness is detected faster when it matches rather than mismatches the content of VWM. It is currently under debate whether such mnemonic influences on perception occur prior to or after conscious access. To address this issue, we investigated whether VWM content modulates the neural response to visual input that remains suppressed from consciousness. We measured fMRI responses to interocularly suppressed stimuli in 20 human participants performing a delayed match-to-sample task: Participants were retro-cued to memorize one of two geometrical shapes for subsequent recognition. During retention, an interocularly suppressed peripheral stimulus (the probe) was briefly presented, which was either of the cued (memorized) or uncued (not memorized) shape category. We found no evidence that VWM content modulated the neural response to the probe. Substantial evidence for the absence of this modulation was found despite leveraging a highly liberal analysis approach: (1) selecting regions of interest that were particularly prone to detecting said modulation, and (2) using directional Bayesian tests favoring the presence of the hypothesized modulation. We did observe faster detection of memory-matching compared to memory-mismatching probes in a behavioral control experiment, thus validating the stimulus set. We conclude that VWM impacts the processing of visual input only once suppression is mostly alleviated., Competing Interests: Declaration of competing interest The authors declare no competing (financial) interests., (Copyright © 2020 The Authors. Published by Elsevier Inc. All rights reserved.)

Techniques of multivariate pattern analysis (MVPA) can be used to decode the discrete experimental condition or a continuous modulator variable from measured brain activity during a particular trial. In functional magnetic resonance imaging (fMRI), trial-wise response amplitudes are sometimes estimated from the measured signal using a general linear model (GLM) with one onset regressor for each trial. When using rapid event-related designs with trials closely spaced in time, those estimates are highly variable and serially correlated due to the temporally extended shape of the hemodynamic response function (HRF). Here, we describe inverse transformed encoding modelling (ITEM), a principled approach of accounting for those serial correlations and decoding from the resulting estimates, at low computational cost and with no loss in statistical power. We use simulated data to show that ITEM outperforms the current standard approach in terms of decoding accuracy and analyze empirical data to demonstrate that ITEM is capable of visual reconstruction from fMRI signals., Competing Interests: Declaration of competing interest The authors have no conflict of interest, financial or otherwise, to declare., (Copyright © 2019 The Authors. Published by Elsevier Inc. All rights reserved.)

Although dietary decision-making is regulated by multiple interacting neural controllers, their impact on dietary treatment success in obesity has only been investigated individually. Here, we used fMRI to test how well interactions between the Pavlovian system (automatically triggering urges of consumption after food cue exposure) and the goal-directed system (considering long-term consequences of food decisions) predict future dietary success achieved in 39 months. Activity of the Pavlovian system was measured with a cue-reactivity task by comparing perception of food versus control pictures, activity of the goal-directed system with a food-specific delay discounting paradigm. Both tasks were applied in 30 individuals with obesity up to five times: Before a 12-week diet, immediately thereafter, and at three annual follow-up visits. Brain activity was analyzed in two steps. In the first, we searched for areas involved in Pavlovian processes and goal-directed control across the 39-month study period with voxel-wise linear mixed-effects (LME) analyses. In the second, we computed network parameters reflecting the covariation of longitudinal voxel activity (i.e. principal components) in the regions identified in the first step and used them to predict body mass changes across the 39 months with LME models. Network analyses testing the link of dietary success with activity of the individual systems as reference found a moderate negative link to Pavlovian activity primarily in left hippocampus and a moderate positive association to goal-directed activity primarily in right inferior parietal gyrus. A cross-paradigm network analysis that integrated activity measured in both tasks revealed a strong positive link for interactions between visual Pavlovian areas and goal-directed decision-making regions mainly located in right insular cortex. We conclude that adaptation of food cue processing resources to goal-directed control activity is an important prerequisite of sustained dietary weight loss, presumably since the latter activity can modulate Pavlovian urges triggered by frequent cue exposure in everyday life., (Copyright © 2018 Elsevier Inc. All rights reserved.)

To fully characterize the activity patterns on the cerebral cortex as measured with fMRI, the spatial scale of the patterns must be ascertained. Here we address this problem by constructing steerable bandpass filters on the discrete, irregular cortical mesh, using an improved Gaussian smoothing in combination with differential operators of directional derivatives. We demonstrate the utility of the algorithm in two ways. First, using modelling we show that our algorithm yields superior results in numerical precision and spatial uniformity of filter kernels compared to the most widely adopted approach for cortical smoothing. As the effective scales of information differ from the nominal filter sizes applied to extract them, we evaluated the effective scales empirically for different filters to make subsequent comparisons well calibrated. Second, we applied the algorithm to an fMRI dataset to assess the scale and pattern form of cortical encoding of information about visual objects in the ventral visual pathway. We found that filtering by our method improved the detection of discriminant information about experimental conditions over previous methods, that the level of categorization (subordinate versus superordinate) of objects was differentially related to the spatial scale of fMRI patterns, and that the spatial scale at which information was encoded increased along the ventral visual pathway. In sum, our results indicate that the proposed algorithm is particularly suited to assess and detect scale-specific information encoding in cortex, and promises further insight into the topography of cortical encoding in the human brain., (Copyright © 2018. Published by Elsevier Inc.)

Standard neuroimaging data analysis based on traditional principles of experimental design, modelling, and statistical inference is increasingly complemented by novel analysis methods, driven e.g. by machine learning methods. While these novel approaches provide new insights into neuroimaging data, they often have unexpected properties, generating a growing literature on possible pitfalls. We propose to meet this challenge by adopting a habit of systematic testing of experimental design, analysis procedures, and statistical inference. Specifically, we suggest to apply the analysis method used for experimental data also to aspects of the experimental design, simulated confounds, simulated null data, and control data. We stress the importance of keeping the analysis method the same in main and test analyses, because only this way possible confounds and unexpected properties can be reliably detected and avoided. We describe and discuss this Same Analysis Approach in detail, and demonstrate it in two worked examples using multivariate decoding. With these examples, we reveal two sources of error: A mismatch between counterbalancing (crossover designs) and cross-validation which leads to systematic below-chance accuracies, and linear decoding of a nonlinear effect, a difference in variance., (Copyright © 2017 Elsevier Inc. All rights reserved.)

For everyday communication, bilingual speakers need to face the complex task of rapidly choosing the most appropriate language given the context, maintaining this choice over the current communicative act, and shielding lexical selection from competing alternatives from non-target languages. Yet, speech production of bilinguals is typically flawless and fluent. Most of the studies available to date constrain speakers' language choice by cueing the target language and conflate language choice with language use. This left largely unexplored the neural mechanisms underlying free language choice, i.e., the voluntary situation of choosing the language to speak. In this study, we used fMRI and Multivariate Pattern Analysis to identify brain regions encoding the target language when bilinguals are free to choose in which language to name pictures. We found that the medial prefrontal cortex encoded the chosen language prior to speaking. By contrast, during language use, language control recruited a wider brain network including the left inferior frontal lobe, the basal ganglia, and the angular and inferior parietal gyrus bilaterally. None of these regions were involved in language choice. We argue that the control processes involved in language choice are different from those involved in language use. Furthermore, our findings confirm that the medial prefrontal cortex is a domain-general region critical for free choice and that bilingual language choice relies on domain general processes., (Copyright © 2018 Elsevier Inc. All rights reserved.)

In functional magnetic resonance imaging (fMRI), model quality of general linear models (GLMs) for first-level analysis is rarely assessed. In recent work (Soch et al., 2016: "How to avoid mismodelling in GLM-based fMRI data analysis: cross-validated Bayesian model selection", NeuroImage, vol. 141, pp. 469-489; http://dx.doi.org/10.1016/j.neuroimage.2016.07.047), we have introduced cross-validated Bayesian model selection (cvBMS) to infer the best model for a group of subjects and use it to guide second-level analysis. While this is the optimal approach given that the same GLM has to be used for all subjects, there is a much more efficient procedure when model selection only addresses nuisance variables and regressors of interest are included in all candidate models. In this work, we propose cross-validated Bayesian model averaging (cvBMA) to improve parameter estimates for these regressors of interest by combining information from all models using their posterior probabilities. This is particularly useful as different models can lead to different conclusions regarding experimental effects and the most complex model is not necessarily the best choice. We find that cvBMS can prevent not detecting established effects and that cvBMA can be more sensitive to experimental effects than just using even the best model in each subject or the model which is best in a group of subjects., (Copyright © 2017 Elsevier Inc. All rights reserved.)

Focused attention meditations have been shown to improve psychological health and wellbeing and are nowadays an integral part of many psychotherapies. While research on the neural correlates of focused attention meditation is increasing, findings vary on whether meditations are associated with high or low activity in the default mode network (DMN). To clarify the relationship between focused attention meditation and the activity in DMN regions, it may be helpful to distinguish internal and external attention as well as different phases within one meditation: During focused attention meditation, the practitioner switches between mindful attention, mind-wandering and refocusing. Here, we employed a thought-probe paradigm to study the neural correlates of these different phases. Twenty healthy, meditation naïve participants were introduced to external (mindfulness of sound) and internal (mindfulness of breathing) attention meditation and then practiced the meditation at home for four consecutive days. They then performed the same focused attention meditations during fMRI scanning, in four runs alternating between internal and external attention. At pseudorandom intervals, participants were asked whether they had just been focused on the task (mindful attention) or had been distracted (mind-wandering). During mindful attention, brain regions typically associated with the DMN, such as the medial prefrontal cortex, posterior cingulate cortex and left temporoparietal junction showed significantly less neural activation compared to mind-wandering phases. Reduced activity of the DMN was found during both external and internal attention, with stronger deactivation in the posterior cingulate cortex during internal attention compared to external attention. Moreover, refocusing after mind-wandering was associated with activity in the left inferior frontal gyrus. Our results support the theory that mindful attention is associated with reduced DMN activity compared to mind-wandering, independent of the practitioner's attention focus (i.e., internal vs. external)., (Copyright © 2017 Elsevier Inc. All rights reserved.)

In multivariate pattern analysis of neuroimaging data, 'second-level' inference is often performed by entering classification accuracies into a t-test vs chance level across subjects. We argue that while the random-effects analysis implemented by the t-test does provide population inference if applied to activation differences, it fails to do so in the case of classification accuracy or other 'information-like' measures, because the true value of such measures can never be below chance level. This constraint changes the meaning of the population-level null hypothesis being tested, which becomes equivalent to the global null hypothesis that there is no effect in any subject in the population. Consequently, rejecting it only allows to infer that there are some subjects in which there is an information effect, but not that it generalizes, rendering it effectively equivalent to fixed-effects analysis. This statement is supported by theoretical arguments as well as simulations. We review possible alternative approaches to population inference for information-based imaging, converging on the idea that it should not target the mean, but the prevalence of the effect in the population. One method to do so, 'permutation-based information prevalence inference using the minimum statistic', is described in detail and applied to empirical data., (Copyright © 2016 Elsevier Inc. All rights reserved.)

Voxel-wise general linear models (GLMs) are a standard approach for analyzing functional magnetic resonance imaging (fMRI) data. An advantage of GLMs is that they are flexible and can be adapted to the requirements of many different data sets. However, the specification of first-level GLMs leaves the researcher with many degrees of freedom which is problematic given recent efforts to ensure robust and reproducible fMRI data analysis. Formal model comparisons that allow a systematic assessment of GLMs are only rarely performed. On the one hand, too simple models may underfit data and leave real effects undiscovered. On the other hand, too complex models might overfit data and also reduce statistical power. Here we present a systematic approach termed cross-validated Bayesian model selection (cvBMS) that allows to decide which GLM best describes a given fMRI data set. Importantly, our approach allows for non-nested model comparison, i.e. comparing more than two models that do not just differ by adding one or more regressors. It also allows for spatially heterogeneous modelling, i.e. using different models for different parts of the brain. We validate our method using simulated data and demonstrate potential applications to empirical data. The increased use of model comparison and model selection should increase the reliability of GLM results and reproducibility of fMRI studies., (Copyright © 2016 Elsevier Inc. All rights reserved.)

A defining trait of human cognition is the capacity to form compounds out of simple thoughts. This ability relies on the logical connectives AND, OR and IF. Simple propositions, e.g., 'There is a fork' and 'There is a knife', can be combined in alternative ways using logical connectives: e.g., 'There is a fork AND there is a knife', 'There is a fork OR there is a knife', 'IF there is a fork, there is a knife'. How does the brain represent compounds based on different logical connectives, and how are compounds evaluated in relation to new facts? In the present study, participants had to maintain and evaluate conjunctive (AND), disjunctive (OR) or conditional (IF) compounds while undergoing functional MRI. Our results suggest that, during maintenance, the left posterior inferior frontal gyrus (pIFG, BA44, or Broca's area) represents the surface form of compounds. During evaluation, the left pIFG switches to processing the full logical meaning of compounds, and two additional areas are recruited: the left anterior inferior frontal gyrus (aIFG, BA47) and the left intraparietal sulcus (IPS, BA40). The aIFG shows a pattern of activation similar to pIFG, and compatible with processing the full logical meaning of compounds, whereas activations in IPS differ with alternative interpretations of conditionals: logical vs conjunctive. These results uncover the functions of a basic cortical network underlying human compositional thought, and provide a shared neural foundation for the cognitive science of language and reasoning., (Copyright © 2016 Elsevier Inc. All rights reserved.)

Intentional action is essential to human behavior, yet its neural basis remains poorly understood. In order to identify neural networks specifically involved in intentional action, freely chosen and externally cued intentions have previously been contrasted. This has led to the identification of a fronto-parietal network, which is involved in freely choosing one's intentions. However, it remains unclear whether this network encodes specific intentions, or whether it merely reflects general preparatory or control processes correlated with intentional action. Here, we used MVPA on fMRI data to identify brain regions encoding non-motor intentions that were either freely chosen or externally cued. We found that a fronto-parietal network, including the lateral prefrontal cortex, premotor, and parietal cortex, contained information about both freely chosen and externally cued intentions. Importantly, MVPA cross-classification indicated that this network represents the content of our intentions similarly, regardless of whether these intentions are freely chosen or externally cued. This finding suggests that the intention network has a general role in processing and representing intentions independent of their origin., (Copyright © 2016 Elsevier Inc. All rights reserved.)

The modulation of neural activity in visual cortex is thought to be a key mechanism of visual attention. The investigation of attentional modulation in high-level visual areas, however, is hampered by the lack of clear tuning or contrast response functions. In the present functional magnetic resonance imaging study we therefore systematically assessed how small voxel-wise biases in object preference across hundreds of voxels in the lateral occipital complex were affected when attention was directed to objects. We found that the strength of attentional modulation depended on a voxel's object preference in the absence of attention, a pattern indicative of an amplificatory mechanism. Our results show that such attentional modulation effectively increased the mutual information between voxel responses and object identity. Further, these local modulatory effects led to improved information-based object readout at the level of multi-voxel activation patterns and to an increased reproducibility of these patterns across repeated presentations. We conclude that attentional modulation enhances object coding in local and distributed object representations of the lateral occipital complex., (Copyright © 2015 Elsevier Inc. All rights reserved.)

A variety of studies suggest that efficient treatments to induce short-term dietary success in obesity exist. However, sustained maintenance of reduced weight is rare as a large proportion of patients start to regain weight when treatment is discontinued. Thus, from a clinical perspective, it would be desirable to identify factors that counteract post-diet weight regain across longer time-scales. To address this question, we extended our previous work on neural impulse control mechanisms of short-term dietary success in obesity and now investigated the mechanisms counteracting long-term weight regain after a diet. Specifically, we measured neural impulse control during a delay discounting task with fMRI at two time points, i.e. the beginning ('T0') and the end ('T12') of a one-year follow-up interval after a 12-week diet. Then, we tested whether activity in the dorsolateral prefrontal cortex (DLPFC) at T0 and whether activity changes across the follow-up period (T0-T12) are linked to success in weight maintenance. The analyses conducted show that control-related DLPFC activity at T0 was coupled to the degree of success in weight maintenance. Consistently, also behavioral measures of control were linked to the degree of success in maintenance. A direct comparison of neural and behavioral control parameters for prognostic weight change modeling revealed that neural signals were more informative. Taken together, neural impulse control in the DLPFC measured with fMRI directly after a diet predicts real-world diet success in obese patients across extended time periods., (Copyright © 2015 Elsevier Inc. All rights reserved.)

A fundamental issue in visual cognition is whether high-level visual areas code objects in a part-based or a view-based (holistic) format. Previous behavioral and neuroimaging studies that examined the viewpoint invariance of object recognition have yielded ambiguous results, providing evidence for either type of representational format. A critical factor distinguishing the two formats could be the availability of attentional resources, as a number of priming studies have found greater viewpoint invariance for attended compared to unattended objects. It has therefore been suggested that the activation of part-based representations requires attention, whereas the activation of holistic representations occurs automatically irrespective of attention. Using functional magnetic resonance imaging in combination with a novel multivariate pattern analysis approach, the present study probed the format of object representations in human lateral occipital complex and its dependence on attention. We presented human participants with intact and half-split versions of objects that were either attended or unattended. Cross-classifying between intact and split objects, we found that the object-related information coded in activation patterns of intact objects is fully preserved in the patterns of split objects and vice versa. Importantly, the generalization between intact and split objects did not depend on attention. We conclude that lateral occipital complex codes objects in a non-holistic format, both in the presence and absence of attention., (Copyright © 2014 Elsevier Inc. All rights reserved.)

Active and flexible manipulations of memory contents "in the mind's eye" are believed to occur in a dedicated neural workspace, frequently referred to as visual working memory. Such a neural workspace should have two important properties: The ability to store sensory information across delay periods and the ability to flexibly transform sensory information. Here we used a combination of functional MRI and multivariate decoding to indentify such neural representations. Subjects were required to memorize a complex artificial pattern for an extended delay, then rotate the mental image as instructed by a cue and memorize this transformed pattern. We found that patterns of brain activity already in early visual areas and posterior parietal cortex encode not only the initially remembered image, but also the transformed contents after mental rotation. Our results thus suggest that the flexible and general neural workspace supporting visual working memory can be realized within posterior brain regions., (Copyright © 2014 Elsevier Inc. All rights reserved.)

Social context plays an important role in human communication. Depending on the nature of the source, the same communication signal might be processed in fundamentally different ways. However, the selective modulation (or "gating") of the flow of neural information during communication is not fully understood. Here, we use multivoxel pattern analysis (MVPA) and multivoxel connectivity analysis (MVCA), a novel technique that allows to analyse context-dependent changes of the strength interregional coupling between ensembles of voxels, to examine how the human brain differentially gates content-specific sensory information during ongoing perception of communication signals. In a simulated electronic communication experiment, participants received two alternative text messages during fMRI ("happy" or "sad") which they believed had been sent either by their real-life friend outside the scanner or by a computer. A region in the dorsal medial prefrontal cortex (dmPFC) selectively increased its functional coupling with sensory-content encoding regions in the visual cortex when a text message was perceived as being sent by the participant's friend, and decreased its functional coupling with these regions when a text message was perceived as being sent by the computer. Furthermore, the strength of neural encoding of content-specific information of text messages in the dmPFC was modulated by the social tie between the participant and her friend: the more of her spare time a participant reported to spend with her friend the stronger was the neural encoding. This suggests that the human brain selectively gates sensory information into the relevant network for processing the mental states of others, depending on the source of the communication signal., (Copyright © 2014 The Authors. Published by Elsevier Inc. All rights reserved.)

Stimulus visibility is associated with neural signals in multiple brain regions, ranging from visual cortex to prefrontal regions. Here we used functional magnetic resonance imaging (fMRI) to investigate to which extent the perceived visibility of a "low-level" grating stimulus is reflected in the brain activity in high-level brain regions. Oriented grating stimuli were presented under varying visibility conditions created by backward masking. Visibility was manipulated using four different stimulus onset asynchronies (SOAs), which created a continuum from invisible to highly visible target stimuli. Brain activity in early visual areas, high-level visual brain regions (fusiform gyrus), as well as parietal and prefrontal brain regions was significantly correlated with subjects' psychometric visibility functions. In addition, increased stimulus visibility was reflected in the functional coupling between low and high-level visual areas. Specifically, neuroimaging signals in the middle occipital gyrus were significantly more correlated with signals in the inferior temporal gyrus when subjects successfully perceived the target stimulus than when they did not. These results provide evidence that not only low-level visual but also high-level brain regions reflect visibility of low-level grating stimuli and that changes in functional connectivity reflect perceived stimulus visibility., (Copyright © 2014 Elsevier Inc. All rights reserved.)

There has been a long history of research on visual working memory. Whereas early studies have focused on the role of lateral prefrontal cortex in the storage of sensory information, this has been challenged by research in humans that has directly assessed the encoding of perceptual contents, pointing towards a role of visual and parietal regions during storage. In a previous study we used pattern classification to investigate the storage of complex visual color patterns across delay periods. This revealed coding of such contents in early visual and parietal brain regions. Here we aim to investigate whether the involvement of visual and parietal cortex is also observable for other types of complex, visuo-spatial pattern stimuli. Specifically, we used a combination of fMRI and multivariate classification to investigate the retention of complex flow-field stimuli defined by the spatial patterning of motion trajectories of random dots. Subjects were trained to memorize the precise spatial layout of these stimuli and to retain this information during an extended delay. We used a multivariate decoding approach to identify brain regions where spatial patterns of activity encoded the memorized stimuli. Content-specific memory signals were observable in motion sensitive visual area MT+ and in posterior parietal cortex that might encode spatial information in a modality independent manner. Interestingly, we also found information about the memorized visual stimulus in somatosensory cortex, suggesting a potential crossmodal contribution to memory. Our findings thus indicate that working memory storage of visual percepts might be distributed across unimodal, multimodal and even crossmodal brain regions., (Copyright © 2014 Elsevier Inc. All rights reserved.)

Multi-voxel pattern analysis (MVPA) is a fruitful and increasingly popular complement to traditional univariate methods of analyzing neuroimaging data. We propose to replace the standard 'decoding' approach to searchlight-based MVPA, measuring the performance of a classifier by its accuracy, with a method based on the multivariate form of the general linear model. Following the well-established methodology of multivariate analysis of variance (MANOVA), we define a measure that directly characterizes the structure of multi-voxel data, the pattern distinctness D. Our measure is related to standard multivariate statistics, but we apply cross-validation to obtain an unbiased estimate of its population value, independent of the amount of data or its partitioning into 'training' and 'test' sets. The estimate D^ can therefore serve not only as a test statistic, but also as an interpretable measure of multivariate effect size. The pattern distinctness generalizes the Mahalanobis distance to an arbitrary number of classes, but also the case where there are no classes of trials because the design is described by parametric regressors. It is defined for arbitrary estimable contrasts, including main effects (pattern differences) and interactions (pattern changes). In this way, our approach makes the full analytical power of complex factorial designs known from univariate fMRI analyses available to MVPA studies. Moreover, we show how the results of a factorial analysis can be used to obtain a measure of pattern stability, the equivalent of 'cross-decoding'., (Copyright © 2013 Elsevier Inc. All rights reserved.)

The increase in spatiotemporal resolution of neuroimaging devices is accompanied by a trend towards more powerful multivariate analysis methods. Often it is desired to interpret the outcome of these methods with respect to the cognitive processes under study. Here we discuss which methods allow for such interpretations, and provide guidelines for choosing an appropriate analysis for a given experimental goal: For a surgeon who needs to decide where to remove brain tissue it is most important to determine the origin of cognitive functions and associated neural processes. In contrast, when communicating with paralyzed or comatose patients via brain-computer interfaces, it is most important to accurately extract the neural processes specific to a certain mental state. These equally important but complementary objectives require different analysis methods. Determining the origin of neural processes in time or space from the parameters of a data-driven model requires what we call a forward model of the data; such a model explains how the measured data was generated from the neural sources. Examples are general linear models (GLMs). Methods for the extraction of neural information from data can be considered as backward models, as they attempt to reverse the data generating process. Examples are multivariate classifiers. Here we demonstrate that the parameters of forward models are neurophysiologically interpretable in the sense that significant nonzero weights are only observed at channels the activity of which is related to the brain process under study. In contrast, the interpretation of backward model parameters can lead to wrong conclusions regarding the spatial or temporal origin of the neural signals of interest, since significant nonzero weights may also be observed at channels the activity of which is statistically independent of the brain process under study. As a remedy for the linear case, we propose a procedure for transforming backward models into forward models. This procedure enables the neurophysiological interpretation of the parameters of linear backward models. We hope that this work raises awareness for an often encountered problem and provides a theoretical basis for conducting better interpretable multivariate neuroimaging analyses., (Copyright © 2013 The Authors. Published by Elsevier Inc. All rights reserved.)

Deficits in impulse control are discussed as key mechanisms for major worldwide health problems such as drug addiction and obesity. For example, obese subjects have difficulty controlling their impulses to overeat when faced with food items. Here, we investigated the role of neural impulse control mechanisms for dietary success in middle-aged obese subjects. Specifically, we used a food-specific delayed gratification paradigm and functional magnetic resonance imaging to measure eating-related impulse-control in middle-aged obese subjects just before they underwent a twelve-week low calorie diet. As expected, we found that subjects with higher behavioral impulse control subsequently lost more weight. Furthermore, brain activity before the diet in VMPFC and DLPFC correlates with subsequent weight loss. Additionally, a connectivity analysis revealed that stronger functional connectivity between these regions is associated with better dietary success and impulse control. Thus, the degree to which subjects can control their eating impulses might depend on the interplay between control regions (DLPFC) and regions signaling the reward of food (VMPFC). This could potentially constitute a general mechanism that also extends to other disorders such as drug addiction or alcohol abuse., (© 2013.)

Visual stimuli can "pop out" if they are different to their background. There has been considerable debate as to the role of primary visual cortex (V1) versus higher visual areas (esp. V4) in pop-out processing. Here we parametrically modulated the relative orientation of stimuli and their backgrounds to investigate the neural correlates of pop-out in visual cortex while subjects were performing a demanding fixation task in a scanner. Whole brain and region of interest analyses confirmed a representation of orientation contrast in extrastriate visual cortex (V4), but not in striate visual cortex (V1). Thus, although previous studies have shown that human V1 can be involved in orientation pop-out, our findings demonstrate that there are cases where V1 is "blind" and pop-out detection is restricted to higher visual areas. Pop-out processing is presumably a distributed process across multiple visual regions., (Copyright © 2013 Elsevier Inc. All rights reserved.)

Individual political preferences as expressed, for instance, in votes or donations are fundamental to democratic societies. However, the relevance of deliberative processing for political preferences has been highly debated, putting automatic processes in the focus of attention. Based on this notion, the present study tested whether brain responses reflect participants' preferences for politicians and their associated political parties in the absence of explicit deliberation and attention. Participants were instructed to perform a demanding visual fixation task while their brain responses were measured using fMRI. Occasionally, task-irrelevant images of German politicians from two major competing parties were presented in the background while the distraction task was continued. Subsequent to scanning, participants' political preferences for these politicians and their affiliated parties were obtained. Brain responses in distinct brain areas predicted automatic political preferences at the different levels of abstraction: activation in the ventral striatum was positively correlated with preference ranks for unattended politicians, whereas participants' preferences for the affiliated political parties were reflected in activity in the insula and the cingulate cortex. Using an additional donation task, we showed that the automatic preference-related processing in the brain extended to real-world behavior that involved actual financial loss to participants. Together, these findings indicate that brain responses triggered by unattended and task-irrelevant political images reflect individual political preferences at different levels of abstraction., (Copyright © 2013 Elsevier Inc. All rights reserved.)

When facing perceptual choices under challenging conditions we might believe to be purely guessing. But which brain regions determine the outcome of our guesses? One possibility is that higher-level, domain-general brain regions might help break the symmetry between equal-appearing choices. Here we directly investigated whether perceptual guesses share brain networks with other types of free decisions. We trained an fMRI-based pattern classifier to distinguish between two perceptual guesses and tested whether it was able to predict the outcome of similar non-perceptual choices, as in conventional free choice tasks. Activation patterns in the medial posterior parietal cortex cross-predicted free decisions from perceptual guesses and vice versa. This inter-changeability strongly speaks for a similar neural code for both types of decisions. The posterior parietal cortex might be part of a domain-general system that helps resolve decision conflicts when no choice option is more or less likely or valuable, thus preventing behavioural stalemate., (Copyright © 2012 Elsevier Inc. All rights reserved.)

What are the brain mechanisms that mediate conscious object recognition? To investigate this question, it is essential to distinguish between brain processes that cause conscious recognition of a stimulus from other correlates of its sensory processing. Previous fMRI studies have identified large-scale brain activity ranging from striate to high-level sensory and prefrontal regions associated with conscious visual perception or recognition. However, the possible role of changes in connectivity during conscious perception between these regions has only rarely been studied. Here, we used fMRI and connectivity analyses, together with 120 custom-generated, two-tone, Mooney images to directly assess whether conscious recognition of an object is accompanied by a dynamical change in the functional coupling between extrastriate cortex and prefrontal areas. We compared recognizing an object versus not recognizing it in 19 naïve subjects using two different response modalities. We find that connectivity between the extrastriate cortex and the dorsolateral prefrontal cortex (DLPFC) increases when objects are consciously recognized. This interaction was independent of the response modality used to report conscious recognition. Furthermore, computing the difference in Granger causality between recognized and not recognized conditions reveals stronger feedforward connectivity than feedback connectivity when subjects recognized the objects. We suggest that frontal and visual brain regions are part of a functional network that supports conscious object recognition by changes in functional connectivity., (Copyright © 2012 Elsevier Inc. All rights reserved.)

Perceptual decision-making entails the transformation of graded sensory signals into categorical judgments. Often, there is a direct mapping between these judgments and specific motor responses. However, when stimulus-response mappings are fixed, neural activity underlying decision-making cannot be separated from neural activity reflecting motor planning. Several human neuroimaging studies have reported changes in brain activity associated with perceptual decisions. Nevertheless, to date it has remained unknown where and how specific choices are encoded in the human brain when motor planning is decoupled from the decision process. We addressed this question by having subjects judge the direction of motion of dynamic random dot patterns at various levels of motion strength while measuring their brain activity with fMRI. We used multivariate decoding analyses to search the whole brain for patterns of brain activity encoding subjects' choices. To decouple the decision process from motor planning, subjects were informed about the required motor response only after stimulus presentation. Patterns of fMRI signals in early visual and inferior parietal cortex predicted subjects' perceptual choices irrespective of motor planning. This was true across several levels of motion strength and even in the absence of any coherent stimulus motion. We also found that the cortical distribution of choice-selective brain signals depended on stimulus strength: While visual cortex carried most choice-selective information for strong motion, information in parietal cortex decreased with increasing motion coherence. These results demonstrate that human visual and inferior parietal cortex carry information about the visual decision in a more abstract format than can be explained by simple motor intentions. Both brain regions may be differentially involved in perceptual decision-making in the face of strong and weak sensory evidence., (Copyright © 2012 Elsevier Inc. All rights reserved.)

Recently, multivariate analysis algorithms have become a popular tool to diagnose neurological diseases based on neuroimaging data. Most studies, however, are biased for one specific scale, namely the scale given by the spatial resolution (i.e. dimension) of the data. In the present study, we propose to use the dual-tree complex wavelet transform to extract information on different spatial scales from structural MRI data and show its relevance for disease classification. Based on the magnitude representation of the complex wavelet coefficients calculated from the MR images, we identified a new class of features taking scale, directionality and potentially local information into account simultaneously. By using a linear support vector machine, these features were shown to discriminate significantly between spatially normalized MR images of 41 patients suffering from multiple sclerosis and 26 healthy controls. Interestingly, the decoding accuracies varied strongly among the different scales and it turned out that scales containing low frequency information were partly superior to scales containing high frequency information. Usually, this type of information is neglected since most decoding studies use only the original scale of the data. In conclusion, our proposed method has not only a high potential to assist in the diagnostic process of multiple sclerosis, but can be applied to other diseases or general decoding problems in structural or functional MRI., (Copyright © 2012 Elsevier Inc. All rights reserved.)

On a daily basis we form numerous intentions to perform specific actions. However, we often have to delay the execution of intended actions while engaging in other demanding activities. Previous research has shown that patterns of activity in human prefrontal cortex (PFC) can reveal our current intentions. However, two fundamental questions have remained unresolved: (a) how does the PFC encode information about future tasks while we are busy engaging in other activities, and (b) how does the PFC enable us to commence a stored task at the intended time? Here we investigate how the brain stores and retrieves future intentions during occupied delays, i.e. while a person is busy performing a different task. For this purpose, we conducted a neuroimaging study with a time-based prospective memory paradigm. Using multivariate pattern classification and fMRI we show that during an occupied delay, activity patterns in the anterior PFC encode the content of 'what' subjects intend to do next, and 'when' they intend to do it. Importantly, distinct anterior PFC regions store the 'what' and 'when' components of future intentions during occupied maintenance and self-initiated retrieval. These results show a role for anterior PFC activity patterns in storing future action plans and ensuring their timely retrieval., (Copyright © 2012 Elsevier Inc. All rights reserved.)

Patients suffering from obsessive-compulsive disorder (OCD) are characterized by dysregulated neuronal processing of disorder-specific and also unspecific affective stimuli. In the present study, we investigated whether generic fear-inducing, disgust-inducing, and neutral stimuli can be decoded from brain patterns of single fMRI time samples of individual OCD patients and healthy controls. Furthermore, we tested whether differences in the underlying encoding provide information to classify subjects into groups (OCD patients or healthy controls). Two pattern classification analyses were conducted. In analysis 1, we used a classifier to decode the category of a currently viewed picture from extended fMRI patterns of single time samples (TR=3s) in individual subjects for several pairs of categories. In analysis 2, we used a searchlight approach to predict subjects' diagnostic status based on local brain patterns. In analysis 1, we obtained significant accuracies for the separation of fear-eliciting from neutral pictures in OCD patients and healthy controls. Separation of disgust-inducing from neutral pictures was significant in healthy controls. In analysis 2, we identified diagnostic information for the presence of OCD in the orbitofrontal cortex, and in the caudate nucleus. Accuracy obtained in these regions was 100% (p<10(-6)). To summarize our findings, by using multivariate pattern classification techniques we were able to identify neurobiological markers providing reliable diagnostic information about OCD. The classifier-based fMRI paradigms proposed here might be integrated in future diagnostic procedures and treatment concepts., (Copyright © 2012 Elsevier Inc. All rights reserved.)

Human perception depends heavily on the quality of sensory information. When objects are hard to see we often believe ourselves to be purely guessing. Here we investigated whether such guesses use brain networks involved in perceptual decision making or independent networks. We used a combination of fMRI and pattern classification to test how visibility affects the signals, which determine choices. We found that decisions regarding clearly visible objects are predicted by signals in sensory brain regions, whereas different regions in parietal cortex became predictive when subjects were shown invisible objects and believed themselves to be purely guessing. This parietal network was highly overlapping with regions, which have previously been shown to encode free decisions. Thus, the brain might use a dedicated network for determining choices when insufficient sensory information is available., (Copyright © 2011 Elsevier Inc. All rights reserved.)

A fundamental challenge for organisms is how to focus on perceptual information relevant to current goals while remaining able to respond to goal-irrelevant stimuli that signal potential threat. Here, we studied how visual threat signals influence the effects of goal-directed spatial attention on the retinotopic distribution of processing resources in early visual cortex. We used a combined blocked and event-related functional magnetic resonance imaging paradigm with target displays comprising diagonal pairs of intact and scrambled faces presented simultaneously in the four visual field quadrants. Faces were male or female and had fearful or neutral emotional expressions. Participants attended covertly to a pair of two diagonally opposite stimuli and performed a gender-discrimination task on the attended intact face. In contrast to the fusiform face area, where attention and fearful emotional expression had additive effects, neural responses to attended and unattended fearful faces were indistinguishable in early retinotopic visual areas: When attended, fearful face expression did not further enhance responses, whereas when unattended, fearful expression increased responses to the level of attended face stimuli. Remarkably, the presence of fearful stimuli augmented the enhancing effect of attention on retinotopic responses to neutral faces in remote visual field locations. We conclude that this redistribution of neural activity in retinotopic visual cortex may serve the purpose of allocating processing resources to task-irrelevant threat-signaling stimuli while at the same time increasing resources for task-relevant stimuli as required for the maintenance of goal-directed behavior., (Copyright © 2011 Elsevier Inc. All rights reserved.)

Neural activity in mammalian brains exhibits large spontaneous fluctuations whose structure reveals the intrinsic functional connectivity of the brain on many spatial and temporal scales. Between remote brain regions, spontaneous activity is organized into large-scale functional networks. To date, it has remained unclear whether the intrinsic functional connectivity between brain regions scales down to the fine detail of anatomical connections, for example the fine-grained topographic connectivity structure in visual cortex. Here, we show that fMRI signal fluctuations reveal a detailed retinotopically organized functional connectivity structure between the visual field maps of remote areas of the human visual cortex. The structured coherent fluctuations were even preserved in complete darkness when all visual input was removed. While the topographic connectivity structure was clearly visible in within hemisphere connections, the between hemisphere connectivity structure differs for representations along the vertical and horizontal meridian respectively. These results suggest a tight link between spontaneous neural activity and the fine-grained topographic connectivity pattern of the human brain. Thus, intrinsic functional connectivity reflects the detailed connectivity structure of the cortex at a fine spatial scale. It might thus be a valuable tool to complement anatomical studies of the human connectome, which is one of the keys to understand the functioning of the human brain., (Copyright © 2011 Elsevier Inc. All rights reserved.)

In recent years, the scope of neuroimaging research has been substantially extended by multivariate decoding methodology. Decoding techniques allow us to address a number of important questions that are frequently neglected in more conventional analyses. They allow us to focus on storage of "mental content" in brain regions, rather than on overall levels of activation. They directly address the question how much information can be "read out" of brain activity patterns, thus inverting the classical direction of inference that attempts to explain brain activity from mental state variables. At the same time, they provide a much higher sensitivity to detection of effects than conventional approaches. This special issue is a showcase of research in this emerging field. Besides five invited review papers by key experts in the field, it presents a representative selection of work showing the diversity and power of multivariate decoding analyses ranging from methodological foundations to cognitive and clinical studies., (Copyright © 2011. Published by Elsevier Inc.)

In everyday life, successful decision making requires precise representations of expected values. However, for most behavioral options more than one attribute can be relevant in order to predict the expected reward. Thus, to make good or even optimal choices the reward predictions of multiple attributes need to be integrated into a combined expected value. Importantly, the individual attributes of such multi-attribute objects can agree or disagree in their reward prediction. Here we address where the brain encodes the combined reward prediction (averaged across attributes) and where it encodes the variability of the value predictions of the individual attributes. We acquired fMRI data while subjects performed a task in which they had to integrate reward predictions from multiple attributes into a combined value. Using time-resolved pattern recognition techniques (support vector regression) we find that (1) the combined value is encoded in distributed fMRI patterns in the ventromedial prefrontal cortex (vmPFC) and that (2) the variability of value predictions of the individual attributes is encoded in the dorsolateral prefrontal cortex (dlPFC). The combined value could be used to guide choices, whereas the variability of the value predictions of individual attributes indicates an ambiguity that results in an increased difficulty of the value-integration. These results demonstrate that the different features defining multi-attribute objects are encoded in non-overlapping brain regions and therefore suggest different roles for vmPFC and dlPFC in multi-attribute decision making., (Copyright © 2010 Elsevier Inc. All rights reserved.)

Local voxel patterns of fMRI signals contain specific information about cognitive processes ranging from basic sensory processing to high level decision making. These patterns can be detected using multivariate pattern classification, and localization of these patterns can be achieved with searchlight methods in which the information content of spherical sub-volumes of the fMRI signal is assessed. The only assumption made by this approach is that the patterns are spatially local. We present a cortical surface-based searchlight approach to pattern localization. Voxels are grouped according to distance along the cortical surface-the intrinsic metric of cortical anatomy-rather than Euclidean distance as in volumetric searchlights. Using a paradigm in which the category of visually presented objects is decoded, we compare the surface-based method to a standard volumetric searchlight technique. Group analyses of accuracy maps produced by both methods show similar distributions of informative regions. The surface-based method achieves a finer spatial specificity with comparable peak values of significance, while the volumetric method appears to be more sensitive to small informative regions and might also capture information not located directly within the gray matter. Furthermore, our findings show that a surface centered in the middle of the gray matter contains more information than to the white-gray boundary or the pial surface., (Copyright © 2010 Elsevier Inc. All rights reserved.)

We are able to recognize objects independent of their location in the visual field. At the same time, we also keep track of the location of objects to orient ourselves and to interact with the environment. The lateral occipital complex (LOC) has been suggested as the prime cortical region for representation of object identity. However, the extent to which LOC also represents object location has remained debated. In this study we used high-resolution fMRI in combination with multivoxel pattern classification to investigate the cortical encoding of three object exemplars from four different categories presented in two different locations. This approach allowed us to study location-tolerant object information and object-tolerant location information in LOC, both at the level of categories and exemplars. We found evidence for both location-tolerant object information and object-tolerant location information in LOC at the level of categories and exemplars. Our results further highlight the mixing of identity and location information in the ventral visual pathway., (Copyright © 2010 Elsevier Inc. All rights reserved.)

When people interact, affective information is transmitted between their brains. Modern imaging techniques permit to investigate the dynamics of this brain-to-brain transfer of information. Here, we used information-based functional magnetic resonance imaging (fMRI) to investigate the flow of affective information between the brains of senders and perceivers engaged in ongoing facial communication of affect. We found that the level of neural activity within a distributed network of the perceiver's brain can be successfully predicted from the neural activity in the same network in the sender's brain, depending on the affect that is currently being communicated. Furthermore, there was a temporal succession in the flow of affective information from the sender's brain to the perceiver's brain, with information in the perceiver's brain being significantly delayed relative to information in the sender's brain. This delay decreased over time, possibly reflecting some 'tuning in' of the perceiver with the sender. Our data support current theories of intersubjectivity by providing direct evidence that during ongoing facial communication a 'shared space' of affect is successively built up between senders and perceivers of affective facial signals., (Copyright © 2010 Elsevier Inc. All rights reserved.)

The flow of information from sensory stimuli to motor responses in the human brain can be flexibly re-routed depending on task demands. However, it has remained unclear which sequence of processes is involved in preparing the brain for an upcoming task. Here, we used a combination of fMRI and multivariate pattern classification to decompose the information flow in a task-switching experiment. Specifically, we present a time-resolved decoding approach that allowed us to track the temporal buildup of task-related information. This approach also allowed us to distinguish encoding of the task from encoding of target stimuli and motor responses, thus separating between different components of information processing. We were able to decode from parietal and lateral prefrontal cortex which specific task-set a subject was currently holding. Importantly, this revealed that the intraparietal sulcus encoded task-set information before prefrontal cortex, and it was the only region to encode the specific task-set before the relevant target stimulus was presented. This suggests that task-related information in parietal cortex does not rely on input from prefrontal cortex as previously suggested. In contrast, our findings suggest that parietal cortex might play a role in establishing task-sets in prefrontal cortex.

Although it is widely held that colour and form are processed separately in early visual cortex, there is growing evidence that primary visual cortex (V1) may show some joint selectivity for orientation and colour. Colour is supplied to V1 via two very different pathways: the parvocellular pathway (which also supports detailed form processing) carries L-M ("red-green") chromatic information, while a koniocellular pathway carries S-cone ("lilac-yellow") information. Therefore on entering V1, S-cone information is segregated from the pathways carrying form information, while L-M information is not. Whether signals from neuronal populations in human V1 reflect combined orientation and S-cone information has not been systematically addressed. We used fMRI in combination with a multivariate data analysis technique to investigate whether BOLD signals recorded from V1 contain information that could directly discriminate between orientations based on different types of chromatic information. We found selectivity in V1 for L-M and luminance-defined orientation signals, and most interestingly, also for S-cone defined orientation. We also found similarly successful orientation discrimination for both colour dimensions in V2 and V3. These results imply that a proportion of cells throughout human visual cortex show joint sensitivity to both colour and orientation. We discuss also the potential role of feedback to V1 from higher visual areas.

Apparent motion (AM) is the illusory perception of real motion created when two spatially distinct stationary visual objects are presented in alternating sequence. In common with many other illusory percepts, activation during AM can be identified in unstimulated regions of V1 representing the illusory motion path. However, little is known about the mechanisms underlying such activation and its relationship with motion-sensitive area hMT+/V5. Using fMRI and a novel AM stimulus, we replicated previous findings showing a correlate of the perceived AM path in V1. To more closely characterize the mechanisms underlying these activations, we performed analyses of effective connectivity and found that the AM-induced activations on the illusory AM path were associated with enhanced feedback (but not feedforward) connectivity from hMT+/V5. These findings provide for the first time evidence for the involvement of cortico-cortical coupling in generating an illusory percept of AM. They therefore emphasize the role of recurrent processing between visual cortical areas in human perceptual awareness.

When a single brief visual flash is accompanied by two auditory bleeps, it is frequently perceived incorrectly as two flashes. Here, we used high field functional MRI in humans to examine the neural basis of this multisensory perceptual illusion. We show that activity in retinotopic visual cortex is increased by the presence of concurrent auditory stimulation, irrespective of any illusory perception. However, when concurrent auditory stimulation gave rise to illusory visual perception, activity in V1 was enhanced, despite auditory and visual stimulation being unchanged. These findings confirm that responses in human V1 can be altered by sound and show that they reflect subjective perception rather than the physically present visual stimulus. Moreover, as the right superior temporal sulcus and superior colliculus were also activated by illusory visual perception, together with V1, they provide a potential neural substrate for the generation of this multisensory illusion.