Search

Abstract The dorsolateral prefrontal cortex (dlPFC) is reliably engaged in working memory (WM) and comprises different cytoarchitectonic layers, yet their functional role in human WM is unclear. Here, participants completed a delayed-match-to-sample task while undergoing functional magnetic resonance imaging (fMRI) at ultra-high resolution. We examine layer-specific activity to manipulations in WM load and motor response. Superficial layers exhibit a preferential response to WM load during the delay and retrieval periods of a WM task, indicating a lamina-specific activation of the frontoparietal network. Multivariate patterns encoding WM load in the superficial layer dynamically change across the three periods of the task. Last, superficial and deep layers are non-differentially involved in the motor response, challenging earlier findings of a preferential deep layer activation. Taken together, our results provide new insights into the functional laminar circuitry of the dlPFC during WM and support a dynamic account of dlPFC coding.

Summary: This study examines the impact of sample size on predicting cognitive and mental health phenotypes from brain imaging via machine learning. Our analysis shows a 3- to 9-fold improvement in prediction performance when sample size increases from 1,000 to 1 M participants. However, despite this increase, the data suggest that prediction accuracy remains worryingly low and far from fully exploiting the predictive potential of brain imaging data. Additionally, we find that integrating multiple imaging modalities boosts prediction accuracy, often equivalent to doubling the sample size. Interestingly, the most informative imaging modality often varied with increasing sample size, emphasizing the need to consider multiple modalities. Despite significant performance reserves for phenotype prediction, achieving substantial improvements may necessitate prohibitively large sample sizes, thus casting doubt on the practical or clinical utility of machine learning in some areas of neuroimaging.

IntroductionResearch on the neural mechanisms of perceptual decision-making has typically focused on simple categorical choices, say between two alternative motion directions. Studies on such discrete alternatives have often suggested that choices are encoded either in a motor-based or in an abstract, categorical format in regions beyond sensory cortex.MethodsIn this study, we used motion stimuli that could vary anywhere between 0° and 360° to assess how the brain encodes choices for features that span the full sensory continuum. We employed a combination of neuroimaging and encoding models based on Gaussian process regression to assess how either stimuli or choices were encoded in brain responses.ResultsWe found that single-voxel tuning patterns could be used to reconstruct the trial-by-trial physical direction of motion as well as the participants’ continuous choices. Importantly, these continuous choice signals were primarily observed in early visual areas. The tuning properties in this region generalized between choice encoding and stimulus encoding, even for reports that reflected pure guessing.DiscussionWe found only little information related to the decision outcome in regions beyond visual cortex, such as parietal cortex, possibly because our task did not involve differential motor preparation. This could suggest that decisions for continuous stimuli take can place already in sensory brain regions, potentially using similar mechanisms to the sensory recruitment in visual working memory.

Background: Childhood maltreatment profoundly alters trajectories of brain development, promoting markedly increased long-term health risks and impaired intellectual development. However, the immediate impact of maltreatment on brain development in children and the extent to which altered global brain volume contributes to intellectual development in children with maltreatment experience is currently unknown. We here utilized MRI data obtained from children within 6 months after the exposure to maltreatment to assess the association of maltreatment severity with global brain volume changes. We further assessed the association between maltreatment severity and intellectual development and tested for the mediating effect of brain volume on this association. Method: We used structural MRI (3T) in a sample of 49 children aged 3–5 years with maltreatment exposure, i.e. emotional and physical abuse and/or neglect within 6 months, to characterize intracranial and tissue-specific volumes. Maltreatment severity was coded using the Maternal Interview for the Classification of Maltreatment. IQ was tested at study entry and after one year using the Snijders Oomen Nonverbal Test. Results: Higher maltreatment severity was significantly correlated with smaller intracranial volume (r = -.393, p = .008), which was mainly driven by lower total brain volume (r = -.393, p = .008), which in turn was primarily due to smaller gray matter volume (r = -.454, p = .002). Furthermore, smaller gray matter volume was associated with lower IQ at study entry (r = -.548, p

Experiments on choice-predictive brain signals have played an important role in the debate on free will. In a seminal study, Benjamin Libet and colleagues found that a negative-going EEG signal, the readiness potential (RP), can be observed over motor-related brain regions even hundreds of ms before the time of the conscious decision to move. If the early onset of the readiness potential is taken as an indicator of the “brain’s decision to move” this could mean that this decision is made early, by unconscious brain activity, rather than later, at the time when the subject believes to have decided. However, an alternative kind of interpretation, involving ongoing stochastic fluctuations, has recently been brought to light. This stochastic decision model (SDM) takes its inspiration from leaky accumulator models of perceptual decision making. It suggests that the RP originates from an accumulation of ongoing stochastic fluctuations. In this view, the decision happens only at a much later stage when an accumulated noisy signal (plus imperative) reaches a threshold. Here, we clarify a number of confusions regarding both the evidence for the stochastic decision model as well as the interpretation that it offers. We will explore several points that we feel are in need of clarification: (a) the empirical evidence for the role of stochastic fluctuations is so far only indirect; (b) the interpretation of animal studies is unclear; (c) a model that is deterministic during the accumulation stage can explain the data in a similar way; (d) the primary focus in the literature has been on the role of random fluctuations whereas the deterministic aspects of the model have been largely ignored; (e) contrary to the original interpretation, the deterministic component of the model is quantitatively the dominant input into the accumulator; and finally (f) there is confusion regarding the role of “imperative” in the SDM and its link to “evidence” in perceptual decision making. Our aim is not to rehabilitate the role of the RP in the free will debate. Rather we aim to address some confusions regarding the evidence for accumulators playing a role in these preparatory brain processes.

Abstract Deciding the best action in social settings requires decision-makers to consider their and others’ preferences, since the outcome depends on the actions of both. Numerous empirical investigations have demonstrated variability of behavior across individuals in strategic situations. While prosocial, moral, and emotional factors have been intensively investigated to explain this diversity, neuro-cognitive determinants of strategic decision-making and their relation with intelligence remain mostly unknown. This study presents a new model of the process of strategic decision-making in repeated interactions, first providing a precise measure of the environment’s complexity, and then analyzing how this complexity affects subjects’ performance and neural response. The results confirm the theoretical predictions of the model. The frequency of deviations from optimal behavior is explained by a combination of higher complexity of the strategic environment and cognitive skills of the individuals. Brain response correlates with strategic complexity, but only in the subgroups with higher cognitive skills. Furthermore, neural effects were only observed in a fronto-parietal network typically involved in single-agent tasks (the Multiple Demand Network), thus suggesting that neural processes dealing with cognitively demanding individual tasks also have a central role in interactive decision-making. Our findings contribute to understanding how cognitive factors shape strategic decision-making and may provide the neural pathway of the reported association between strategic sophistication and fluid intelligence.

Working memory contents are represented in neural activity patterns across multiple regions of the cortical hierarchy. A division of labor has been proposed where more anterior regions harbor increasingly abstract and categorical representations while the most detailed representations are held in primary sensory cortices. Here, using fMRI and multivariate encoding modeling, we demonstrate that for color stimuli categorical codes are already present at the level of extrastriate visual cortex (V4 and VO1), even when subjects are neither implicitly nor explicitly encouraged to categorize the stimuli. Importantly, this categorical coding was observed during working memory, but not during perception. Thus, visual working memory is likely to rely at least in part on categorical representations. Significance statement: Working memory is the representational basis for human cognition. Recent work has demonstrated that numerous regions across the human brain can represent the contents of working memory. We use fMRI brain scanning and machine learning methods to demonstrate that different regions can represent the same content differently during working memory. Reading out the neural codes used to store working memory contents, we show that already in sensory cortex, areas V4 and VO1 represent color in a categorical format rather than a purely sensory fashion. Thereby, we provide a better understanding of how different regions of the brain might serve working memory and cognition.

Many studies have identified networks in parietal and prefrontal cortex that are involved in intentional action. Yet, our understanding of the way these networks are involved in intentions is still very limited. In this study, we investigate two characteristics of these processes: context- and reason-dependence of the neural states associated with intentions. We ask whether these states depend on the context a person is in and the reasons they have for choosing an action. We used a combination of functional magnetic resonance imaging (fMRI) and multivariate decoding to directly assess the context- and reason-dependency of the neural states underlying intentions. We show that action intentions can be decoded from fMRI data based on a classifier trained in the same context and with the same reason, in line with previous decoding studies. Furthermore, we found that intentions can be decoded across different reasons for choosing an action. However, decoding across different contexts was not successful. We found anecdotal to moderate evidence against context-invariant information in all regions of interest and for all conditions but one. These results suggest that the neural states associated with intentions are modulated by the context of the action.

Abstract Surgical face masks reduce the spread of airborne pathogens but also disturb the flow of information between individuals. The risk of getting seriously ill after infection with SARS-COV-2 during the present COVID-19 pandemic amplifies with age, suggesting that face masks should be worn especially during face-to-face contact with and between older people. However, the ability to accurately perceive and understand communication signals decreases with age, and it is currently unknown whether face masks impair facial communication more severely in older people. We compared the impact of surgical face masks on dynamic facial emotion recognition in younger (18–30 years) and older (65–85 years) adults (N = 96) in an online study. Participants watched short video clips of young women who facially expressed anger, fear, contempt or sadness. Faces of half of the women were covered by a digitally added surgical face mask. As expected, emotion recognition accuracy declined with age, and face masks reduced emotion recognition accuracy in both younger and older participants. Unexpectedly, the effect of face masks did not differ between age groups. Further analyses showed that masks also reduced the participants’ overall confidence in their emotion judgements, but not their performance awareness (the difference between their confidence ratings for correct and incorrect responses). Again, there were no mask-by-age interactions. Finally, data obtained with a newly developed questionnaire (attitudes towards face masks, atom) suggest that younger and older people do not differ in how much they feel impaired in their understanding of other people’s emotions by face masks or how useful they find face masks in confining the COVID-19 pandemic. In sum, these findings do not provide evidence that the impact of face masks on the decoding of facial signals is disproportionally larger in older people.

Four right-handed, healthy subjects participated in a visual stimulation experiment. Subjects were viewing a dartboard-shaped flickering checkerboard stimulus, divided into 4 rings and 12 segments, defining 48 sectors in the visual field. Local contrast in each sector was continuously varying across four levels and updated every 3 s. To maintain fixation, subjects had to respond to a stimulus at the center of the visual field.During the entire experiment, in which subjects performed 8 runs, each consisting of 100 trials, brain activity was measured with functional magnetic resonance imaging (MRI). Using a 3-T Siemens Trio MRI scanner, 220 echo-planar images were acquired in each run, with a repetition time of 1.5 s and voxel size of 3 x 3 x 3 mm.The dataset is publicly available from OpenNeuro and additionally includes region of interest maps for visual areas V1 to V4, left and right, obtained from another retinotopic mapping experiment. As such, the dataset allows for accurate mapping of receptive fields and their properties across several stages of human visual cortex.

Randomness is a fundamental property of human behavior. It occurs both in the form of intrinsic random variability, say when repetitions of a task yield slightly different behavioral outcomes, or in the form of explicit randomness, say when a person tries to avoid being predicted in a game of rock, paper and scissors. Randomness has frequently been studied using random sequence generation tasks (RSG). A key finding has been that humans are poor at deliberately producing random behavior. At the same time, it has been shown that people might be better randomizers if randomness is only an implicit (rather than an explicit) requirement of the task. We therefore hypothesized that randomization performance might vary with the exact instructions with which randomness is elicited. To test this, we acquired data from a large online sample (n = 388), where every participant made 1,000 binary choices based on one of the following instructions: choose either randomly, freely, irregularly, according to an imaginary coin toss or perform a perceptual guessing task. Our results show significant differences in randomness between the conditions as quantified by conditional entropy and estimated Markov order. The randomization scores were highest in the conditions where people were asked to be irregular or mentally simulate a random event (coin toss) thus yielding recommendations for future studies on randomization behavior.

Abstract Background Although convolutional neural networks (CNNs) achieve high diagnostic accuracy for detecting Alzheimer’s disease (AD) dementia based on magnetic resonance imaging (MRI) scans, they are not yet applied in clinical routine. One important reason for this is a lack of model comprehensibility. Recently developed visualization methods for deriving CNN relevance maps may help to fill this gap as they allow the visualization of key input image features that drive the decision of the model. We investigated whether models with higher accuracy also rely more on discriminative brain regions predefined by prior knowledge. Methods We trained a CNN for the detection of AD in N = 663 T1-weighted MRI scans of patients with dementia and amnestic mild cognitive impairment (MCI) and verified the accuracy of the models via cross-validation and in three independent samples including in total N = 1655 cases. We evaluated the association of relevance scores and hippocampus volume to validate the clinical utility of this approach. To improve model comprehensibility, we implemented an interactive visualization of 3D CNN relevance maps, thereby allowing intuitive model inspection. Results Across the three independent datasets, group separation showed high accuracy for AD dementia versus controls (AUC ≥ 0.91) and moderate accuracy for amnestic MCI versus controls (AUC ≈ 0.74). Relevance maps indicated that hippocampal atrophy was considered the most informative factor for AD detection, with additional contributions from atrophy in other cortical and subcortical regions. Relevance scores within the hippocampus were highly correlated with hippocampal volumes (Pearson’s r ≈ −0.86, p < 0.001). Conclusion The relevance maps highlighted atrophy in regions that we had hypothesized a priori. This strengthens the comprehensibility of the CNN models, which were trained in a purely data-driven manner based on the scans and diagnosis labels. The high hippocampus relevance scores as well as the high performance achieved in independent samples support the validity of the CNN models in the detection of AD-related MRI abnormalities. The presented data-driven and hypothesis-free CNN modeling approach might provide a useful tool to automatically derive discriminative features for complex diagnostic tasks where clear clinical criteria are still missing, for instance for the differential diagnosis between various types of dementia.

Abstract Despite recent developments in integrating autonomous and human-like robots into many aspects of everyday life, social interactions with robots are still a challenge. Here, we focus on a central tool for social interaction: verbal communication. We assess the extent to which humans co-represent (simulate and predict) a robot’s verbal actions. During a joint picture naming task, participants took turns in naming objects together with a social robot (Pepper, Softbank Robotics). Previous findings using this task with human partners revealed internal simulations on behalf of the partner down to the level of selecting words from the mental lexicon, reflected in partner-elicited inhibitory effects on subsequent naming. Here, with the robot, the partner-elicited inhibitory effects were not observed. Instead, naming was facilitated, as revealed by faster naming of word categories co-named with the robot. This facilitation suggests that robots, unlike humans, are not simulated down to the level of lexical selection. Instead, a robot’s speaking appears to be simulated at the initial level of language production where the meaning of the verbal message is generated, resulting in facilitated language production due to conceptual priming. We conclude that robots facilitate core conceptualization processes when humans transform thoughts to language during speaking.

Abstract Neurobiological models of emotion focus traditionally on limbic/paralimbic regions as neural substrates of emotion generation, and insular cortex (in conjunction with isocortical anterior cingulate cortex, ACC) as the neural substrate of feelings. An emerging view, however, highlights the importance of isocortical regions beyond insula and ACC for the subjective feeling of emotions. We used music to evoke feelings of joy and fear, and multivariate pattern analysis (MVPA) to decode representations of feeling states in functional magnetic resonance (fMRI) data of n = 24 participants. Most of the brain regions providing information about feeling representations were neocortical regions. These included, in addition to granular insula and cingulate cortex, primary and secondary somatosensory cortex, premotor cortex, frontal operculum, and auditory cortex. The multivoxel activity patterns corresponding to feeling representations emerged within a few seconds, gained in strength with increasing stimulus duration, and replicated results of a hypothesis-generating decoding analysis from an independent experiment. Our results indicate that several neocortical regions (including insula, cingulate, somatosensory and premotor cortices) are important for the generation and modulation of feeling states. We propose that secondary somatosensory cortex, which covers the parietal operculum and encroaches on the posterior insula, is of particular importance for the encoding of emotion percepts, i.e., preverbal representations of subjective feeling.

Abstract One major bottleneck in the ongoing COVID-19 pandemic is the limited number of critical care beds. Due to the dynamic development of infections and the time lag between when patients are infected and when a proportion of them enters an intensive care unit (ICU), the need for future intensive care can easily be underestimated. To infer future ICU load from reported infections, we suggest a simple statistical model that (1) accounts for time lags and (2) allows for making predictions depending on different future growth of infections. We have evaluated our model for three heavily affected regions in Europe, namely Berlin (Germany), Lombardy (Italy), and Madrid (Spain). Before extensive containment measures made an impact, we first estimate the region-specific model parameters, namely ICU rate, time lag between infection, and ICU admission as well as length of stay in ICU. Whereas for Berlin, an ICU rate of 6%, a time lag of 6 days, and a stay of 12 days in ICU provide the best fit of the data, for Lombardy and Madrid the ICU rate was higher (18% and 15%) and the time lag (0 and 3 days) and the stay in ICU (3 and 8 days) shorter. The region-specific models are then used to predict future ICU load assuming either a continued exponential phase with varying growth rates (0–15%) or linear growth. By keeping the growth rates flexible, this model allows for taking into account the potential effect of diverse containment measures. Thus, the model can help to predict a potential exceedance of ICU capacity depending on future growth. A sensitivity analysis for an extended time period shows that the proposed model is particularly useful for exponential phases of the disease.

Objectives: This study evaluated the impact of the interaction between the anorexigenic incretin hormone glucagon-like peptide-1 (GLP-1) and reward-related brain activity in the dorsolateral prefrontal cortex (DLPFC), a key area of behavioral control, on future weight loss in obese individuals. Methods: We performed a weight loss-weight maintenance intervention study over 27 months. We applied an fMRI food-cue reactivity paradigm during which the participants were passively exposed to food pictures to evaluate neuronal activity in the DLPFC. Additionally, we measured concentrations of circulating GLP-1 levels during a standard oral glucose tolerance test. Phenotyping was performed consecutively before and after a 3-month low-calorie diet as well as after a randomized 12-month trial, investigating the effect of a combined behavioral intervention on body weight maintenance. Participants were then followed-up for another 12 months without further intervention. Results: Using voxel-wise linear mixed-effects regression analyses, we evaluated 56 measurements and identified a strong interaction between circulating, endogenous GLP-1 levels and DLPFC activity predicting body weight change over the total observation period (t = −6.17, p = 1.6 · 10−7). While neither the GLP-1 nor the DLPFC response individually predicted the subsequent weight change, participants achieved body weight loss when the GLP-1 and the DLPFC responses occurred concurrently. Conclusions: Our data demonstrate an interaction between a peripheral hormonal signal and central nervous activity as robust predictor of body weight change throughout the different periods of a long-term life-style intervention. The preeminent role of their interdependency compared to the partly ambivalent effects of the single components argues for integrative approaches to improve sensitivity and reliability of weight prediction conventionally based on individual biomarkers. Keywords: Obesity, Body weight regulation, GLP-1, DLPFC, Food-cue reactivity, Voxel-wise linear mixed-effects regression

Health-related quality of life (HRQoL) is an essential complementary parameter in the assessment of disease burden and treatment outcome in multiple sclerosis (MS) and can be affected by neuropsychiatric symptoms, which in turn are sensitive to psychological stress. However, until now, the impact of neurobiological stress and relaxation on HRQoL in MS has not been investigated. We thus evaluated whether the activity of neural networks triggered by mild psychological stress (elicited in an fMRI task comprising mental arithmetic with feedback) or by stress termination (i.e., relaxation) at baseline (T0) predicts HRQoL variations occurring between T0 and a follow-up visit (T1) in 28 patients using a robust regression and permutation testing. The median delay between T0 and T1 was 902 (range: 363–1,169) days. We assessed HRQoL based on the Hamburg Quality of Life Questionnaire in MS (HAQUAMS) and accounted for the impact of established HRQoL predictors and the cognitive performance of the participants. Relaxation-triggered activity of a widespread neural network predicted future variations in overall HRQoL (t = 3.68, pfamily−wise error [FWE]-corrected = 0.008). Complementary analyses showed that relaxation-triggered activity of the same network at baseline was associated with variations in the HAQUAMS mood subscale on an αFWE = 0.1 level (t = 3.37, pFWE = 0.087). Finally, stress-induced activity of a prefronto-limbic network predicted future variations in the HAQUAMS lower limb mobility subscale (t = −3.62, pFWE = 0.020). Functional neural network measures of psychological stress and relaxation contain prognostic information for future HRQoL evolution in MS independent of clinical predictors.

The diversified methodology and expertise of interdisciplinary research teams provide the opportunity to overcome the limited perspectives of individual disciplines. This is particularly true at the interface of Robotics, Neuroscience, and Psychology as the three fields have quite different perspectives and approaches to offer. Nonetheless, aligning backgrounds and interdisciplinary expectations can present challenges due to varied research cultures and practices. Overcoming these challenges stands at the beginning of each productive collaboration and thus is a mandatory step in cognitive neurorobotics. In this article, we share eight lessons that we learned from our ongoing interdisciplinary project on human-robot and robot-robot interaction in social settings. These lessons provide practical advice for scientists initiating interdisciplinary research endeavors. Our advice can help to avoid early problems and deal with differences between research fields, prepare for and anticipate challenges, align project expectations, and speed up research progress, thus promoting effective interdisciplinary research across Robotics, Neuroscience, and Psychology.

Representations of sensory working memory can be found across the entire neocortex. But how are verbal working memory (VWM) contents retained in the human brain? Here we used fMRI and multi-voxel pattern analyses to study Chinese native speakers (15 males, 13 females) memorizing Chinese characters. Chinese characters are uniquely suitable to study VWM because verbal encoding is encouraged by their complex visual appearance and monosyllabic pronunciation. We found that activity patterns in Broca's area and left premotor cortex carried information about the memorized characters. These language-related areas carried (1) significantly more information about cued characters than those not cued for memorization, (2) significantly more information on the left than the right hemisphere and (3) significantly more information about Chinese symbols than complex visual patterns which are hard to verbalize. In contrast, early visual cortex carries a comparable amount of information about cued and uncued stimuli and is thus unlikely to be involved in memory retention. This study provides evidence for verbal working memory maintenance in a distributed network of language-related brain regions, consistent with distributed accounts of WM. The results also suggest that Broca's area and left premotor cortex form the articulatory network which serves articulatory rehearsal in the retention of verbal working memory contents.

Background: Multiple sclerosis (MS) is characterized by two neuropathological key aspects: inflammation and neurodegeneration. Clinical studies support a prospective link between psychological stress and subsequent inflammatory disease activity. However, it is unknown if a similar link exists for grey matter (GM) degeneration as the key driver of irreversible disability. Methods: We tested whether neural network activity triggered in a psychological fMRI stress paradigm (a mental arithmetic task including social evaluation) conducted at a baseline time point predicts future GM atrophy in 25 persons with MS (14 females). Atrophy was determined between the baseline and a follow-up time point with a median delay of 1012 (Rg: 717–1439) days. Additionally, atrophy was assessed in 22 healthy subjects (13 females; median delay 771 [Rg: 740–908] days between baseline and follow-up) for comparison. Results: An analysis of longitudinal atrophy in patients revealed GM loss in frontal, parietal, and cerebellar areas. Cerebellar atrophy was more pronounced in patients than controls. Future parietal and cerebellar atrophy could be predicted based on activity of two networks. Perceived psychological stress was negatively related to future parietal atrophy in patients and activity of the network predictive of parietal atrophy was positively linked to perceived stress. Conclusions: We have shown that blunted neural and psychological stress processing have a detrimental effect on the course of MS and are interrelated. Together with research showing that psychological and neural stress processing can be altered through interventions, our findings suggest that stress processing might constitute an important modifiable disease factor.

Background: Psychological stress can influence the severity of multiple sclerosis (MS), but little is known about neurobiological factors potentially counteracting these effects.Objective: To identify gray matter (GM) brain regions related to relaxation after stress exposure in persons with MS (PwMS).Methods: 36 PwMS and 21 healthy controls (HCs) reported their feeling of relaxation during a mild stress task. These markers were related to regional GM volumes, heart rate, and depressive symptoms.Results: Relaxation was differentially linked to heart rate in both groups (t = 2.20, p = 0.017), i.e., both markers were only related in HCs. Relaxation was positively linked to depressive symptoms across all participants (t = 1.99, p = 0.045) although this link differed weakly between groups (t = 1.62, p = 0.108). Primarily, the volume in medial temporal gyrus was negatively linked to relaxation in PwMS (t = −5.55, pfamily−wise−error(FWE)corrected = 0.018). A group-specific coupling of relaxation and GM volume was found in ventromedial prefrontal cortex (VMPFC) (t = −4.89, pFWE = 0.039).Conclusion: PwMS appear unable to integrate peripheral stress signals into their perception of relaxation. Together with the group-specific coupling of relaxation and VMPFC volume, a key area of the brain reward system for valuation of affectively relevant stimuli, this finding suggests a clinically relevant misinterpretation of stress-related affective stimuli in MS.

Visual working memory (VWM) allows for keeping visual information available for upcoming goal-directed behavior, while new visual input is processed concurrently. Interactions between the mnemonic and perceptual systems cause VWM to affect the processing of visual input in a content-specific manner: visual input that is initially suppressed from consciousness is detected faster when it matches rather than mismatches the content of VWM. It is currently under debate whether such mnemonic influences on perception occur prior to or after conscious access. To address this issue, we investigated whether VWM content modulates the neural response to visual input that remains suppressed from consciousness. We measured fMRI responses to interocularly suppressed stimuli in 20 human participants performing a delayed match-to-sample task: Participants were retro-cued to memorize one of two geometrical shapes for subsequent recognition. During retention, an interocularly suppressed peripheral stimulus (the probe) was briefly presented, which was either of the cued (memorized) or uncued (not memorized) shape category. We found no evidence that VWM content modulated the neural response to the probe. Substantial evidence for the absence of this modulation was found despite leveraging a highly liberal analysis approach: (1) selecting regions of interest that were particularly prone to detecting said modulation, and (2) using directional Bayesian tests favoring the presence of the hypothesized modulation. We did observe faster detection of memory-matching compared to memory-mismatching probes in a behavioral control experiment, thus validating the stimulus set. We conclude that VWM impacts the processing of visual input only once suppression is mostly alleviated.

Techniques of multivariate pattern analysis (MVPA) can be used to decode the discrete experimental condition or a continuous modulator variable from measured brain activity during a particular trial. In functional magnetic resonance imaging (fMRI), trial-wise response amplitudes are sometimes estimated from the measured signal using a general linear model (GLM) with one onset regressor for each trial. When using rapid event-related designs with trials closely spaced in time, those estimates are highly variable and serially correlated due to the temporally extended shape of the hemodynamic response function (HRF). Here, we describe inverse transformed encoding modelling (ITEM), a principled approach of accounting for those serial correlations and decoding from the resulting estimates, at low computational cost and with no loss in statistical power. We use simulated data to show that ITEM outperforms the current standard approach in terms of decoding accuracy and analyze empirical data to demonstrate that ITEM is capable of visual reconstruction from fMRI signals.

A long-standing position in philosophy, law, and theology is that a person can be held morally responsible for an action only if they had the freedom to choose and to act otherwise. Thus, many philosophers consider freedom to be a necessary condition for moral responsibility. However, empirical findings suggest that this assumption might not be in line with common sense thinking. For example, in a recent study we used surveys to show that – counter to positions held by many philosophers – lay people consider actions to be free when they are spontaneous rather than being based on reasons. In contrast, responsibility is often considered to require that someone has thought about the alternative options. In this study we used an online survey to directly test the degree to which lay judgements of freedom and responsibility match. Specifically, we tested whether manipulations of deliberation affect freedom and responsibility judgements in the same way. Furthermore, we also tested the dependency of these judgements on a person’s belief that their decision had consequences for their personal life. We found that deliberation had an opposite effect on freedom and responsibility judgements. People were considered more free when they acted spontaneously, whereas they were considered more responsible when they deliberated about their actions. These results seem to suggest that deliberating about reasons is crucially important for the lay concept of responsibility, while for the lay notion of freedom it is perceived to be detrimental. One way of interpreting our findings for the interdisciplinary debate on free will and responsibility could be to suggest that lay beliefs match the philosophical position of semi-compatibilism. Semi-compatibilists insist that the metaphysical debate on the nature of free will can be separated from the debate on conditions of responsible agency. According to our findings the beliefs of lay people are in line with views held by semi-compatibilists, even though we did not test whether they endorse that position explicitly.

Most people believe in free will. Whether this belief is warranted or not, free will beliefs (FWB) are foundational for many legal systems and reducing FWB has effects on behavior from the motor to the social level. This raises the important question as to which specific FWB people hold. There are many different ways to conceptualize free will, and some might see physical determinism as a threat that might reduce FWB, while others might not. Here, we investigate lay FWB in a large, representative, replicated online survey study in the US and Singapore (n = 1800), assessing differences in FWB with unprecedented depth within and between cultures. Specifically, we assess the relation of FWB, as measured using the Free Will Inventory, to determinism, dualism and related concepts like libertarianism and compatibilism. We find that libertarian, compatibilist, and dualist, intuitions were related to FWB, but that these intuitions were often logically inconsistent. Importantly, direct comparisons suggest that dualism was more predictive of FWB than other intuitions. Thus, believing in free will goes hand-in-hand with a belief in a non-physical mind. Highlighting the importance of dualism for FWB impacts academic debates on free will, which currently largely focus on its relation to determinism. Our findings also shed light on how recent (neuro)scientific findings might impact FWB. Demonstrating physical determinism in the brain need not have a strong impact on FWB, due to a wide-spread belief in dualism.

Machine learning-based imaging diagnostics has recently reached or even surpassed the level of clinical experts in several clinical domains. However, classification decisions of a trained machine learning system are typically non-transparent, a major hindrance for clinical integration, error tracking or knowledge discovery. In this study, we present a transparent deep learning framework relying on 3D convolutional neural networks (CNNs) and layer-wise relevance propagation (LRP) for diagnosing multiple sclerosis (MS), the most widespread autoimmune neuroinflammatory disease. MS is commonly diagnosed utilizing a combination of clinical presentation and conventional magnetic resonance imaging (MRI), specifically the occurrence and presentation of white matter lesions in T2-weighted images. We hypothesized that using LRP in a naive predictive model would enable us to uncover relevant image features that a trained CNN uses for decision-making. Since imaging markers in MS are well-established this would enable us to validate the respective CNN model. First, we pre-trained a CNN on MRI data from the Alzheimer's Disease Neuroimaging Initiative (n = 921), afterwards specializing the CNN to discriminate between MS patients (n = 76) and healthy controls (n = 71). Using LRP, we then produced a heatmap for each subject in the holdout set depicting the voxel-wise relevance for a particular classification decision. The resulting CNN model resulted in a balanced accuracy of 87.04% and an area under the curve of 96.08% in a receiver operating characteristic curve. The subsequent LRP visualization revealed that the CNN model focuses indeed on individual lesions, but also incorporates additional information such as lesion location, non-lesional white matter or gray matter areas such as the thalamus, which are established conventional and advanced MRI markers in MS. We conclude that LRP and the proposed framework have the capability to make diagnostic decisions of CNN models transparent, which could serve to justify classification decisions for clinical review, verify diagnosis-relevant features and potentially gather new disease knowledge. Keywords: Convolutional neural networks deep learning multiple sclerosis MRI, Layer-wise relevance propagation, Visualization transfer learning

When attention has to be maintained over prolonged periods performance slowly fluctuates and errors can occur. It has been shown that lapses of attention are correlated with BOLD signals in frontal and parietal cortex. This raises the question how attentional fluctuations are linked to the fronto-parietal default network. Because the attentional state fluctuates slowly we expect that potential links between attentional fluctuations and brain activity should be observable on longer time scales and importantly also before the execution of the task. In the present study we used fMRI to identify brain activity that is correlated with vigilance, defined as fluctuations of reaction times (RT) during a sustained attention task. We found that brain activity in visual cortex, parietal lobe (PL), inferior and superior frontal gyrus, and supplementary motor area (SMA) was higher when the subject had a relatively long RT. In contrast to our expectations, activity in the default network (DN) was higher when subjects had a relatively short RT, that means when the performance was improved. This modulation in the DN was present already several seconds before the task execution, thus pointing to activity in the DN as a potential cause of performance increases in simple repetitive tasks.

Abstract Background This study investigates the prediction of mild cognitive impairment‐to‐Alzheimer's disease (MCI‐to‐AD) conversion based on extensive multimodal data with varying degrees of missing values. Methods Based on Alzheimer's Disease Neuroimaging Initiative data from MCI‐patients including all available modalities, we predicted the conversion to AD within 3 years. Different ways of replacing missing data in combination with different classification algorithms are compared. The performance was evaluated on features prioritized by experts and automatically selected features. Results The conversion to AD could be predicted with a maximal accuracy of 73% using support vector machines and features chosen by experts. Among data modalities, neuropsychological, magnetic resonance imaging, and positron emission tomography data were most informative. The best single feature was the functional activities questionnaire. Conclusion Extensive multimodal and incomplete data can be adequately handled by a combination of missing data substitution, feature selection, and classification.

Functional neuroimaging has provided evidence for altered function of mesolimbic circuits implicated in reward processing, first and foremost the ventral striatum, in patients with schizophrenia. While such findings based on significant group differences in brain activations can provide important insights into the pathomechanisms of mental disorders, the use of neuroimaging results from standard univariate statistical analysis for individual diagnosis has proven difficult. In this proof of concept study, we tested whether the predictive accuracy for the diagnostic classification of schizophrenia patients vs. healthy controls could be improved using multivariate pattern analysis (MVPA) of regional functional magnetic resonance imaging (fMRI) activation patterns for the anticipation of monetary reward. With a searchlight MVPA approach using support vector machine classification, we found that the diagnostic category could be predicted from local activation patterns in frontal, temporal, occipital and midbrain regions, with a maximal cluster peak classification accuracy of 93% for the right pallidum. Region-of-interest based MVPA for the ventral striatum achieved a maximal cluster peak accuracy of 88%, whereas the classification accuracy on the basis of standard univariate analysis reached only 75%. Moreover, using support vector regression we could additionally predict the severity of negative symptoms from ventral striatal activation patterns. These results show that MVPA can be used to substantially increase the accuracy of diagnostic classification on the basis of task-related fMRI signal patterns in a regionally specific way.

Sustained, long-term cognitive workload is associated with variations and decrements in performance. Such fluctuations in vigilance can be a risk factor especially during dangerous attention demanding activities. Functional MRI studies have shown that attentional performance is correlated with BOLD-signals, especially in parietal and prefrontal cortical regions. An interesting question is whether these BOLD-signals could be measured in real-world scenarios, say to warn in a dangerous workplace whenever a subjects' vigilance is low. Because fMRI lacks the mobility needed for such applications, we tested whether the monitoring of vigilance might be possible using Near-Infrared Spectroscopy (NIRS). NIRS is a highly mobile technique that measures hemodynamics in the surface of the brain. We demonstrate that non-invasive NIRS signals correlate with vigilance. These signals carry enough information to decode subjects' reaction times at a single trial level.

ObjectiveHere, we use pattern-classification to investigate diagnostic information for multiple sclerosis (MS; relapsing-remitting type) in lesioned areas, areas of normal-appearing grey matter (NAGM), and normal-appearing white matter (NAWM) as measured by standard MR techniques.MethodsA lesion mapping was carried out by an experienced neurologist for Turbo Inversion Recovery Magnitude (TIRM) images of individual subjects. Combining this mapping with templates from a neuroanatomic atlas, the TIRM images were segmented into three areas of homogenous tissue types (Lesions, NAGM, and NAWM) after spatial standardization. For each area, a linear Support Vector Machine algorithm was used in multiple local classification analyses to determine the diagnostic accuracy in separating MS patients from healthy controls based on voxel tissue intensity patterns extracted from small spherical subregions of these larger areas. To control for covariates, we also excluded group-specific biases in deformation fields as a potential source of information.ResultsAmong regions containing lesions a posterior parietal WM area was maximally informative about the clinical status (96% accuracy, pInterpretationWe identified regions indicating MS in lesioned, but also NAGM, and NAWM areas. This complements the current perception that standard MR techniques mainly capture macroscopic tissue variations due to focal lesion processes. Compared to current diagnostic guidelines for MS that define areas of diagnostic information with moderate spatial specificity, we identified hotspots of MS associated tissue alterations with high specificity defined on a millimeter scale.

Recently, we demonstrated using functional magnetic resonance imaging (fMRI) that the outcome of free decisions can be decoded from brain activity several seconds before reaching conscious awareness. Activity patterns in anterior frontopolar cortex (BA 10) were temporally the first to carry intention-related information and thus a candidate region for the unconscious generation of free decisions. In the present study, the original paradigm was replicated and multivariate pattern classification was applied to functional images of frontopolar cortex, acquired using ultra-high field fMRI at 7 Tesla. Here, we show that predictive activity patterns recorded before a decision was made became increasingly stable with increasing temporal proximity to the time point of the conscious decision. Furthermore, detailed questionnaires exploring subjects' thoughts before and during the decision confirmed that decisions were made spontaneously and subjects were unaware of the evolution of their decision outcomes. These results give further evidence that FPC stands at the top of the prefrontal executive hierarchy in the unconscious generation of free decisions.

The perception of brightness depends on spatial context: the same stimulus can appear light or dark depending on what surrounds it. A less well-known but equally important contextual phenomenon is that the colour of a stimulus can also alter its brightness. Specifically, stimuli that are more saturated (i.e. purer in colour) appear brighter than stimuli that are less saturated at the same luminance. Similarly, stimuli that are red or blue appear brighter than equiluminant yellow and green stimuli. This non-linear relationship between stimulus intensity and brightness, called the Helmholtz-Kohlrausch (HK) effect, was first described in the nineteenth century but has never been explained. Here, we take advantage of the relative simplicity of this 'illusion' to explain it and contextual effects more generally, by using a simple Bayesian ideal observer model of the human visual ecology. We also use fMRI brain scans to identify the neural correlates of brightness without changing the spatial context of the stimulus, which has complicated the interpretation of related fMRI studies.Rather than modelling human vision directly, we use a Bayesian ideal observer to model human visual ecology. We show that the HK effect is a result of encoding the non-linear statistical relationship between retinal images and natural scenes that would have been experienced by the human visual system in the past. We further show that the complexity of this relationship is due to the response functions of the cone photoreceptors, which themselves are thought to represent an efficient solution to encoding the statistics of images. Finally, we show that the locus of the response to the relationship between images and scenes lies in the primary visual cortex (V1), if not earlier in the visual system, since the brightness of colours (as opposed to their luminance) accords with activity in V1 as measured with fMRI.The data suggest that perceptions of brightness represent a robust visual response to the likely sources of stimuli, as determined, in this instance, by the known statistical relationship between scenes and their retinal responses. While the responses of the early visual system (receptors in this case) may represent specifically the statistics of images, post receptor responses are more likely represent the statistical relationship between images and scenes. A corollary of this suggestion is that the visual cortex is adapted to relate the retinal image to behaviour given the statistics of its past interactions with the sources of retinal images: the visual cortex is adapted to the signals it receives from the eyes, and not directly to the world beyond.