Search

Background: Standardized neuropsychological testing serves to quantify cognitive impairment in multiple sclerosis (MS) patients. However, the exact mechanism underlying the translation of cognitive dysfunction into difficulties in everyday tasks has remained unclear. To answer this question, we tested if MS patients with intact vs. impaired information processing speed measured by the Symbol Digit Modalities Test (SDMT) differ in their visual search behavior during ecologically valid tasks reflecting everyday activities., Methods: Forty-three patients with relapsing-remitting MS enrolled in an eye-tracking experiment consisting of a visual search task with naturalistic images. Patients were grouped into "impaired" and "unimpaired" according to their SDMT performance. Reaction time, accuracy and eye-tracking parameters were measured., Results: The groups did not differ regarding age, gender, and visual acuity. Patients with impaired SDMT (cut-off SDMT-z-score < -1.5) performance needed more time to find and fixate the target ( q = 0.006). They spent less time fixating the target ( q = 0.042). Impaired patients had slower reaction times and were less accurate (both q = 0.0495) even after controlling for patients' upper extremity function. Exploratory analysis revealed that unimpaired patients had higher accuracy than impaired patients particularly when the announced target was in unexpected location ( p = 0.037). Correlational analysis suggested that SDMT performance is inversely linked to the time to first fixation of the target only if the announced target was in its expected location ( r = -0.498, p = 0.003 vs. r = -0.212, p = 0.229)., Conclusion: Dysfunctional visual search behavior may be one of the mechanisms translating cognitive deficits into difficulties in everyday tasks in MS patients. Our results suggest that cognitively impaired patients search their visual environment less efficiently and this is particularly evident when top-down processes have to be employed., Competing Interests: YY has been supported by travel grants from Novartis and Sanofi Genzyme, has received an honorarium for active participation in an advisory board by Sanofi Genzyme as well as speaking honoraria by Roche and Sanofi Genzyme. CF reports speaker honoraria and honoraria for participating in advisory boards from Alexion, Bristol Myers Sqibb, Novartis, Teva, Merck, Sanofi-Genzyme, and Roche. CF received research support from Novartis and Sanofi-Genzyme. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2022 Gehrig, Bergmann, Fadai, Soydaş, Buschenlange, Naumer, Kaiser, Frisch, Behrens, Foerch and Yalachkov.)

Aging is accompanied by unisensory decline. To compensate for this, two complementary strategies are potentially relied upon increasingly: first, older adults integrate more information from different sensory organs. Second, according to the predictive coding (PC) model, we form "templates" (internal models or "priors") of the environment through our experiences. It is through increased life experience that older adults may rely more on these templates compared to younger adults. Multisensory integration and predictive coding would be effective strategies for the perception of near-threshold stimuli, which may however come at the cost of integrating irrelevant information. Both strategies can be studied in multisensory illusions because these require the integration of different sensory information, as well as an internal model of the world that can take precedence over sensory input. Here, we elicited a classic multisensory illusion, the sound-induced flash illusion, in younger (mean: 27 years, N = 25) and older (mean: 67 years, N = 28) adult participants while recording the magnetoencephalogram. Older adults perceived more illusions than younger adults. Older adults had increased pre-stimulus beta-band activity compared to younger adults as predicted by microcircuit theories of predictive coding, which suggest priors and predictions are linked to beta-band activity. Transfer entropy analysis and dynamic causal modeling of pre-stimulus magnetoencephalography data revealed a stronger illusion-related modulation of cross-modal connectivity from auditory to visual cortices in older compared to younger adults. We interpret this as the neural correlate of increased reliance on a cross-modal predictive template in older adults leading to the illusory percept., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2021 Chan, Wibral, Stawowsky, Brandl, Helbling, Naumer, Kaiser and Wollstadt.)

In Autism Spectrum Disorders (ASD), impaired response inhibition and lack of adaptation are hypothesized to underlie core ASD symptoms, such as social communication and repetitive, stereotyped behavior. Thus, the aim of the present study was to compare neural correlates of inhibition, post-error adaptation, and reaction time variability in ASD and neuro-typical control (NTC) participants by investigating possible differences in error-related changes of oscillatory MEG activity. Twelve male NTC (mean age 20.3 ± 3.7) and fourteen male patients with ASD (mean age 17.8 ± 2.9) were included in the analysis. Subjects with ASD showed increased error-related reaction time variability. MEG analysis revealed decreased beta power in the ASD group in comparison to the NTC group over the centro-parietal channels in both, the pre-stimulus and post-response interval. In the ASD group, mean centro-parietal beta power negatively correlated with dimensional autism symptoms. In both groups, false alarms were followed by an early increase in temporo-frontal theta to alpha power; and by a later decrease in alpha to beta power at central and posterior sensors. Single trial correlations were additionally studied in the ASD group, who showed a positive correlation of pre-stimulus beta power with post-response theta, alpha, and beta power, particularly after hit trials. On a broader scale, the results deliver important insights into top-down control deficits that may relate to core symptoms observed in ASD.

Multisensory integration strongly depends on the temporal proximity between two inputs. In the audio-visual domain, stimulus pairs with delays up to a few hundred milliseconds can be perceived as simultaneous and integrated into a unified percept. Previous research has shown that the size of this temporal window of integration can be narrowed by feedback-guided training on an audio-visual simultaneity judgment task. Yet, it has remained uncertain how the neural network that processes audio-visual asynchronies is affected by the training. In the present study, participants were trained on a 2-interval forced choice audio-visual simultaneity judgment task. We recorded their neural activity with magnetoencephalography in response to three different stimulus onset asynchronies (0 ms, each participant's individual binding window, 300 ms) before, and one day following training. The Individual Window stimulus onset asynchrony condition was derived by assessing each participant's point of subjective simultaneity. Training improved performance in both asynchronous stimulus onset conditions (300 ms, Individual Window). Furthermore, beta-band amplitude (12-30 Hz) increased from pre-compared to post-training sessions. This increase moved across central, parietal, and temporal sensors during the time window of 80-410 ms post-stimulus onset. Considering the putative role of beta oscillations in carrying feedback from higher to lower cortical areas, these findings suggest that enhanced top-down modulation of sensory processing is responsible for the improved temporal acuity after training. As beta oscillations can be assumed to also preferentially support neural communication over longer conduction delays, the widespread topography of our effect could indicate that training modulates not only processing within primary sensory cortex, but rather the communication within a large-scale network., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Objective: To determine whether the performance of multiple sclerosis (MS) patients in the sound-induced flash illusion (SiFi), a multisensory perceptual illusion, would reflect their cognitive impairment. Methods: We performed the SiFi task as well as an extensive neuropsychological testing in 95 subjects [39 patients with relapse-remitting MS (RRMS), 16 subjects with progressive multiple sclerosis (PMS) and 40 healthy control subjects (HC)]. Results: MS patients reported more frequently the multisensory SiFi than HC. In contrast, there were no group differences in the control conditions. Essentially, patients with progressive type of MS continued to perceive the illusion at stimulus onset asynchronies (SOA) that were more than three times longer than the SOA at which the illusion was already disrupted for healthy controls. Furthermore, MS patients' degree of cognitive impairment measured with a broad neuropsychological battery encompassing tests for memory, attention, executive functions, and fluency was predicted by their performance in the SiFi task for the longest SOA of 500 ms. Conclusions: These findings support the notion that MS patients exhibit an altered multisensory perception in the SiFi task and that their susceptibility to the perceptual illusion is negatively correlated with their neuropsychological test performance. Since MS lesions affect white matter tracts and cortical regions which seem to be involved in the transfer and processing of both crossmodal and cognitive information, this might be one possible explanation for our findings. SiFi might be considered as a brief, non-expensive, language- and education-independent screening test for cognitive deficits in MS patients.

The neurophysiological underpinnings of the nonsocial symptoms of autism spectrum disorder (ASD) which include sensory and perceptual atypicalities remain poorly understood. Well-known accounts of less dominant top-down influences and more dominant bottom-up processes compete to explain these characteristics. These accounts have been recently embedded in the popular framework of predictive coding theory. To differentiate between competing accounts, we studied altered information dynamics in ASD by quantifying predictable information in neural signals. Predictable information in neural signals measures the amount of stored information that is used for the next time step of a neural process. Thus, predictable information limits the (prior) information which might be available for other brain areas, for example, to build predictions for upcoming sensory information. We studied predictable information in neural signals based on resting-state magnetoencephalography (MEG) recordings of 19 ASD patients and 19 neurotypical controls aged between 14 and 27 years. Using whole-brain beamformer source analysis, we found reduced predictable information in ASD patients across the whole brain, but in particular in posterior regions of the default mode network. In these regions, epoch-by-epoch predictable information was positively correlated with source power in the alpha and beta frequency range as well as autocorrelation decay time. Predictable information in precuneus and cerebellum was negatively associated with nonsocial symptom severity, indicating a relevance of the analysis of predictable information for clinical research in ASD. Our findings are compatible with the assumption that use or precision of prior knowledge is reduced in ASD patients., (© 2018 Wiley Periodicals, Inc.)

Our ability to select relevant information from the environment is limited by the resolution of attention - i.e., the minimum size of the region that can be selected. Neural mechanisms that underlie this limit and its development are not yet understood. Functional magnetic resonance imaging (fMRI) was performed during an object tracking task in 7- and 11-year-old children, and in young adults. Object tracking activated canonical fronto-parietal attention systems and motion-sensitive area MT in children as young as 7 years. Object tracking performance improved with age, together with stronger recruitment of parietal attention areas and a shift from low-level to higher-level visual areas. Increasing the required resolution of spatial attention - which was implemented by varying the distance between target and distractors in the object tracking task - led to activation increases in fronto-insular cortex, medial frontal cortex including anterior cingulate cortex (ACC) and supplementary motor area, superior colliculi, and thalamus. This core circuitry for attentional precision was recruited by all age groups, but ACC showed an age-related activation reduction. Our results suggest that age-related improvements in selective visual attention and in the resolution of attention are characterized by an increased use of more functionally specialized brain regions during the course of development.

In the later stages of addiction, automatized processes play a prominent role in guiding drug-seeking and drug-taking behavior. However, little is known about the neural correlates of automatized drug-taking skills and drug-related action knowledge in humans. We employed functional magnetic resonance imaging (fMRI) while smokers and non-smokers performed an orientation affordance task, where compatibility between the hand used for a behavioral response and the spatial orientation of a priming stimulus leads to shorter reaction times resulting from activation of the corresponding motor representations. While non-smokers exhibited this behavioral effect only for control objects, smokers showed the affordance effect for both control and smoking-related objects. Furthermore, smokers exhibited reduced fMRI activation for smoking-related as compared to control objects for compatible stimulus-response pairings in a sensorimotor brain network consisting of the right primary motor cortex, supplementary motor area, middle occipital gyrus, left fusiform gyrus and bilateral cingulate gyrus. In the incompatible condition, we found higher fMRI activation in smokers for smoking-related as compared to control objects in the right primary motor cortex, cingulate gyrus, and left fusiform gyrus. This suggests that the activation and performance of deeply embedded, automatized drug-taking schemata employ less brain resources. This might reduce the threshold for relapsing in individuals trying to abstain from smoking. In contrast, the interruption or modification of already triggered automatized action representations require increased neural resources.

Visuo-haptic integration contributes essentially to object shape recognition. Although there has been a considerable advance in elucidating the neural underpinnings of multisensory perception, it is still unclear whether seeing an object and exploring it with the dominant hand elicits the same brain response as compared to the non-dominant hand. Using fMRI to measure brain activation in right-handed participants, we found that for both left- and right-hand stimulation the left lateral occipital complex (LOC) and anterior cerebellum (aCER) were involved in visuo-haptic integration of familiar objects. These two brain regions were then further investigated in another study, where unfamiliar, novel objects were presented to a different group of right-handers. Here the left LOC and aCER were more strongly activated by bimodal than unimodal stimuli only when the left but not the right hand was used. A direct comparison indicated that the multisensory gain of the fMRI activation was significantly higher for the left than the right hand. These findings are in line with the principle of "inverse effectiveness", implying that processing of bimodally presented stimuli is particularly enhanced when the unimodal stimuli are weak. This applies also when right-handed subjects see and simultaneously touch unfamiliar objects with their non-dominant left hand. Thus, the fMRI signal in the left LOC and aCER induced by visuo-haptic stimulation is dependent on which hand was employed for haptic exploration., (Copyright © 2015 Elsevier B.V. All rights reserved.)

Previous studies investigating mild cognitive impairment (MCI) have focused primarily on cognitive, memory, attention, and executive function deficits. There has been relatively little research on the perceptual deficits people with MCI may exhibit. This is surprising given that it has been suggested that sensory and cognitive functions share a common cortical framework [1]. In the following study, we presented the sound-induced flash illusion (SiFi) to a group of participants with mild cognitive impairment (MCI) and healthy controls (HC). The SiFi is an audio-visual illusion whereby two-beeps and one-flash are presented. Participants tend to perceive two flashes when the time-interval between the auditory beeps is small [2, 3]. Participants with MCI perceived significantly more illusions compared to HC over longer auditory time-intervals. This suggests that MCIs integrate more (arguably irrelevant) audiovisual information compared to HCs. By incorporating perceptual tasks into a clinical diagnosis it may be possible to gain a more comprehensive understanding into the disease, as well as provide a more accurate diagnose to those who may have a language impairment.

In this article, we review a recent paper by Stevenson et al. (J Neurosci 34: 691-697, 2014). This paper illustrates the need to present different forms of stimuli in order to characterize the perceptual abilities of people with autism spectrum disorder (ASD). Furthermore, we will discuss their behavioral results and offer an opposing viewpoint to the suggested neuronal drivers of ASD., (Copyright © 2014 the American Physiological Society.)

The car dependence of people living in contemporary cities is a major concern for policy makers, who often find it difficult to persuade people into more sustainable transport modes. By contrast, recent insights from neuroscience have shown that a broad spectrum of behaviors can become habitual and, thus, resistant to change. Here, we outline the potential of collaboration between neuroscience and human geography aiming at a better understanding of habits that determine everyday commuting routines., (Copyright © 2014 Elsevier Ltd. All rights reserved.)

The most common form of synaesthesia is grapheme-colour synaesthesia. However, rarer forms of synaesthesia also exist, such as word-gustatory and olfactory-gustatory synaesthesia, whereby a word or smell will induce a specific. In this study we describe a single individual (LJ) who experiences a concurrent olfactory stimulus when presented with congruent visual images. For some visual stimuli, he perceives a strong and automatic olfactory percept, which has existed throughout his life. In this study, we explore whether his experiences are a new form of synaesthesia or simply vivid imagery. Unlike other forms of synaesthesia, the concurrent odour is congruent to the visual inducer. For example, a photograph of dress shoes will elicit the smell of leather. We presented LJ and several control participants with 75 images of everyday objects. Their task was to indicate the strength of any perceived odours induced by the visual images. LJ rated several of the images as inducing a concurrent odour, while controls did not have any such percept. Images that LJ reported as inducing the strongest odours were used, along with colour-matched control images, in the context of an fMRI experiment. Participants were given a one-back task to maintain attention. A block-design odour localizer was presented to localize the piriform cortex (primary olfactory cortex). We found an increased BOLD response in the piriform cortex for the odour-inducing images compared to the control images in LJ. There was no difference in BOLD response between these two stimulus types in the control participants. A subsequent olfactory imagery task did not elicit enhanced activity in the piriform cortex in LJ, suggesting his perceptual experiences may not be based on olfactory imagery.

Human neuroimaging studies suggest that neural cue reactivity is strongly associated with indices of drug use, including addiction severity and treatment success. However, little is known about factors that modulate cue reactivity. The goal of this review, in which we survey published fMRI and PET studies on drug cue reactivity in cocaine, alcohol, and tobacco cigarette users, is to highlight major factors that modulate brain reactivity to drug cues. First, we describe cue reactivity paradigms used in neuroimaging research and outline the brain circuits that underlie cue reactivity. We then discuss major factors that have been shown to modulate cue reactivity and review specific evidence as well as outstanding questions related to each factor. Building on previous model-building reviews on the topic, we then outline a simplified model that includes the key modulatory factors and a tentative ranking of their relative impact. We conclude with a discussion of outstanding challenges and future research directions, which can inform future neuroimaging studies as well as the design of treatment and prevention programs., (Published by Elsevier Ltd.)

Rationale: Behavioral experiments have demonstrated that the sensory modality of presentation modulates drug cue reactivity., Objectives: The present study on nicotine addiction tested whether neural responses to smoking cues are modulated by the sensory modality of stimulus presentation., Methods: We measured brain activation using functional magnetic resonance imaging (fMRI) in 15 smokers and 15 nonsmokers while they viewed images of smoking paraphernalia and control objects and while they touched the same objects without seeing them., Results: Haptically presented, smoking-related stimuli induced more pronounced neural cue reactivity than visual cues in the left dorsal striatum in smokers compared to nonsmokers. The severity of nicotine dependence correlated positively with the preference for haptically explored smoking cues in the left inferior parietal lobule/somatosensory cortex, right fusiform gyrus/inferior temporal cortex/cerebellum, hippocampus/parahippocampal gyrus, posterior cingulate cortex, and supplementary motor area., Conclusions: These observations are in line with the hypothesized role of the dorsal striatum for the expression of drug habits and the well-established concept of drug-related automatized schemata, since haptic perception is more closely linked to the corresponding object-specific action pattern than visual perception. Moreover, our findings demonstrate that with the growing severity of nicotine dependence, brain regions involved in object perception, memory, self-processing, and motor control exhibit an increasing preference for haptic over visual smoking cues. This difference was not found for control stimuli. Considering the sensory modality of the presented cues could serve to develop more reliable fMRI-specific biomarkers, more ecologically valid experimental designs, and more effective cue-exposure therapies of addiction.

Neuroimaging studies on cue reactivity have substantially contributed to the understanding of addiction. In the majority of studies drug cues were presented in the visual modality. However, exposure to conditioned cues in real life occurs often simultaneously in more than one sensory modality. Therefore, multisensory cues should elicit cue reactivity more consistently than unisensory stimuli and increase the ecological validity and the reliability of brain activation measurements. This review includes the data from 44 whole-brain functional neuroimaging studies with a total of 1168 subjects (812 patients and 356 controls). Correlations between neural cue reactivity and clinical covariates such as craving have been reported significantly more often for multisensory than unisensory cues in the motor cortex, insula and posterior cingulate cortex. Thus, multisensory drug cues are particularly effective in revealing brain-behavior relationships in neurocircuits of addiction responsible for motivation, craving awareness and self-related processing., (Copyright © 2011 Elsevier Ltd. All rights reserved.)

Human neuroplasticity of multisensory integration has been studied mainly in the context of natural or artificial training situations in healthy subjects. However, regular smokers also offer the opportunity to assess the impact of intensive daily multisensory interactions with smoking-related objects on the neural correlates of crossmodal object processing. The present functional magnetic resonance imaging study revealed that smokers show a comparable visuo-haptic integration pattern for both smoking paraphernalia and control objects in the left lateral occipital complex, a region playing a crucial role in crossmodal object recognition. Moreover, the degree of nicotine dependence correlated positively with the magnitude of visuo-haptic integration in the left lateral occipital complex (LOC) for smoking-associated but not for control objects. In contrast, in the left LOC non-smokers displayed a visuo-haptic integration pattern for control objects, but not for smoking paraphernalia. This suggests that prolonged smoking-related multisensory experiences in smokers facilitate the merging of visual and haptic inputs in the lateral occipital complex for the respective stimuli. Studying clinical populations who engage in compulsive activities may represent an ecologically valid approach to investigating the neuroplasticity of multisensory integration., (Copyright © 2011 Elsevier Inc. All rights reserved.)

Primate multisensory object perception involves distributed brain regions. To investigate the network character of these regions of the human brain, we applied data-driven group spatial independent component analysis (ICA) to a functional magnetic resonance imaging (fMRI) data set acquired during a passive audio-visual (AV) experiment with common object stimuli. We labeled three group-level independent component (IC) maps as auditory (A), visual (V), and AV, based on their spatial layouts and activation time courses. The overlap between these IC maps served as definition of a distributed network of multisensory candidate regions including superior temporal, ventral occipito-temporal, posterior parietal and prefrontal regions. During an independent second fMRI experiment, we explicitly tested their involvement in AV integration. Activations in nine out of these twelve regions met the max-criterion (A < AV > V) for multisensory integration. Comparison of this approach with a general linear model-based region-of-interest definition revealed its complementary value for multisensory neuroimaging. In conclusion, we estimated functional networks of uni- and multisensory functional connectivity from one dataset and validated their functional roles in an independent dataset. These findings demonstrate the particular value of ICA for multisensory neuroimaging research and using independent datasets to test hypotheses generated from a data-driven analysis.

The study of Wagner et al. (J Neurosci 31: 894-898, 2011) reveals the neural correlates of spontaneously activated action representations in smokers when subjects watch movie characters smoke. We stress the importance of differentiating how these representations are activated: while the anterior intraparietal sulcus and inferior frontal gyrus are part of the mirror neuron system of smokers, the middle frontal gyrus, premotor cortex, and superior parietal lobule represent the smoking-related tool use skills and action knowledge activated by smoking paraphernalia.

The processing of visual and haptic inputs, occurring either separately or jointly, is crucial for everyday-life object recognition, and has been a focus of recent neuroimaging research. Previously, visuohaptic convergence has been mostly investigated with matching-task paradigms. However, much less is known about visuohaptic convergence in the absence of additional task demands. We conducted two functional magnetic resonance imaging experiments in which subjects actively touched and/or viewed unfamiliar object stimuli without any additional task demands. In addition, we performed two control experiments with audiovisual and audiohaptic stimulation to examine the specificity of the observed visuohaptic convergence effects. We found robust visuohaptic convergence in bilateral lateral occipital cortex and anterior cerebellum. In contrast, neither the anterior cerebellum nor the lateral occipital cortex showed any involvement in audiovisual or audiohaptic convergence, indicating that multisensory convergence in these regions is specifically geared to visual and haptic inputs. These data suggest that in humans the lateral occipital cortex and the anterior cerebellum play an important role in visuohaptic processing even in the absence of additional task demands.

Research on the psychological and neuronal underpinnings of addiction has concentrated mostly on affective, motivational, learning, and executive processes and the brain regions subserving these functions. In contrast, sensory and motor aspects of addiction have largely been neglected even though they may be highly relevant for the development and preservation of addiction. The aim of the present review is to emphasize the significance of sensory and motor processes for the better understanding and treatment of addiction. We focus on research investigating substance-related addictions and on brain imaging studies in humans that have assessed the contribution of cortical and cerebellar systems to sensory and motor mechanisms of addiction. Activations of sensory and motor brain regions in response to drug-associated cues can predict relapse and correlate with craving, severity of dependence and automatized behavioral reactions towards drug-related stimuli. We propose a model of how sensory and motor processes might be involved in the different stages of addiction., (Copyright 2009 Elsevier B.V. All rights reserved.)

Information integration across different sensory modalities contributes to object recognition, the generation of associations and long-term memory representations. Here, we used functional magnetic resonance imaging adaptation to investigate the presence of sensory integrative effects at cortical levels as early as nonprimary auditory and extrastriate visual cortices, which are implicated in intermediate stages of object processing. Stimulation consisted of an adapting audiovisual stimulus S(1) and a subsequent stimulus S(2) from the same basic-level category (e.g., cat). The stimuli were carefully balanced with respect to stimulus complexity and semantic congruency and presented in four experimental conditions: (1) the same image and vocalization for S(1) and S(2), (2) the same image and a different vocalization, (3) different images and the same vocalization, or (4) different images and vocalizations. This two-by-two factorial design allowed us to assess the contributions of auditory and visual stimulus repetitions and changes in a statistically orthogonal manner. Responses in visual regions of right fusiform gyrus and right lateral occipital cortex were reduced for repeated visual stimuli (repetition suppression). Surprisingly, left lateral occipital cortex showed stronger responses to repeated auditory stimuli (repetition enhancement). Similarly, auditory regions of interest of the right middle superior temporal gyrus and sulcus exhibited repetition suppression to auditory repetitions and repetition enhancement to visual repetitions. Our findings of crossmodal repetition-related effects in cortices of the respective other sensory modality add to the emerging view that in human subjects sensory integrative mechanisms operate on earlier cortical processing levels than previously assumed.

In mammals smooth retinotopic maps of the visual field are formed along the visual processing pathway whereby the left visual field is represented in the right hemisphere and vice versa. The reorganization of retinotopic maps in the lateral geniculate nucleus (LGN) of the thalamus and early visual areas (V1-V3) is studied in a patient who was born with only one cerebral hemisphere. Before the seventh week of embryonic gestation, the development of the patient's right cerebral hemisphere terminated. Despite the complete loss of her right hemisphere (di- and telencephalon) at birth, the patient's remaining hemisphere has not only developed maps of the contralateral (right) visual hemifield but, surprisingly, also maps of the ipsilateral (left) visual hemifield. Retinal ganglion-cells changed their predetermined crossing pattern in the optic chiasm and grew to the ipsilateral LGN. In the visual cortex, islands of ipsilateral visual field representations were located along the representations of the vertical meridian. In V1, smooth and continuous maps from contra- and ipsilateral hemifield overlap each other, whereas in ventral V2 and V3 ipsilateral quarter field representations invaded small distinct cortical patches. This reveals a surprising flexibility of the self-organizing developmental mechanisms responsible for map formation.

To date, noninvasive neuroimaging research on multisensory perception has focused on cortical activations. In a series of elegant functional magnetic resonance imaging experiments, Beauchamp and Ro recently investigated altered cortical activations associated with acquired sound-touch synesthesia resulting from a thalamic lesion. Their findings highlight the important role of intact thalamocortical projections for preventing illusory crossmodal perception and for underlying reliable multisensory integration.

Several regions in human temporal and frontal cortex are known to integrate visual and auditory object features. The processing of audio-visual (AV) associations in these regions has been found to be modulated by object familiarity. The aim of the present study was to explore training-induced plasticity in human cortical AV integration. We used functional magnetic resonance imaging to analyze the neural correlates of AV integration for unfamiliar artificial object sounds and images in naïve subjects (PRE training) and after a behavioral training session in which subjects acquired associations between some of these sounds and images (POST-training). In the PRE-training session, unfamiliar artificial object sounds and images were mainly integrated in right inferior frontal cortex (IFC). The POST-training results showed extended integration-related IFC activations bilaterally, and a recruitment of additional regions in bilateral superior temporal gyrus/sulcus and intraparietal sulcus. Furthermore, training-induced differential response patterns to mismatching compared with matching (i.e., associated) artificial AV stimuli were most pronounced in left IFC. These effects were accompanied by complementary training-induced congruency effects in right posterior middle temporal gyrus and fusiform gyrus. Together, these findings demonstrate that short-term cross-modal association learning was sufficient to induce plastic changes of both AV integration of object stimuli and mechanisms of AV congruency processing.

In addition to reward- and craving-related processes, habitual mechanisms play an important role in addiction. While the dorsal striatum has been proposed to code for the motivational state of habitual drug-seeking actions, the neural underpinnings of the corresponding drug-taking skills and action knowledge remain poorly understood. We used functional magnetic resonance imaging (fMRI) and a behavioral orientation affordance paradigm to investigate the neural and behavioral correlates of automatized drug-taking actions in nicotine dependence. Smokers exhibited higher fMRI activations than nonsmokers when viewing smoking-related but not when viewing control images. These group differences in fMRI activations were located not only in brain regions associated with craving and habitual learning (left ventral and dorsal striatum, dorsolateral prefrontal cortex, insula, uncus, medial frontal gyrus, right subcallosal gyrus, and bilateral parahippocampal gyrus), but also in a network of brain regions which has been strongly implicated in the encoding of action knowledge and tool use skills (bilateral premotor cortex, left superior parietal lobule, and right lateral cerebellum). A behavioral affordance reaction-time task indicated that smokers, but not nonsmokers, showed an automatized responsiveness to smoking paraphernalia similar to everyday objects. Moreover, smokers showed strong intercorrelations between fMRI activations in tool use-related brain regions, behavioral responsiveness to smoking-related cues, and severity of nicotine dependence. Apparently smoking-related action representations in smokers are stored in brain regions typically representing tool use skills and action knowledge. Most importantly, cortical and behavioral correlates of the respective drug-taking skills vary with the individual degree of nicotine dependence.

By using meaningful stimuli, multisensory research has recently started to investigate the impact of stimulus content on crossmodal integration. Variations in this respect have often been termed as "semantic". In this paper we will review work related to the question for which tasks the influence of semantic factors has been found and which cortical networks are most likely to mediate these effects. More specifically, the focus of this paper will be on processing of object stimuli presented in the auditory and visual sensory modalities. Furthermore, we will investigate which cortical regions are particularly responsive to experimental variations of content by comparing semantically matching ("congruent") and mismatching ("incongruent") experimental conditions. In this context, recent neuroimaging studies point toward a possible functional differentiation of temporal and frontal cortical regions, with the former being more responsive to semantically congruent and the latter to semantically incongruent audio-visual (AV) stimulation. To account for these differential effects, we will suggest in the final section of this paper a possible synthesis of these data on semantic modulation of AV integration with findings from neuroimaging studies and theoretical accounts of semantic memory.

Earlier studies reported evidence suggesting distinct category-related auditory representations for environmental sounds such as animal vocalizations and tool sounds in superior and middle temporal regions of the temporal lobe. However, the degree of selectivity of these representations remains to be determined. The present study combined functional magnetic resonance imaging (fMRI) adaptation with a silent acquisition protocol to further investigate category-related auditory processing of environmental sounds. To this end, we consecutively presented pairs of sounds taken from the categories 'tool sounds' or 'animal vocalizations' with either the same or different identity/category. We examined the degree of selectivity as evidenced by adaptation effects to both or only one sound category in the course of whole-brain as well as functionally and anatomically constrained region of interest analyses. While most regions predominately in the temporal cortex showed an adaptation to both sound categories, particularly the left superior temporal gyrus (STG) and the left posterior middle temporal gyrus (pMTG) selectively adapted to animal vocalizations and tool sounds, respectively. However, the activation profiles of these regions differed with respect to the general responsiveness to sounds. While tool sounds still produced fMRI signals significantly different from fixation baseline in the STG, this was not the case for animal vocalizations in pMTG. Consistent with the interpretation of STG as an intermediate auditory processing stage, this region might differentiate auditory stimuli into categories based on variations of physical stimulus properties. However, processing in left pMTG seems to be even more restricted to action-related sounds of man-made objects.

In complex natural environments, auditory and visual information often have to be processed simultaneously. Previous functional magnetic resonance imaging (fMRI) studies focused on the spatial localization of brain areas involved in audiovisual (AV) information processing, but the temporal characteristics of AV information flow in these regions remained unclear. In this study, we used fMRI and a novel information-theoretic approach to study the flow of AV sensory information. Subjects passively perceived sounds and images of objects presented either alone or simultaneously. Applying the measure of mutual information, we computed for each voxel the latency in which the blood oxygenation level-dependent signal had the highest information content about the preceding stimulus. The results indicate that, after AV stimulation, the earliest informative activity occurs in right Heschl's gyrus, left primary visual cortex, and the posterior portion of the superior temporal gyrus, which is known as a region involved in object-related AV integration. Informative activity in the anterior portion of superior temporal gyrus, middle temporal gyrus, right occipital cortex, and inferior frontal cortex was found at a later latency. Moreover, AV presentation resulted in shorter latencies in multiple cortical areas compared with isolated auditory or visual presentation. The results provide evidence for bottom-up processing from primary sensory areas into higher association areas during AV integration in humans and suggest that AV presentation shortens processing time in early sensory cortices.

The cortical integration of auditory and visual features is crucial for efficient object recognition. Previous studies have shown that audiovisual (AV) integration is affected by where and when auditory and visual features occur. However, because relatively little is known about the impact of what is integrated, we here investigated the impact of semantic congruency and object familiarity on the neural correlates of AV integration. We used functional magnetic resonance imaging to identify regions involved in the integration of both (congruent and incongruent) familiar animal sounds and images and of arbitrary combinations of unfamiliar artificial sounds and object images. Unfamiliar object images and sounds were integrated in the inferior frontal cortex (IFC), possibly reflecting learning of novel AV associations. Integration of familiar, but semantically incongruent combinations also correlated with IFC activation and additionally involved the posterior superior temporal sulcus (pSTS). For highly familiar semantically congruent AV pairings, we again found AV integration effects in pSTS and additionally in superior temporal gyrus. These findings demonstrate that the neural correlates of object-related AV integration reflect both semantic congruency and familiarity of the integrated sounds and images.

The successful integration of visual and auditory stimuli requires information about whether visual and auditory signals originate from corresponding places in the external world. Here we report crossmodal effects of spatially congruent and incongruent audio-visual (AV) stimulation. Visual and auditory stimuli were presented from one of four horizontal locations in external space. Seven healthy human subjects had to assess the spatial fit of a visual stimulus (i.e. a gray-scaled picture of a cartoon dog) and a simultaneously presented auditory stimulus (i.e. a barking sound). Functional magnetic resonance imaging (fMRI) revealed two distinct networks of cortical regions that processed preferentially either spatially congruent or spatially incongruent AV stimuli. Whereas earlier visual areas responded preferentially to incongruent AV stimulation, higher visual areas of the temporal and parietal cortex (left inferior temporal gyrus [ITG], right posterior superior temporal gyrus/sulcus [pSTG/STS], left intra-parietal sulcus [IPS]) and frontal regions (left pre-central gyrus [PreCG], left dorsolateral pre-frontal cortex [DLPFC]) responded preferentially to congruent AV stimulation. A position-resolved analysis revealed three robust cortical representations for each of the four visual stimulus locations in retinotopic visual regions corresponding to the representation of the horizontal meridian in area V1 and at the dorsal and ventral borders between areas V2 and V3. While these regions of interest (ROIs) did not show any significant effect of spatial congruency, we found subregions within ROIs in the right hemisphere that showed an incongruency effect (i.e. an increased fMRI signal during spatially incongruent compared to congruent AV stimulation). We interpret this finding as a correlate of spatially distributed recurrent feedback during mismatch processing: whenever a spatial mismatch is detected in multisensory regions (such as the IPS), processing resources are re-directed to low-level visual areas.

The perception of objects is a cognitive function of prime importance. In everyday life, object perception benefits from the coordinated interplay of vision, audition, and touch. The different sensory modalities provide both complementary and redundant information about objects, which may improve recognition speed and accuracy in many circumstances. We review crossmodal studies of object recognition in humans that mainly employed functional magnetic resonance imaging (fMRI). These studies show that visual, tactile, and auditory information about objects can activate cortical association areas that were once believed to be modality-specific. Processing converges either in multisensory zones or via direct crossmodal interaction of modality-specific cortices without relay through multisensory regions. We integrate these findings with existing theories about semantic processing and propose a general mechanism for crossmodal object recognition: The recruitment and location of multisensory convergence zones varies depending on the information content and the dominant modality.