Search

Research on attentional control has largely focused on single senses and the importance of behavioural goals in controlling attention. However, everyday situations are multisensory and contain regularities, both likely influencing attention. We investigated how visual attentional capture is simultaneously impacted by top-down goals, the multisensory nature of stimuli, and the contextual factors of stimuli's semantic relationship and temporal predictability. Participants performed a multisensory version of the Folk et al. (1992) spatial cueing paradigm, searching for a target of a predefined colour (e.g. a red bar) within an array preceded by a distractor. We manipulated: 1) stimuli's goal-relevance via distractor's colour (matching vs. mismatching the target), 2) stimuli's multisensory nature (colour distractors appearing alone vs. with tones), 3) the relationship between the distractor sound and colour (arbitrary vs. semantically congruent) and 4) the temporal predictability of distractor onset. Reaction-time spatial cueing served as a behavioural measure of attentional selection. We also recorded 129-channel event-related potentials (ERPs), analysing the distractor-elicited N2pc component both canonically and using a multivariate electrical neuroimaging framework. Behaviourally, arbitrary target-matching distractors captured attention more strongly than semantically congruent ones, with no evidence for context modulating multisensory enhancements of capture. Notably, electrical neuroimaging of surface-level EEG analyses revealed context-based influences on attention to both visual and multisensory distractors, in how strongly they activated the brain and type of activated brain networks. For both processes, the context-driven brain response modulations occurred long before the N2pc time-window, with topographic (network-based) modulations at ∼30 ms, followed by strength-based modulations at ∼100 ms post-distractor onset. Our results reveal that both stimulus meaning and predictability modulate attentional selection, and they interact while doing so. Meaning, in addition to temporal predictability, is thus a second source of contextual information facilitating goal-directed behaviour. More broadly, in everyday situations, attention is controlled by an interplay between one's goals, stimuli's perceptual salience, meaning and predictability. Our study calls for a revision of attentional control theories to account for the role of contextual and multisensory control., (Copyright © 2021. Published by Elsevier Inc.)

Illusory contours (ICs) are perceptions of visual borders despite absent contrast gradients. The psychophysical and neurobiological mechanisms of IC processes have been studied across species and diverse brain imaging/mapping techniques. Nonetheless, debate continues regarding whether IC sensitivity results from a (presumably) feedforward process within low-level visual cortices (V1/V2) or instead are processed first within higher-order brain regions, such as lateral occipital cortices (LOC). Studies in animal models, which generally favour a feedforward mechanism within V1/V2, have typically involved stimuli inducing IC lines. By contrast, studies in humans generally favour a mechanism where IC sensitivity is mediated by LOC and have typically involved stimuli inducing IC forms or shapes. Thus, the particular stimulus features used may strongly contribute to the model of IC sensitivity supported. To address this, we recorded visual evoked potentials (VEPs) while presenting human observers with an array of 10 inducers within the central 5°, two of which could be oriented to induce an IC line on a given trial. VEPs were analysed using an electrical neuroimaging framework. Sensitivity to the presence vs. absence of centrally-presented IC lines was first apparent at ∼200 ms post-stimulus onset and was evident as topographic differences across conditions. We also localized these differences to the LOC. The timing and localization of these effects are consistent with a model of IC sensitivity commencing within higher-level visual cortices. We propose that prior observations of effects within lower-tier cortices (V1/V2) are the result of feedback from IC sensitivity that originates instead within higher-tier cortices (LOC)., (Copyright © 2018 Elsevier Inc. All rights reserved.)

Everyday vision includes the detection of stimuli, figure-ground segregation, as well as object localization and recognition. Such processes must often surmount impoverished or noisy conditions; borders are perceived despite occlusion or absent contrast gradients. These illusory contours (ICs) are an example of so-called mid-level vision, with an event-related potential (ERP) correlate at ∼100-150 ms post-stimulus onset and originating within lateral-occipital cortices (the IC effect ). Presently, visual completion processes supporting IC perception are considered exclusively visual; any influence from other sensory modalities is currently unknown. It is now well-established that multisensory processes can influence both low-level vision (e.g. detection) as well as higher-level object recognition. By contrast, it is unknown if mid-level vision exhibits multisensory benefits and, if so, through what mechanisms. We hypothesized that sounds would impact the IC effect . We recorded 128-channel ERPs from 17 healthy, sighted participants who viewed ICs or no-contour (NC) counterparts either in the presence or absence of task-irrelevant sounds. The IC effect was enhanced by sounds and resulted in the recruitment of a distinct configuration of active brain areas over the 70-170 ms post-stimulus period. IC-related source-level activity within the lateral occipital cortex (LOC), inferior parietal lobe (IPL), as well as primary visual cortex (V1) were enhanced by sounds. Moreover, the activity in these regions was correlated when sounds were present, but not when absent. Results from a control experiment, which employed amodal variants of the stimuli, suggested that sounds impact the perceived brightness of the IC rather than shape formation per se. We provide the first demonstration that multisensory processes augment mid-level vision and everyday visual completion processes, and that one of the mechanisms is brightness enhancement. These results have important implications for the design of treatments and/or visual aids for low-vision patients., (Copyright © 2018 Elsevier Inc. All rights reserved.)

Distinct anatomical and functional pathways are postulated for analysing a sound's object-related ('what') and space-related ('where') information. It remains unresolved to which extent distinct or overlapping neural resources subserve specific object-related dimensions (i.e. who is speaking and what is being said can both be derived from the same acoustic input). To address this issue, we recorded high-density auditory evoked potentials (AEPs) while participants selectively attended and discriminated sounds according to their pitch, speaker identity, uttered syllable ('what' dimensions) or their location ('where'). Sound acoustics were held constant across blocks; the only manipulation involved the sound dimension that participants had to attend to. The task-relevant dimension was varied across blocks. AEPs from healthy participants were analysed within an electrical neuroimaging framework to differentiate modulations in response strength from modulations in response topography; the latter of which forcibly follow from changes in the configuration of underlying sources. There were no behavioural differences in discrimination of sounds across the 4 feature dimensions. As early as 90ms post-stimulus onset, AEP topographies differed across 'what' conditions, supporting a functional sub-segregation within the auditory 'what' pathway. This study characterises the spatio-temporal dynamics of segregated, yet parallel, processing of multiple sound object-related feature dimensions when selective attention is directed to them., (Copyright © 2018 Elsevier Inc. All rights reserved.)

How food valuation and decision-making influence the perception of food is of major interest to better understand food intake behavior and, by extension, body weight management. Our study investigated behavioral responses and spatio-temporal brain dynamics by means of visual evoked potentials (VEPs) in twenty-two normal-weight participants when viewing pairs of food photographs. Participants rated how much they liked each food item (valuation) and subsequently chose between the two alternative food images. Unsurprisingly, strongly liked foods were also chosen most often. Foods were rated faster as strongly liked than as mildly liked or disliked irrespective of whether they were subsequently chosen over an alternative. Moreover, strongly liked foods were subsequently also chosen faster than the less liked alternatives. Response times during valuation and choice were positively correlated, but only when foods were liked; the faster participants rated foods as strongly liked, the faster they were in choosing the food item over an alternative. VEP modulations by the level of liking attributed as well as the subsequent choice were found as early as 135-180ms after food image onset. Analyses of neural source activity patterns over this time interval revealed an interaction between liking and the subsequent choice within the insula, dorsal frontal and superior parietal regions. The neural responses to food viewing were found to be modulated by the attributed level of liking only when foods were chosen, not when they were dismissed for an alternative. Therein, the responses to disliked foods were generally greater than those to foods that were liked more. Moreover, the responses to disliked but chosen foods were greater than responses to disliked foods which were subsequently dismissed for an alternative offer. Our findings show that the spatio-temporal brain dynamics to food viewing are immediately influenced both by how much foods are liked and by choices taken on them. These valuation and choice processes are subserved by brain regions involved in salience and reward attribution as well as in decision-making processes, which are likely to influence prospective dietary choices in everyday life., (Copyright © 2015 Elsevier Inc. All rights reserved.)

Objects' borders are readily perceived despite absent contrast gradients, e.g. due to poor lighting or occlusion. In humans, a visual evoked potential (VEP) correlate of illusory contour (IC) sensitivity, the "IC effect", has been identified with an onset at ~90 ms and generators within bilateral lateral occipital cortices (LOC). The IC effect is observed across a wide range of stimulus parameters, though until now it always involved high-contrast achromatic stimuli. Whether IC perception and its brain mechanisms differ as a function of the type of stimulus cue remains unknown. Resolving such will provide insights on whether there is a unique or multiple solutions to how the brain binds together spatially fractionated information into a cohesive perception. Here, participants discriminated IC from no-contour (NC) control stimuli that were either comprised of low-contrast achromatic stimuli or instead isoluminant chromatic contrast stimuli (presumably biasing processing to the magnocellular and parvocellular pathways, respectively) on separate blocks of trials. Behavioural analyses revealed that ICs were readily perceived independently of the stimulus cue--i.e. when defined by either chromatic or luminance contrast. VEPs were analysed within an electrical neuroimaging framework and revealed a generally similar timing of IC effects across both stimulus contrasts (i.e. at ~90 ms). Additionally, an overall phase shift of the VEP on the order of ~30 ms was consistently observed in response to chromatic vs. luminance contrast independently of the presence/absence of ICs. Critically, topographic differences in the IC effect were observed over the ~110-160 ms period; different configurations of intracranial sources contributed to IC sensitivity as a function of stimulus contrast. Distributed source estimations localized these differences to LOC as well as V1/V2. The present data expand current models by demonstrating the existence of multiple, cue-dependent circuits in the brain for generating perceptions of illusory contours., (Copyright © 2016 Elsevier Inc. All rights reserved.)

Real-world environments are nearly always multisensory in nature. Processing in such situations confers perceptual advantages, but its automaticity remains poorly understood. Automaticity has been invoked to explain the activation of visual cortices by laterally-presented sounds. This has been observed even when the sounds were task-irrelevant and spatially uninformative about subsequent targets. An auditory-evoked contralateral occipital positivity (ACOP) at ~250ms post-sound onset has been postulated as the event-related potential (ERP) correlate of this cross-modal effect. However, the spatial dimension of the stimuli was nevertheless relevant in virtually all prior studies where the ACOP was observed. By manipulating the implicit predictability of the location of lateralised sounds in a passive auditory paradigm, we tested the automaticity of cross-modal activations of visual cortices. 128-channel ERP data from healthy participants were analysed within an electrical neuroimaging framework. The timing, topography, and localisation resembled previous characterisations of the ACOP. However, the cross-modal activations of visual cortices by sounds were critically dependent on whether the sound location was (un)predictable. Our results are the first direct evidence that this particular cross-modal process is not (fully) automatic; instead, it is context-contingent. More generally, the present findings provide novel insights into the importance of context-related factors in controlling information processing across the senses, and call for a revision of current models of automaticity in cognitive sciences., (Copyright © 2015 Elsevier Inc. All rights reserved.)

Recognition of environmental sounds is believed to proceed through discrimination steps from broad to more narrow categories. Very little is known about the neural processes that underlie fine-grained discrimination within narrow categories or about their plasticity in relation to newly acquired expertise. We investigated how the cortical representation of birdsongs is modulated by brief training to recognize individual species. During a 60-minute session, participants learned to recognize a set of birdsongs; they improved significantly their performance for trained (T) but not control species (C), which were counterbalanced across participants. Auditory evoked potentials (AEPs) were recorded during pre- and post-training sessions. Pre vs. post changes in AEPs were significantly different between T and C i) at 206-232ms post stimulus onset within a cluster on the anterior part of the left superior temporal gyrus; ii) at 246-291ms in the left middle frontal gyrus; and iii) 512-545ms in the left middle temporal gyrus as well as bilaterally in the cingulate cortex. All effects were driven by weaker activity for T than C species. Thus, expertise in discriminating T species modulated early stages of semantic processing, during and immediately after the time window that sustains the discrimination between human vs. animal vocalizations. Moreover, the training-induced plasticity is reflected by the sharpening of a left lateralized semantic network, including the anterior part of the temporal convexity and the frontal cortex. Training to identify birdsongs influenced, however, also the processing of C species, but at a much later stage. Correct discrimination of untrained sounds seems to require an additional step which results from lower-level features analysis such as apperception. We therefore suggest that the access to objects within an auditory semantic category is different and depends on subject's level of expertise. More specifically, correct intra-categorical auditory discrimination for untrained items follows the temporal hierarchy and transpires in a late stage of semantic processing. On the other hand, correct categorization of individually trained stimuli occurs earlier, during a period contemporaneous with human vs. animal vocalization discrimination, and involves a parallel semantic pathway requiring expertise., (Copyright © 2015 Elsevier Inc. All rights reserved.)

Although neuroimaging research has evidenced specific responses to visual food stimuli based on their nutritional quality (e.g., energy density, fat content), brain processes underlying portion size selection remain largely unexplored. We identified spatio-temporal brain dynamics in response to meal images varying in portion size during a task of ideal portion selection for prospective lunch intake and expected satiety. Brain responses to meal portions judged by the participants as 'too small', 'ideal' and 'too big' were measured by means of electro-encephalographic (EEG) recordings in 21 normal-weight women. During an early stage of meal viewing (105-145 ms), data showed an incremental increase of the head-surface global electric field strength (quantified via global field power; GFP) as portion judgments ranged from 'too small' to 'too big'. Estimations of neural source activity revealed that brain regions underlying this effect were located in the insula, middle frontal gyrus and middle temporal gyrus, and are similar to those reported in previous studies investigating responses to changes in food nutritional content. In contrast, during a later stage (230-270 ms), GFP was maximal for the 'ideal' relative to the 'non-ideal' portion sizes. Greater neural source activity to 'ideal' vs. 'non-ideal' portion sizes was observed in the inferior parietal lobule, superior temporal gyrus and mid-posterior cingulate gyrus. Collectively, our results provide evidence that several brain regions involved in attention and adaptive behavior track 'ideal' meal portion sizes as early as 230 ms during visual encounter. That is, responses do not show an increase paralleling the amount of food viewed (and, in extension, the amount of reward), but are shaped by regulatory mechanisms., (Copyright © 2015. Published by Elsevier Inc.)

The influence of external factors on food preferences and choices is poorly understood. Knowing which and how food-external cues impact the sensory processing and cognitive valuation of food would provide a strong benefit toward a more integrative understanding of food intake behavior and potential means of interfering with deviant eating patterns to avoid detrimental health consequences for individuals in the long run. We investigated whether written labels with positive and negative (as opposed to 'neutral') valence differentially modulate the spatio-temporal brain dynamics in response to the subsequent viewing of high- and low-energetic food images. Electrical neuroimaging analyses were applied to visual evoked potentials (VEPs) from 20 normal-weight participants. VEPs and source estimations in response to high- and low- energy foods were differentially affected by the valence of preceding word labels over the ~260-300 ms post-stimulus period. These effects were only observed when high-energy foods were preceded by labels with positive valence. Neural sources in occipital as well as posterior, frontal, insular and cingulate regions were down-regulated. These findings favor cognitive-affective influences especially on the visual responses to high-energetic food cues, potentially indicating decreases in cognitive control and goal-adaptive behavior. Inverse correlations between insular activity and effectiveness in food classification further indicate that this down-regulation directly impacts food-related behavior., (© 2013.)

For the recognition of sounds to benefit perception and action, their neural representations should also encode their current spatial position and their changes in position over time. The dual-stream model of auditory processing postulates separate (albeit interacting) processing streams for sound meaning and for sound location. Using a repetition priming paradigm in conjunction with distributed source modeling of auditory evoked potentials, we determined how individual sound objects are represented within these streams. Changes in perceived location were induced by interaural intensity differences, and sound location was either held constant or shifted across initial and repeated presentations (from one hemispace to the other in the main experiment or between locations within the right hemispace in a follow-up experiment). Location-linked representations were characterized by differences in priming effects between pairs presented to the same vs. different simulated lateralizations. These effects were significant at 20-39 ms post-stimulus onset within a cluster on the posterior part of the left superior and middle temporal gyri; and at 143-162 ms within a cluster on the left inferior and middle frontal gyri. Location-independent representations were characterized by a difference between initial and repeated presentations, independently of whether or not their simulated lateralization was held constant across repetitions. This effect was significant at 42-63 ms within three clusters on the right temporo-frontal region; and at 165-215 ms in a large cluster on the left temporo-parietal convexity. Our results reveal two varieties of representations of sound objects within the ventral/What stream: one location-independent, as initially postulated in the dual-stream model, and the other location-linked., (Copyright © 2013 Elsevier Inc. All rights reserved.)

In natural settings the same sound source is often heard repeatedly, with variations in spectro-temporal and spatial characteristics. We investigated how such repetitions influence sound representations and in particular how auditory cortices keep track of recently vs. often heard objects. A set of 40 environmental sounds was presented twice, i.e. as prime and as repeat, while subjects categorized the corresponding sound sources as living vs. non-living. Electrical neuroimaging analyses were applied to auditory evoked potentials (AEPs) comparing primes vs. repeats (effect of presentation) and the four experimental sections. Dynamic analysis of distributed source estimations revealed i) a significant main effect of presentation within the left temporal convexity at 164-215 ms post-stimulus onset; and ii) a significant main effect of section in the right temporo-parietal junction at 166-213 ms. A 3-way repeated measures ANOVA (hemisphere×presentation×section) applied to neural activity of the above clusters during the common time window confirmed the specificity of the left hemisphere for the effect of presentation, but not that of the right hemisphere for the effect of section. In conclusion, spatio-temporal dynamics of neural activity encode the temporal history of exposure to sound objects. Rapidly occurring plastic changes within the semantic representations of the left hemisphere keep track of objects heard a few seconds before, independent of the more general sound exposure history. Progressively occurring and more long-lasting plastic changes occurring predominantly within right hemispheric networks, which are known to code for perceptual, semantic and spatial aspects of sound objects, keep track of multiple exposures., (Copyright © 2012 Elsevier Inc. All rights reserved.)

Multisensory experiences influence subsequent memory performance and brain responses. Studies have thus far concentrated on semantically congruent pairings, leaving unresolved the influence of stimulus pairing and memory sub-types. Here, we paired images with unique, meaningless sounds during a continuous recognition task to determine if purely episodic, single-trial multisensory experiences can incidentally impact subsequent visual object discrimination. Psychophysics and electrical neuroimaging analyses of visual evoked potentials (VEPs) compared responses to repeated images either paired or not with a meaningless sound during initial encounters. Recognition accuracy was significantly impaired for images initially presented as multisensory pairs and could not be explained in terms of differential attention or transfer of effects from encoding to retrieval. VEP modulations occurred at 100-130 ms and 270-310 ms and stemmed from topographic differences indicative of network configuration changes within the brain. Distributed source estimations localized the earlier effect to regions of the right posterior temporal gyrus (STG) and the later effect to regions of the middle temporal gyrus (MTG). Responses in these regions were stronger for images previously encountered as multisensory pairs. Only the later effect correlated with performance such that greater MTG activity in response to repeated visual stimuli was linked with greater performance decrements. The present findings suggest that brain networks involved in this discrimination may critically depend on whether multisensory events facilitate or impair later visual memory performance. More generally, the data support models whereby effects of multisensory interactions persist to incidentally affect subsequent behavior as well as visual processing during its initial stages., (Copyright © 2012 Elsevier Inc. All rights reserved.)

Recent advances in signal analysis have engendered EEG with the status of a true brain mapping and brain imaging method capable of providing spatio-temporal information regarding brain (dys)function. Because of the increasing interest in the temporal dynamics of brain networks, and because of the straightforward compatibility of the EEG with other brain imaging techniques, EEG is increasingly used in the neuroimaging community. However, the full capability of EEG is highly underestimated. Many combined EEG-fMRI studies use the EEG only as a spike-counter or an oscilloscope. Many cognitive and clinical EEG studies use the EEG still in its traditional way and analyze grapho-elements at certain electrodes and latencies. We here show that this way of using the EEG is not only dangerous because it leads to misinterpretations, but it is also largely ignoring the spatial aspects of the signals. In fact, EEG primarily measures the electric potential field at the scalp surface in the same way as MEG measures the magnetic field. By properly sampling and correctly analyzing this electric field, EEG can provide reliable information about the neuronal activity in the brain and the temporal dynamics of this activity in the millisecond range. This review explains some of these analysis methods and illustrates their potential in clinical and experimental applications., (Copyright © 2011 Elsevier Inc. All rights reserved.)

Decision-making in an uncertain environment is driven by two major needs: exploring the environment to gather information or exploiting acquired knowledge to maximize reward. The neural processes underlying exploratory decision-making have been mainly studied by means of functional magnetic resonance imaging, overlooking any information about the time when decisions are made. Here, we carried out an electroencephalography (EEG) experiment, in order to detect the time when the brain generators responsible for these decisions have been sufficiently activated to lead to the next decision. Our analyses, based on a classification scheme, extract time-unlocked voltage topographies during reward presentation and use them to predict the type of decisions made on the subsequent trial. Classification accuracy, measured as the area under the Receiver Operator's Characteristic curve was on average 0.65 across 7 subjects. Classification accuracy was above chance levels already after 516 ms on average, across subjects. We speculate that decisions were already made before this critical period, as confirmed by a positive correlation with reaction times across subjects. On an individual subject basis, distributed source estimations were performed on the extracted topographies to statistically evaluate the neural correlates of decision-making. For trials leading to exploration, there was significantly higher activity in dorsolateral prefrontal cortex and the right supramarginal gyrus; areas responsible for modulating behavior under risk and deduction. No area was more active during exploitation. We show for the first time the temporal evolution of differential patterns of brain activation in an exploratory decision-making task on a single-trial basis., (Copyright © 2012 Elsevier Inc. All rights reserved.)

Discriminating complex sounds relies on multiple stages of differential brain activity. The specific roles of these stages and their links to perception were the focus of the present study. We presented 250 ms duration sounds of living and man-made objects while recording 160-channel electroencephalography (EEG). Subjects categorized each sound as that of a living, man-made or unknown item. We tested whether/when the brain discriminates between sound categories even when not transpiring behaviorally. We applied a single-trial classifier that identified voltage topographies and latencies at which brain responses are most discriminative. For sounds that the subjects could not categorize, we could successfully decode the semantic category based on differences in voltage topographies during the 116-174 ms post-stimulus period. Sounds that were correctly categorized as that of a living or man-made item by the same subjects exhibited two periods of differences in voltage topographies at the single-trial level. Subjects exhibited differential activity before the sound ended (starting at 112 ms) and on a separate period at ~270 ms post-stimulus onset. Because each of these periods could be used to reliably decode semantic categories, we interpreted the first as being related to an implicit tuning for sound representations and the second as being linked to perceptual decision-making processes. Collectively, our results show that the brain discriminates environmental sounds during early stages and independently of behavioral proficiency and that explicit sound categorization requires a subsequent processing stage., (Copyright © 2012 Elsevier Inc. All rights reserved.)

The scenario considered here is one where brain connectivity is represented as a network and an experimenter wishes to assess the evidence for an experimental effect at each of the typically thousands of connections comprising the network. To do this, a univariate model is independently fitted to each connection. It would be unwise to declare significance based on an uncorrected threshold of α=0.05, since the expected number of false positives for a network comprising N=90 nodes and N(N-1)/2=4005 connections would be 200. Control of Type I errors over all connections is therefore necessary. The network-based statistic (NBS) and spatial pairwise clustering (SPC) are two distinct methods that have been used to control family-wise errors when assessing the evidence for an experimental effect with mass univariate testing. The basic principle of the NBS and SPC is the same as supra-threshold voxel clustering. Unlike voxel clustering, where the definition of a voxel cluster is unambiguous, 'clusters' formed among supra-threshold connections can be defined in different ways. The NBS defines clusters using the graph theoretical concept of connected components. SPC on the other hand uses a more stringent pairwise clustering concept. The purpose of this article is to compare the pros and cons of the NBS and SPC, provide some guidelines on their practical use and demonstrate their utility using a case study involving neuroimaging data., (Copyright © 2012 Elsevier Inc. All rights reserved.)

Despite myriad studies, neurophysiologic mechanisms mediating illusory contour (IC) sensitivity remain controversial. Among the competing models one favors feed-forward effects within lower-tier cortices (V1/V2). Another situates IC sensitivity first within higher-tier cortices, principally lateral-occipital cortices (LOC), with later feedback effects in V1/V2. Still others postulate that LOC are sensitive to salient regions demarcated by the inducing stimuli, whereas V1/V2 effects specifically support IC sensitivity. We resolved these discordances by using misaligned line gratings, oriented either horizontally or vertically, to induce ICs. Line orientation provides an established assay of V1/V2 modulations independently of IC presence, and gratings lack salient regions. Electrical neuroimaging analyses of visual evoked potentials (VEPs) disambiguated the relative timing and localization of IC sensitivity with respect to that for grating orientation. Millisecond-by-millisecond analyses of VEPs and distributed source estimations revealed a main effect of grating orientation beginning at 65 ms post-stimulus onset within the calcarine sulcus that was followed by a main effect of IC presence beginning at 85 ms post-stimulus onset within the LOC. There was no evidence for differential processing of ICs as a function of the orientation of the grating. These results support models wherein IC sensitivity occurs first within the LOC., (Copyright © 2011 Elsevier Inc. All rights reserved.)

The repeated presentation of simple objects as well as biologically salient objects can cause the adaptation of behavioral and neural responses during the visual categorization of these objects. Mechanisms of response adaptation during repeated food viewing are of particular interest for better understanding food intake beyond energetic needs. Here, we measured visual evoked potentials (VEPs) and conducted neural source estimations to initial and repeated presentations of high-energy and low-energy foods as well as non-food images. The results of our study show that the behavioral and neural responses to food and food-related objects are not uniformly affected by repetition. While the repetition of images displaying low-energy foods and non-food modulated VEPs as well as their underlying neural sources and increased behavioral categorization accuracy, the responses to high-energy images remained largely invariant between initial and repeated encounters. Brain mechanisms when viewing images of high-energy foods thus appear less susceptible to repetition effects than responses to low-energy and non-food images. This finding is likely related to the superior reward value of high-energy foods and might be one reason why in particular high-energetic foods are indulged although potentially leading to detrimental health consequences., (Copyright © 2011 Elsevier Ltd. All rights reserved.)

Accurate perception of the order of occurrence of sensory information is critical for the building up of coherent representations of the external world from ongoing flows of sensory inputs. While some psychophysical evidence reports that performance on temporal perception can improve, the underlying neural mechanisms remain unresolved. Using electrical neuroimaging analyses of auditory evoked potentials (AEPs), we identified the brain dynamics and mechanism supporting improvements in auditory temporal order judgment (TOJ) during the course of the first vs. latter half of the experiment. Training-induced changes in brain activity were first evident 43-76 ms post stimulus onset and followed from topographic, rather than pure strength, AEP modulations. Improvements in auditory TOJ accuracy thus followed from changes in the configuration of the underlying brain networks during the initial stages of sensory processing. Source estimations revealed an increase in the lateralization of initially bilateral posterior sylvian region (PSR) responses at the beginning of the experiment to left-hemisphere dominance at its end. Further supporting the critical role of left and right PSR in auditory TOJ proficiency, as the experiment progressed, responses in the left and right PSR went from being correlated to un-correlated. These collective findings provide insights on the neurophysiologic mechanism and plasticity of temporal processing of sounds and are consistent with models based on spike timing dependent plasticity., (Copyright 2010 Elsevier Inc. All rights reserved.)

Action representations can interact with object recognition processes. For example, so-called mirror neurons respond both when performing an action and when seeing or hearing such actions. Investigations of auditory object processing have largely focused on categorical discrimination, which begins within the initial 100 ms post-stimulus onset and subsequently engages distinct cortical networks. Whether action representations themselves contribute to auditory object recognition and the precise kinds of actions recruiting the auditory-visual mirror neuron system remain poorly understood. We applied electrical neuroimaging analyses to auditory evoked potentials (AEPs) in response to sounds of man-made objects that were further subdivided between sounds conveying a socio-functional context and typically cuing a responsive action by the listener (e.g. a ringing telephone) and those that are not linked to such a context and do not typically elicit responsive actions (e.g. notes on a piano). This distinction was validated psychophysically by a separate cohort of listeners. Beginning approximately 300 ms, responses to such context-related sounds significantly differed from context-free sounds both in the strength and topography of the electric field. This latency is >200 ms subsequent to general categorical discrimination. Additionally, such topographic differences indicate that sounds of different action sub-types engage distinct configurations of intracranial generators. Statistical analysis of source estimations identified differential activity within premotor and inferior (pre)frontal regions (Brodmann's areas (BA) 6, BA8, and BA45/46/47) in response to sounds of actions typically cuing a responsive action. We discuss our results in terms of a spatio-temporal model of auditory object processing and the interplay between semantic and action representations.

Do our brains implicitly track the energetic content of the foods we see? Using electrical neuroimaging of visual evoked potentials (VEPs) we show that the human brain can rapidly discern food's energetic value, vis à vis its fat content, solely from its visual presentation. Responses to images of high-energy and low-energy food differed over two distinct time periods. The first period, starting at approximately 165 ms post-stimulus onset, followed from modulations in VEP topography and by extension in the configuration of the underlying brain network. Statistical comparison of source estimations identified differences distributed across a wide network including both posterior occipital regions and temporo-parietal cortices typically associated with object processing, and also inferior frontal cortices typically associated with decision-making. During a successive processing stage (starting at approximately 300 ms), responses differed both topographically and in terms of strength, with source estimations differing predominantly within prefrontal cortical regions implicated in reward assessment and decision-making. These effects occur orthogonally to the task that is actually being performed and suggest that reward properties such as a food's energetic content are treated rapidly and in parallel by a distributed network of brain regions involved in object categorization, reward assessment, and decision-making.

Switching from one functional or cognitive operation to another is thought to rely on executive/control processes. The efficacy of these processes may depend on the extent of overlap between neural circuitry mediating the different tasks; more effective task preparation (and by extension smaller switch costs) is achieved when this overlap is small. We investigated the performance costs associated with switching tasks and/or switching sensory modalities. Participants discriminated either the identity or spatial location of objects that were presented either visually or acoustically. Switch costs between tasks were significantly smaller when the sensory modality of the task switched versus when it repeated. This was the case irrespective of whether the pre-trial cue informed participants only of the upcoming task, but not sensory modality (Experiment 1) or whether the pre-trial cue was informative about both the upcoming task and sensory modality (Experiment 2). In addition, in both experiments switch costs between the senses were positively correlated when the sensory modality of the task repeated across trials and not when it switched. The collective evidence supports the independence of control processes mediating task switching and modality switching and also the hypothesis that switch costs reflect competitive interference between neural circuits.

Auditory scene analysis requires the accurate encoding and comparison of the perceived spatial positions of sound sources. The electrophysiological correlates of auditory spatial discrimination and their relationship to performance accuracy were studied in humans by applying electrical neuroimaging analyses to auditory evoked potentials (AEPs) that were recorded during the completion of a near-threshold S1-S2 paradigm within the right hemispace. Data were sorted as a function of performance accuracy, and AEP responses 75-117 ms after the presentation of the first sound differed topographically between trials leading to correct and incorrect spatial discrimination. Distributed source estimations revealed that this followed from significantly stronger activity within the left (i.e. contralateral) supratemporal plane (STP) and the left inferior parietal lobule prior to correct versus incorrect discrimination performance. Successful spatial discrimination thus depends on the activity of distinct configurations of active brain networks within the contralateral temporo-parietal cortex over a time period when the first sound position is being encoded. Furthermore, significant positive correlations were observed between performance accuracy and the intracranial activity estimated within the left STP. The efficacy of S1 processing within the STP is thus predictive of behavioral performance outcome during auditory spatial discrimination. Our data support a model wherein refinement of spatial representations occurs within the STP and that interactions with parietal structures allow for transformations into coordinate frames that are required for higher-order computations including absolute localization of sound sources.

The rapid and precise processing of environmental sounds contributes to communication functions as well as both object recognition and localization. Plasticity in (accessing) the neural representations of environmental sounds is likewise essential for an adaptive organism, in particular humans, and can be indexed by repetition priming. How the brain achieves such plasticity with representations of environmental sounds is presently unresolved. Electrical neuroimaging of 64-channel auditory evoked potentials (AEPs) in humans identified the spatio-temporal brain mechanisms of repetition priming involving sounds of environmental objects. Subjects performed an 'oddball' target detection task, based on the semantic category of stimuli (living vs. man-made objects). Repetition priming effects were observed behaviorally as a speeding of reaction times and electrophysiologically as a suppression of the strength of responses to repeated sound presentations over the 156-215 ms post-stimulus period. These effects of plasticity were furthermore localized, using statistical analyses of a distributed linear inverse solution, to the left middle temporal gyrus and superior temporal sulcus (BA22), which have been implicated in associating sounds with their abstract representations and actions. These effects are subsequent to and occur in different brain regions from what has been previously identified as the earliest discrimination of auditory object categories. Plasticity in associative-semantic, rather than perceptual-discriminative functions, may underlie repetition priming of sounds of objects. We present a multi-stage mechanism of auditory object processing akin to what has been described for visual object processing and which also provides a framework for accessing multisensory object representations.

Whether the somatosensory system, like its visual and auditory counterparts, is comprised of parallel functional pathways for processing identity and spatial attributes (so-called what and where pathways, respectively) has hitherto been studied in humans using neuropsychological and hemodynamic methods. Here, electrical neuroimaging of somatosensory evoked potentials (SEPs) identified the spatio-temporal mechanisms subserving vibrotactile processing during two types of blocks of trials. What blocks varied stimuli in their frequency (22.5 Hz vs. 110 Hz) independently of their location (left vs. right hand). Where blocks varied the same stimuli in their location independently of their frequency. In this way, there was a 2x2 within-subjects factorial design, counterbalancing the hand stimulated (left/right) and trial type (what/where). Responses to physically identical somatosensory stimuli differed within 200 ms post-stimulus onset, which is within the same timeframe we previously identified for audition (De Santis, L., Clarke, S., Murray, M.M., 2007. Automatic and intrinsic auditory "what" and "where" processing in humans revealed by electrical neuroimaging. Cereb Cortex 17, 9-17.). Initially (100-147 ms), responses to each hand were stronger to the what than where condition in a statistically indistinguishable network within the hemisphere contralateral to the stimulated hand, arguing against hemispheric specialization as the principal basis for somatosensory what and where pathways. Later (149-189 ms) responses differed topographically, indicative of the engagement of distinct configurations of brain networks. A common topography described responses to the where condition irrespective of the hand stimulated. By contrast, different topographies accounted for the what condition and also as a function of the hand stimulated. Parallel, functionally specialized pathways are observed across sensory systems and may be indicative of a computationally advantageous organization for processing spatial and identity information.

Effects of multisensory interactions on how subsequent sensory inputs are processed remain poorly understood. We investigated whether multisensory interactions between rudimentary visual and auditory stimuli (flashes and beeps) affect later visual processing. A 2 x 3 design varied the number of flashes (1 or 2) with the number of beeps (0, 1, or 2) presented on each trial, such that '2F1B' refers to the presentation of 2 flashes with 1 beep. Beeps, when present, were synchronous with the first flash, and pairs of stimuli within a trial were separated by 52 ms ISI. Subjects indicated the number of flashes presented. Electrical neuroimaging of 128-channel event-related potentials assessed both the electric field strength and topography. Isolation of responses a visual stimulus that was preceded by a multisensory event was achieved by calculating the difference between the 2F1B and 1F1B conditions, and responses to a visual stimulus preceded by a unisensory event were isolated by calculating the difference between the 2F0B and 1F0B conditions (MUL and VIS, respectively). Comparison of MUL and VIS revealed that the treatment of visual information was significantly attenuated approximately 160 ms after the onset of the second flash when it was preceded by a multisensory event. Source estimations further indicated that this attenuation occurred within low-level visual cortices. Multisensory interactions are ongoing in low-level visual cortices and affect incoming sensory processing. These data provide evidence that multisensory interactions are not restricted in time and can dramatically influence the treatment of subsequent stimuli, opening new lines of multisensory research.

Multisensory experiences enhance perceptions and facilitate memory retrieval processes, even when only unisensory information is available for accessing such memories. Using fMRI, we identified human brain regions involved in discriminating visual stimuli according to past multisensory vs. unisensory experiences. Subjects performed a completely orthogonal task, discriminating repeated from initial image presentations intermixed within a continuous recognition task. Half of initial presentations were multisensory, and all repetitions were exclusively visual. Despite only single-trial exposures to initial image presentations, accuracy in indicating image repetitions was significantly improved by past auditory-visual multisensory experiences over images only encountered visually. Similarly, regions within the lateral-occipital complex-areas typically associated with visual object recognition processes-were more active to visual stimuli with multisensory than unisensory pasts. Additional differential responses were observed in the anterior cingulate and frontal cortices. Multisensory experiences are registered by the brain even when of no immediate behavioral relevance and can be used to categorize memories. These data reveal the functional efficacy of multisensory processing.

Functional electrical neuroimaging investigated incidental emotional word processing. Previous research suggests that the brain may differentially respond to the emotional content of linguistic stimuli pre-lexically (i.e., before distinguishing that these stimuli are words). We investigated the spatiotemporal brain mechanisms of this apparent paradox and in particular whether the initial differentiation of emotional stimuli is marked by different brain generator configurations using high-density, event-related potentials. Such would support the existence of specific cerebral resources dedicated to emotional word processing. A related issue concerns the possibility of right-hemispheric specialization in the processing of emotional stimuli. Thirteen healthy men performed a go/no-go lexical decision task with bilateral word/non-word or non-word/non-word stimulus pairs. Words included equal numbers of neutral and emotional stimuli, but subjects made no explicit discrimination along this dimension. Emotional words appearing in the right visual field (ERVF) yielded the best overall performance, although the difference between emotional and neutral words was larger for left than for right visual field presentations. Electrophysiologically, ERVF presentations were distinguished from all other conditions over the 100-140 ms period by a distinct scalp topography, indicative of different intracranial generator configurations. A distributed linear source estimation (LAURA) of this distinct scalp potential field revealed bilateral lateral-occipital sources with a right hemisphere current density maximum. These data support the existence of a specialized brain network triggered by the emotional connotation of words at a very early processing stage.

This paper proposes and implements biophysical constraints to select a unique solution to the bioelectromagnetic inverse problem. It first shows that the brain's electric fields and potentials are predominantly due to ohmic currents. This serves to reformulate the inverse problem in terms of a restricted source model permitting noninvasive estimations of Local Field Potentials (LFPs) in depth from scalp-recorded data. Uniqueness in the solution is achieved by a physically derived regularization strategy that imposes a spatial structure on the solution based upon the physical laws that describe electromagnetic fields in biological media. The regularization strategy and the source model emulate the properties of brain activity's actual generators. This added information is independent of both the recorded data and head model and suffices for obtaining a unique solution compatible with and aimed at analyzing experimental data. The inverse solution's features are evaluated with event-related potentials (ERPs) from a healthy subject performing a visuo-motor task. Two aspects are addressed: the concordance between available neurophysiological evidence and inverse solution results, and the functional localization provided by fMRI data from the same subject under identical experimental conditions. The localization results are spatially and temporally concordant with experimental evidence, and the areas detected as functionally activated in both imaging modalities are similar, providing indices of localization accuracy. We conclude that biophysically driven inverse solutions offer a novel and reliable possibility for studying brain function with the temporal resolution required to advance our understanding of the brain's functional networks.

Though commonly held that multisensory experiences enrich our memories and that memories influence ongoing sensory processes, their neural mechanisms remain unresolved. Here, electrical neuroimaging shows that auditory-visual multisensory experiences alter subsequent processing of unisensory visual stimuli during the same block of trials at early stages poststimulus onset and within visual object recognition areas. We show this with a stepwise analysis of scalp-recorded event-related potentials (ERPs) that statistically tested (1) ERP morphology and amplitude, (2) global electric field power, (3) topographic stability of and changes in the electric field configuration, and (4) intracranial distributed linear source estimations. Subjects performed a continuous recognition task, discriminating repeated vs. initial image presentations. Corresponding, but task-irrelevant, sounds accompanied half of the initial presentations during a given block of trials. On repeated presentations within a block of trials, only images appeared, yielding two situations-the image's prior presentation was only visual or with a sound. Image repetitions that had been accompanied by sounds yielded improved memory performance accuracy (old or new discrimination) and were differentiated as early as approximately 60-136 ms from images that had not been accompanied by sounds through generator changes in areas of the right lateral-occipital complex (LOC). It thus appears that unisensory percepts trigger multisensory representations associated with them. The collective data support the hypothesis that perceptual or memory traces for multisensory auditory-visual events involve a distinct cortical network that is rapidly activated by subsequent repetition of just the unisensory visual component.

Recent studies have revealed a distinct cortical network activated during the analysis of sounds' spatial properties. Whether common brain regions in this auditory where pathway are involved in both auditory motion and location processing is unresolved. We investigated this question with multichannel auditory evoked potentials (AEPs) in 11 subjects. Stimuli were binaural 500-ms white noise bursts. Interaural time differences (ITD) created the sensation of moving or stationary sounds within each auditory hemifield, and subjects discriminated either their position or direction of motion in a blocked design. Scalp potential distributions (AEP maps) differentiated electric field configurations across stimulus classes. The initial approximately 250-ms poststimulus yielded common topographies for both stimulus classes and hemifields. After approximately 250-ms, moving and stationary sounds engaged distinct cortical networks at two time periods, again with no differences observed between hemifields. The first ( approximately 250- to 350-ms poststimulus onset) was during stimulus presentation, and the second ( approximately 550- to 900-ms poststimulus onset) occurred after stimulus offset. Distributed linear inverse solutions of the maps over the 250- to 350-ms time period revealed not only bilateral inferior frontal activation for both types of auditory spatial processing, but also strong right inferior parietal activation in the case of auditory motion discrimination. During the later 550-to 900-ms time period, right inferior parietal and bilateral inferior frontal activity was again observed for moving sounds, whereas strong bilateral superior frontal activity was seen in the case of stationary sounds. Collectively, the evidence supports the existence of partly segregated networks within the auditory where pathway for auditory location and auditory motion processing., (2002 Elsevier Science (USA).)