Search

Spatial constraints on multisensory integration of auditory (A) and visual (V) stimuli were investigated in humans using behavioral and electrophysiological measures. The aim was to find out whether cross-modal interactions between A and V stimuli depend on their spatial congruity, as has been found for multisensory neurons in animal studies (Stein & Meredith, 1993). Randomized sequences of unimodal (A or V) and simultaneous bimodal (AV) stimuli were presented to right- or left-field locations while subjects made speeded responses to infrequent targets of greater intensity that occurred in either or both modalities. Behavioral responses to the bimodal stimuli were faster and more accurate than to the unimodal stimuli for both same-location and different-location AV pairings. The neural basis of this cross-modal facilitation was studied by comparing event-related potentials (ERPs) to the bimodal AV stimuli with the summed ERPs to the unimodal A and V stimuli. These comparisons revealed neural interactions localized to the ventral occipito-temporal cortex (at 190 msec) and to the superior temporal cortical areas (at 260 msec) for both same- and different-location AV pairings. In contrast, ERP interactions that differed according to spatial congruity included a phase and amplitude modulation of visual-evoked activity localized to the ventral occipito-temporal cortex at 100-400 msec and an amplitude modulation of activity localized to the superior temporal region at 260-280 msec. These results demonstrate overlapping but distinctive patterns of multisensory integration for spatially congruent and incongruent AV stimuli.

Orienting attention involuntarily to the location of a sudden sound improves perception of subsequent visual stimuli that appear nearby. The neural substrates of this cross-modal attention effect were investigated by recording event-related potentials to the visual stimuli using a dense electrode array and localizing their brain sources through inverse dipole modeling. A spatially nonpredictive auditory precue modulated visual-evoked neural activity first in the superior temporal cortex at 120-140 msec and then in the ventral occipital cortex of the fusiform gyrus 15-25 msec later. This spatio-temporal sequence of brain activity suggests that enhanced visual perception produced by the cross-modal orienting of spatial attention results from neural feedback from the multimodal superior temporal cortex to the visual cortex of the ventral processing stream.