Search

The superior temporal sulcus has been suggested to play a significant role in the integration of auditory and visual sensory information. Here, we presented vowels and short video clips of the corresponding articulatory gestures to healthy adult humans with two auditory-visual stimulus intervals during sparse sampling 3-T functional magnetic resonance imaging to detect which brain areas are sensitive to synchrony of speech sounds and associated articulatory gestures. The upper bank of the left middle superior temporal sulcus showed stronger activation during naturally asynchronous stimulation than during simultaneous stimulus presentation. It is possible that this reflects sensitivity of the left middle superior temporal sulcus to temporal synchrony of audio-visual speech stimuli.

To test for the feature specificity of adaptation of auditory-cortex magnetoencephalographic N1m responses to phonemes during lipreading, we presented eight healthy volunteers with a simplified sine-wave first-formant (F1) transition shared by /ba/, /ga/, and /da/, and a continuum of second-formant (F2) transitions contained in /ba/ (ascending), /da/ (level), and /ga/ (descending), during lipreading of /ba/ vs. /ga/ vs. a still-face baseline. N1m responses to the F1 transition were suppressed during lipreading, further, visual /ga/ (vs. /ba/) significantly suppressed left-hemisphere N1m responses to the F2 transition contained in /ga/. This suggests that visual speech activates and adapts auditory cortex neural populations tuned to formant transitions, the basic sound-sweep constituents of phonemes, potentially explaining enhanced speech perception during lipreading.

Recent studies have yielded contradictory evidence on whether visual speech perception (watching articulatory gestures) can activate the human primary auditory cortex. To circumvent confounds due to inter-individual anatomical variation, we defined our subjects' Heschl's gyri and assessed blood oxygenation-dependent signal changes at 3 T within this confined region during visual speech perception and observation of moving circles. Visual speech perception activated Heschl's gyri in nine subjects, with activation in seven of them extending to the area of primary auditory cortex. Activation was significantly stronger during visual speech perception than during observation of the moving circles. Further, a significant hemisphere by stimulus interaction occurred, suggesting left Heschl's gyrus specialization for visual speech processing.

Event related potentials (ERPs) were recorded from 11 subjects watching photographs of angry and happy faces with different gaze directions. ERPs to right averted gaze differed from those to straight and left averted gaze at 85 and 460 ms whereas ERPs to happy and angry expressions differed at 115, 330 and 380 ms. We suggest that short-latency effects, maximal over occipital cortex, reflect the involvement of visual cortex in the early analysis of socially-relevant stimuli. Interaction of gaze and expressions was reflected in ERPs at 270 - 450 ms. We conclude that gaze and emotional expressions are analyzed in parallel at the early stages of visual processing. The interaction of these two processing streams starts no earlier than at 270 ms.

We investigated the brain mechanisms enabling one automatically discriminate phoneme category irrespective of the large inter-speaker variability in the acoustic features of the voices. For this purpose, subjects were presented with 450 different speech stimuli, each uttered by a different speaker, belonging to three vowel categories, while a 306-channel magnetoencephalogram (MEG) was obtained to record the magnetic counterpart of the mismatch negativity (MMNm), elicited only when sensory memory traces for repetitive sounds are formed in the auditory cortex. Despite this wide acoustic variation, category changes elicited prominent MMNm responses, which were considerably stronger in the left than in the right hemisphere in the right-handed subjects. These results implicate the presence of long-term memory traces for vowels, which can recognize the vowel-specific invariant code enabling correct vowel percept even in the presence of realistic acoustic variation., (Copyright 2002 Lippincott Williams & Wilkins)

Transient frequency and amplitude modulations (FMs, AMs) of sound are requisite to speech recognition. We recorded whole-head magnetoencephalographic signals from seven subjects to binaural 620 ms 667 Hz tones, with 3, 30, or 300 ms FMs or AMs in the beginning or middle of the tone. Responses were significantly larger and earlier for FMs than AMs, for rapid than slow modulations, and for modulations at the beginning (BEG) than in the middle (MID) of the sound. BEG 3 ms FMs elicited strongest signals in the left and MID 3 ms FMs in the right hemisphere. Fast MID modulations produced significantly stronger responses in the right than left hemisphere. These differences may reflect different functions of the left and right hemisphere in speech perception.

Neuromagnetic signals evoked by synthesized syllables (/bae/ and /gae/) were recorded over the left auditory cortex of healthy humans. The fundamental frequencies of the syllables varied as if the same speaker had pronounced them at 16 different pitches. Specific mismatch responses to infrequent syllables among frequent syllables of the other type indicated that phonetically invariant information had been extracted at the level of the auditory cortex from the extensive irrelevant pitch variation. Such a detection mechanism is necessary for perceiving speech sounds in natural situations with a great deal of acoustic variation present.