Search

The McGurk illusion occurs when listeners hear an illusory percept (i.e., “da”), resulting from mismatched pairings of audiovisual (AV) speech stimuli (i.e., auditory/ba/paired with visual/ga/). Hearing a third percept—distinct from both the auditory and visual input—has been used as evidence of AV fusion. We examined whether the McGurk illusion is instead driven by visual dominance, whereby the third percept, e.g., “da,” represents a default percept for visemes with an ambiguous place of articulation (POA), like/ga/. Participants watched videos of a talker uttering various consonant vowels (CVs) with (AV) and without (V-only) audios of/ba/. Individuals transcribed the CV they saw (V-only) or heard (AV). In the V-only condition, individuals predominantly saw “da”/“ta” when viewing CVs with indiscernible POAs. Likewise, in the AV condition, upon perceiving an illusion, they predominantly heard “da”/“ta” for CVs with indiscernible POAs. The illusion was stronger in individuals who exhibited weak/ba/auditory encoding (examined using a control auditory-only task). In Experiment2, we attempted to replicate these findings using stimuli recorded from a different talker. The V-only results were not replicated, but again individuals predominately heard “da”/“ta”/“tha” as an illusory percept for various AV combinations, and the illusion was stronger in individuals who exhibited weak/ba/auditory encoding. These results demonstrate that when visual CVs with indiscernible POAs are paired with a weakly encoded auditory/ba/, listeners default to hearing “da”/“ta”/“tha”—thus, tempering the AV fusion account, and favoring a default mechanism triggered when both AV stimuli are ambiguous.

Lip-reading improves intelligibility in noisy acoustical environments. We hypothesized that watching mouth movements benefits speech comprehension in a ‘cocktail party’ by strengthening the encoding of the neural representations of the visually paired speech stream. In an audiovisual (AV) task, EEG was recorded as participants watched and listened to videos of a speaker uttering a sentence while also hearing a concurrent sentence by a speaker of the opposite gender. A key manipulation was that each audio sentence had a 200-ms segment replaced by white noise. To assess comprehension, subjects were tasked with transcribing the AV-attended sentence on randomly selected trials. In the auditory-only trials, subjects listened to the same sentences and completed the same task while watching a static picture of a speaker of either gender. Subjects directed their listening to the voice of the gender of the speaker in the video. We found that the N1 auditory-evoked potential (AEP) time-locked to white noise onsets was significantly more inhibited for the AV-attended sentences than for those of the auditorily-attended (A-attended) and AV-unattended sentences. N1 inhibition to noise onsets has been shown to index restoration of phonemic representations of degraded speech. These results underscore that attention and congruency in the AV setting help streamline the complex auditory scene, partly by reinforcing the neural representations of the visually attended stream, heightening the perception of continuity and comprehension.

We tested the predictions of the Dynamic Reweighting Model (DRM) of audiovisual (AV) speech integration, which posits that spectrotemporally reliable (informative) AV speech stimuli induce a reweighting of processing from low-level to high-level auditory networks. This reweighting decreases sensitivity to acoustic onsets and in turn increases tolerance to AV onset asynchronies (AVOA). EEG was recorded while subjects watched videos of a speaker uttering tri-syllabic nonwords that varied in spectrotemporal reliability and asynchrony of the visual and auditory input. Subjects judged the stimuli as in-sync or out-of-sync. Results showed that subjects exhibited greater AVOA tolerance for non-blurred than blurred visual speech and for less than more degraded acoustic speech. Increased AVOA tolerance was reflected in reduced amplitude of the P1–P2 auditory evoked potentials, a neurophysiological indication of reduced sensitivity to acoustic onsets and successful AV integration. There was also sustained visual alpha band (8–14 Hz) suppression (desynchronization) following acoustic speech onsets for non-blurred versus blurred visual speech, consistent with continuous engagement of the visual system as the speech unfolds. The current findings suggest that increased spectrotemporal reliability of acoustic and visual speech promotes robust AV integration, partly by suppressing sensitivity to acoustic onsets, in support of the DRM’s reweighting mechanism. Increased visual signal reliability also sustains the engagement of the visual system with the auditory system to maintain alignment of information across modalities.

Audiovisual (AV) integration is essential for speech comprehension, especially in adverse listening situations. Divergent, but not mutually exclusive, theories have been proposed to explain the neural mechanisms underlying AV integration. One theory advocates that this process occurs via interactions between the auditory and visual cortices, as opposed to fusion of AV percepts in a multisensory integrator. Building upon this idea, we proposed that AV integration in spoken language reflects visually induced weighting of phonetic representations at the auditory cortex. EEG was recorded while male and female human subjects watched and listened to videos of a speaker uttering consonant vowel (CV) syllables /ba/ and /fa/, presented in Auditory-only, AV congruent or incongruent contexts. Subjects reported whether they heard /ba/ or /fa/. We hypothesized that vision alters phonetic encoding by dynamically weighting which phonetic representation in the auditory cortex is strengthened or weakened. That is, when subjects are presented with visual /fa/ and acoustic /ba/ and hear /fa/ (illusion-fa), the visual input strengthens the weighting of the phone /f/ representation. When subjects are presented with visual /ba/ and acoustic /fa/ and hear /ba/ (illusion-ba), the visual input weakens the weighting of the phone /f/ representation. Indeed, we found an enlarged N1 auditory evoked potential when subjects perceivedillusion-ba, and a reduced N1 when they perceivedillusion-fa, mirroring the N1 behavior for /ba/ and /fa/ in Auditory-only settings. These effects were especially pronounced in individuals with more robust illusory perception. These findings provide evidence that visual speech modifies phonetic encoding at the auditory cortex.SIGNIFICANCE STATEMENTThe current study presents evidence that audiovisual integration in spoken language occurs when one modality (vision) acts on representations of a second modality (audition). Using the McGurk illusion, we show that visual context primes phonetic representations at the auditory cortex, altering the auditory percept, evidenced by changes in the N1 auditory evoked potential. This finding reinforces the theory that audiovisual integration occurs via visual networks influencing phonetic representations in the auditory cortex. We believe that this will lead to the generation of new hypotheses regarding cross-modal mapping, particularly whether it occurs via direct or indirect routes (e.g., via a multisensory mediator).

The McGurk illusion is experienced to various degrees among the general population. Previous studies have implicated the left superior temporal sulcus (STS) and auditory cortex (AC) as regions associated with this interindividual variability. We sought to further investigate the neurophysiology underlying this variability using a variant of the McGurk illusion design. Electroencephalography (EEG) was recorded while human subjects were presented with videos of a speaker uttering the consonant-vowels (CVs) /ba/ and /fa/, which were mixed and matched with audio of /ba/ and /fa/ to produce congruent and incongruent conditions. Subjects were also presented with unimodal stimuli of silent videos and audios of the CVs. They responded to whether they heard (or saw in the silent condition) /ba/ or /fa/. An illusion during the incongruent conditions was deemed successful when individuals heard the syllable conveyed by mouth movements. We hypothesized that individuals who experience the illusion more strongly should exhibit more robust desynchronization of alpha (7–12 Hz) at fronto-central and temporal sites, emphasizing more engagement of neural generators at the AC and STS. We found, however, that compared to weaker illusion perceivers, stronger illusion perceivers exhibited greater alpha synchronization at fronto-central and posterior temporal sites, which is consistent with inhibition of auditory representations. These findings suggest that stronger McGurk illusion perceivers possess more robust cross-modal sensory gating mechanisms whereby phonetic representations not conveyed by the visual system are inhibited, and in turn reinforcing perception of the visually targeted phonemes.

In spoken language, audiovisual (AV) perception occurs when the visual modality influences encoding of acoustic features (e.g., phonetic representations) at the auditory cortex. We examined how visual speech (mouth movements) transforms phonetic representations, indexed by changes to the N1 auditory evoked potential (AEP). EEG was acquired while human subjects watched and listened to videos of a speaker uttering consonant vowel (CV) syllables, /ba/ and /wa/, presented in auditory-only or AV congruent or incongruent contexts or in a context in which the consonants were replaced by white noise (noise replaced). Subjects reported whether they heard “ba” or “wa.” We hypothesized that the auditory N1 amplitude during illusory perception (caused by incongruent AV input, as in the McGurk illusion, or white noise-replaced consonants in CV utterances) should shift to reflect the auditory N1 characteristics of the phonemes conveyed visually (by mouth movements) as opposed to acoustically. Indeed, the N1 AEP became larger and occurred earlier when listeners experienced illusory “ba” (video /ba/, audio /wa/, heard as “ba”) and vice versa when they experienced illusory “wa” (video /wa/, audio /ba/, heard as “wa”), mirroring the N1 AEP characteristics for /ba/ and /wa/ observed in natural acoustic situations (e.g., auditory-only setting). This visually mediated N1 behavior was also observed for noise-replaced CVs. Taken together, the findings suggest that information relayed by the visual modality modifies phonetic representations at the auditory cortex and that similar neural mechanisms support the McGurk illusion and visually mediated phonemic restoration. NEW & NOTEWORTHY Using a variant of the McGurk illusion experimental design (using the syllables /ba/ and /wa/), we demonstrate that lipreading influences phonetic encoding at the auditory cortex. We show that the N1 auditory evoked potential morphology shifts to resemble the N1 morphology of the syllable conveyed visually. We also show similar N1 shifts when the consonants are replaced by white noise, suggesting that the McGurk illusion and the visually mediated phonemic restoration rely on common mechanisms.

We examined the relationship between tolerance for audiovisual onset asynchrony (AVOA) and the spectrotemporal fidelity of the spoken words and the speaker's mouth movements. In two experiments that only varied in the temporal order of sensory modality, visual speech leading (exp1) or lagging (exp2) acoustic speech, participants watched intact and blurred videos of a speaker uttering trisyllabic words and nonwords that were noise vocoded with 4-, 8-, 16-, and 32-channels. They judged whether the speaker's mouth movements and the speech sounds were in-sync or out-of-sync. Individuals perceived synchrony (tolerated AVOA) on more trials when the acoustic speech was more speech-like (8 channels and higher vs. 4 channels), and when visual speech was intact than blurred (exp1 only). These findings suggest that enhanced spectrotemporal fidelity of the audiovisual (AV) signal prompts the brain to widen the window of integration promoting the fusion of temporally distant AV percepts.

Purpose Several acoustic cues specify any single phonemic contrast. Nonetheless, adult, native speakers of a language share weighting strategies, showing preferential attention to some properties over others. Cochlear implant (CI) signal processing disrupts the salience of some cues: In general, amplitude structure remains readily available, but spectral structure less so. This study asked how well speech recognition is supported if CI users shift attention to salient cues not weighted strongly by native speakers. Method Twenty adults with CIs participated. The /bɑ/-/wɑ/ contrast was used because spectral and amplitude structure varies in correlated fashion for this contrast. Adults with normal hearing weight the spectral cue strongly but the amplitude cue negligibly. Three measurements were made: labeling decisions, spectral and amplitude discrimination, and word recognition. Results Outcomes varied across listeners: Some weighted the spectral cue strongly, some weighted the amplitude cue, and some weighted neither. Spectral discrimination predicted spectral weighting. Spectral weighting explained the most variance in word recognition. Age of onset of hearing loss predicted spectral weighting but not unique variance in word recognition. Conclusion The weighting strategies of listeners with normal hearing likely support speech recognition best, so efforts in implant design, fitting, and training should focus on developing those strategies.

Listeners assign different weights to spectral dynamics, such as formant rise time (FRT), and temporal dynamics, such as amplitude rise time (ART), during phonetic judgments. We examined the neurophysiological basis of FRT and ART weighting in the /ba/–/wa/ contrast. Electroencephalography was recorded for thirteen adult English speakers during a mismatch negativity (MMN) design using synthetic stimuli: a /ba/ with /ba/-like FRT and ART; a /wa/ with /wa/-like FRT and ART; and a /ba/ wa with /ba/-like FRT and /wa/-like ART. We hypothesized that because of stronger reliance on FRT, subjects would encode a stronger memory trace and exhibit larger MMN during the FRT than the ART contrast. Results supported this hypothesis. The effect was most robust in the later portion of MMN. Findings suggest that MMN is generated by multiple sources, differentially reflecting acoustic change detection (earlier MMN, bottom-up process) and perceptual weighting of ART and FRT (later MMN, top-down process).

We investigated the development of weighting strategies for acoustic cues by examining the morphology of the N1–P2 auditory evoked potential (AEP) to changes in amplitude rise time (ART) and rate of formant transition (RFT) of consonant–vowel (CV) pairs in 4–6-year olds and adults. In the AEP session, individuals listened passively to the CVs /ba/, /wa/, and a /ba/ with a superimposed slower-rising /wa/ envelope (/ba/ wa ). In the behavioral session, individuals listened to the same stimuli and judged whether they heard a /ba/ or /wa/. We hypothesized that a developmental shift in weighting strategies should be reflected in a change in the morphology of the N1–P2 AEP. In 6-year olds and adults, the N1–P2 amplitude at the vertex reflected a change in RFT but not in ART. In contrast, in the 4–5-year olds, the vertex N1–P2 did not show specificity to changes in ART or RFT. In all groups, the N1–P2 amplitude at channel C4 (right hemisphere) reflected a change in ART but not in RFT. Behaviorally, 6-year olds and adults predominately utilized RFT cues (classified /ba/ wa as /ba/) during phonetic judgments, as opposed to 4–5-year olds which utilized both cues equally. Our findings suggest that both ART and RFT are encoded in the auditory cortex, but an N1–P2 shift toward the vertex following age 4–5 indicates a shift toward an adult-like weighting strategy, such that, to utilize RFT to a greater extent.

Objectives: We describe a novel use of cortical auditory evoked potentials in the preoperative workup to determine ear candidacy for cochlear implantation. Methods: A 71-year-old male was evaluated who had a long-deafened right ear, had never worn a hearing aid in that ear, and relied heavily on use of a left-sided hearing aid. Electroencephalographic testing was performed using free field auditory stimulation of each ear independently with pure tones at 1000 and 2000 Hz at approximately 10 dB above pure-tone thresholds for each frequency and for each ear. Results: Mature cortical potentials were identified through auditory stimulation of the long-deafened ear. The patient underwent successful implantation of that ear. He experienced progressively improving aided pure-tone thresholds and binaural speech recognition benefit (AzBio score of 74%). Conclusions: Findings suggest that use of cortical auditory evoked potentials may serve a preoperative role in ear selection prior to cochlear implantation.

This study examined the neurophysiological mechanisms of speech segmentation, the process of parsing the continuous speech signal into isolated words. Individuals listened to sequences of two monosyllabic words (e.g. gas source) and non-words (e.g. nas sorf). When these phrases are spoken, talkers usually produce one continuous s-sound, not two distinct s-sounds, making it unclear where one word ends and the next one begins. This ambiguity in the signal can also result in perceptual ambiguity, causing the sequence to be heard as one word (failed to segment) or two words (segmented). We compared listeners' electroencephalogram activity when they reported hearing one word or two words, and found that bursts of fronto-central alpha activity (9-14 Hz), following the onset of the physical /s/ and end of phrase, indexed speech segmentation. Left-lateralized beta activity (14-18 Hz) following the end of phrase distinguished word from non-word segmentation. A hallmark of enhanced alpha activity is that it reflects inhibition of task-irrelevant neural populations. Thus, the current results suggest that disengagement of neural processes that become irrelevant as the words unfold marks word boundaries in continuous speech, leading to segmentation. Beta activity is likely associated with unifying word representations into coherent phrases.

In reverberant environments, the brain can suppress echoes so that auditory perception is dominated by the primary or leading sounds. Echo suppression comprises at least two distinct phenomena whose neural bases are unknown: spatial translocation of an echo toward the primary sound, and object capture to combine echo and primary sounds into a single event. In an electroencephalography study, we presented subjects with primary-echo (leading–lagging) click pairs in virtual acoustic space, with interclick delay at the individual's 50% suppression threshold. On each trial, subjects reported both click location (one or both hemifields) and the number of clicks they heard (one or two). Thus, the threshold stimulus led to two common percepts:SuppressedandNot Suppressed. On some trials, a subset of subjects reported an intermediate percept, in which two clicks were perceived in the same hemifield as the leading click, providing a dissociation between spatial translocation and object capture. We conducted time–frequency and event-related potential analyses to examine the time course of the neural mechanisms mediating echo suppression. Enhanced gamma band phase synchronization (peaking at ∼40 Hz) specific to successful echo suppression was evident from 20 to 60 ms after stimulus onset. N1 latency provided a categorical neural marker of spatial translocation, whereas N1 amplitude still reflected the physical presence of a second (lagging) click. These results provide evidence that (1) echo suppression begins early, at the latest when the acoustic signal first reaches cortex, and (2) the brain spatially translocates a perceived echo before the primary sound captures it.

The auditory cortex undergoes functional and anatomical development that reflects specialization for learned sounds. In humans, auditory maturation is evident in transient auditory-evoked potentials (AEPs) elicited by speech or music. However, neural oscillations at specific frequencies are also known to play an important role in perceptual processing. We hypothesized that, if oscillatory activity in different frequency bands reflects different aspects of sound processing, the development of phase-locking to stimulus attributes at these frequencies may have different trajectories. We examined the development of phase-locking of oscillatory responses to music sounds and to pure tones matched to the fundamental frequency of the music sounds. Phase-locking for theta (4–8 Hz), alpha (8–14 Hz), lower-to-mid beta (14–25 Hz), and upper-beta and gamma (25–70 Hz) bands strengthened with age. Phase-locking in the upper-beta and gamma range matured later than in lower frequencies and was stronger for music sounds than for pure tones, likely reflecting the maturation of neural networks that code spectral complexity. Phase-locking for theta, alpha, and lower-to-mid beta was sensitive to temporal onset (rise time) sound characteristics. The data were also consistent with phase-locked oscillatory effects of acoustic (spectrotemporal) complexity and timbre familiarity. Future studies are called for to evaluate developmental trajectories for oscillatory activity, using stimuli selected to address hypotheses related to familiarity and spectral and temporal encoding suggested by the current findings.

Changes in oscillatory brain activity have been related to perceptual and cognitive processes such as selective attention and memory matching. Here we examined brain oscillations, measured with electroencephalography (EEG), during a semantic speech processing task that required both lexically mediated memory matching and selective attention. Participants listened to nouns spoken in male and female voices, and detected an animate target (p = 20%) in a train of inanimate standards or vice versa. For a control task, subjects listened to the same words and detected a target male voice in standards of a female voice or vice versa. The standard trials of the semantic task showed enhanced upper beta (25–30 Hz) and gamma band (GBA, 30-60 Hz) activity compared to the voice task. Upper beta and GBA enhancement was accompanied by a suppression of alpha (8–12 Hz) and lower to mid beta (13–20 Hz) activity mainly localized to posterior electrodes. Enhancement of phase-locked theta activity peaking near 275 ms also occurred over the midline electrodes. Theta, upper beta, and gamma band enhancement may reflect lexically mediated template matching in auditory memory, whereas the alpha and beta suppression likely indicate increased attentional processes and memory demands.

A number of studies suggest that musical training has benefits for other cognitive domains, such as language and mathematics, and studies of children and adults indicate structural as well as functional differences between the brains of musicians and nonmusicians. The induced gamma-band response has been associated with attentional, expectation, memory retrieval, and integration of top-down, bottom-up, and multisensory processes. Here we report data indicating that the induced gamma-band response to musical sounds is larger in adult musicians than in nonmusicians and that it develops in children after 1 year of musical training beginning at age 4.5 years, but not in children of this age who are not engaged in musical lessons. We conclude that musical training affects oscillatory networks in the brain associated with executive functions, and that superior executive functioning could enhance learning and performance in many cognitive domains.

Phonemic restoration occurs when speech is perceived to be continuous through noisy interruptions, even when the speech signal is artificially removed from the interrupted epochs. This temporal filling-in illusion helps maintain robust comprehension in adverse environments and illustrates how contextual knowledge through the auditory modality (e.g., lexical) can improve perception. This study investigated how one important form of context, visual speech, affects phonemic restoration. The hypothesis was that audio-visual integration of speech should improve phonemic restoration, allowing the perceived continuity to span longer temporal gaps. Subjects listened to tri-syllabic words with a portion of each word replaced by white noise while watching lip-movement that was either congruent, temporally reversed (incongruent), or static. For each word, subjects judged whether the utterance sounded continuous or interrupted, where a “continuous” response indicated an illusory percept. Results showed that illusory filling-in of longer white noise durations (longer missing segments) occurred when the mouth movement was congruent with the spoken word compared to the other conditions, with no differences occurring between the static and incongruent conditions. Thus, phonemic restoration is enhanced when applying contextual knowledge through multisensory integration.

Objectives: Formant rise time (FRT) and amplitude rise time (ART) are acoustic cues that inform phonetic identity. FRT represents the rate of transition of the formant(s) to a steady state, while ART represents the rate at which the sound reaches its peak amplitude. Normal-hearing (NH) native English speakers weight FRT more than ART during the perceptual labeling of the /ba/–/wa/ contrast. This weighting strategy is reflected neurophysiologically in the magnitude of the mismatch negativity (MMN)—MMN is larger during the FRT than the ART distinction. The present study examined the neurophysiological basis of acoustic cue weighting in adult cochlear implant (CI) listeners using the MMN design. It was hypothesized that individuals with CIs who weight ART more in behavioral labeling (ART users) would show larger MMNs during the ART than the FRT contrast, and the opposite would be seen for FRT users. Design: Electroencephalography was recorded while 20 adults with CIs listened passively to combinations of 3 synthetic speech stimuli: a /ba/ with /ba/-like FRT and ART; a /wa/ with /wa/-like FRT and ART; and a /ba/wa stimulus with /ba/-like FRT and /wa/-like ART. The MMN response was elicited during the FRT contrast by having participants passively listen to a train of /wa/ stimuli interrupted occasionally by /ba/wa stimuli, and vice versa. For the ART contrast, the same procedure was implemented using the /ba/ and /ba/wa stimuli. Results: Both ART and FRT users with CIs elicited MMNs that were equal in magnitudes during FRT and ART contrasts, with the exception that FRT users exhibited MMNs for ART and FRT contrasts that were temporally segregated. That is, their MMNs occurred significantly earlier during the ART contrast (~100 msec following sound onset) than during the FRT contrast (~200 msec). In contrast, the MMNs for ART users of both contrasts occurred later and were not significantly separable in time (~230 msec). Interestingly, this temporal segregation observed in FRT users is consistent with the MMN behavior in NH listeners. Conclusions: Results suggest that listeners with CIs who learn to classify phonemes based on formant dynamics, consistent with NH listeners, develop a strategy similar to NH listeners, in which the organization of the amplitude and spectral representations of phonemes in auditory memory are temporally segregated.

Acoustic complexity of a stimulus has been shown to modulate the electromagnetic N1 (latency approximately 110 ms) and P2 (latency 190 ms) auditory evoked responses. We compared the relative sensitivity of electroencephalography (EEG) and magnetoencephalography (MEG) to these neural correlates of sensation. Simultaneous EEG and MEG were recorded while participants listened to three variants of a piano tone. The piano stimuli differed in their number of harmonics: the fundamental frequency (f ( 0 )), only, or f ( 0 ) and the first two or eight harmonics. The root mean square (RMS) of the amplitude of P2 but not N1 increased with spectral complexity of the piano tones in EEG and MEG. The RMS increase for P2 was more prominent in EEG than MEG, suggesting important radial sources contributing to the P2 only in EEG. Source analysis revealing contributions from radial and tangential sources was conducted to test this hypothesis. Source waveforms revealed a significant increase in the P2 radial source amplitude in EEG with increased spectral complexity of piano tones. The P2 of the tangential source waveforms also increased in amplitude with increased spectral complexity in EEG and MEG. The P2 auditory evoked response is thus represented by both tangential (gyri) and radial (sulci) activities. The radial contribution is expressed preferentially in EEG, highlighting the importance of combining EEG with MEG where complex source configurations are suspected.

Auditory evoked potentials (AEPs) express the development of mature synaptic connections in the upper neocortical laminae known to occur between 4 and 15 years of age. AEPs evoked by piano, violin, and pure tones were measured twice in a group of 4- to 5-year-old children enrolled in Suzuki music lessons and in non-musician controls. P1 was larger in the Suzuki pupils for all tones whereas P2 was enhanced specifically for the instrument of practice (piano or violin). AEPs observed for the instrument of practice were comparable to those of non-musician children about 3 years older in chronological age. The findings set into relief a general process by which the neocortical synaptic matrix is shaped by an accumulation of specific auditory experiences.

Several studies of the effects of musical experience on sound representations in the auditory cortex are reviewed. Auditory evoked potentials are compared in response to pure tones, violin tones, and piano tones in adult musicians versus nonmusicians as well as in 4- to 5-year-old children who have either had or not had extensive musical experience. In addition, the effects of auditory frequency discrimination training in adult nonmusicians on auditory evoked potentials are examined. It was found that the P2-evoked response is larger in both adult and child musicians than in nonmusicians and that auditory training enhances this component in nonmusician adults. The results suggest that the P2 is particularly neuroplastic and that the effects of musical experience can be seen early in development. They also suggest that although the effects of musical training on cortical representations may be greater if training begins in childhood, the adult brain is also open to change. These results are discussed with respect to potential benefits of early musical training as well as potential benefits of musical experience in aging.

P2 and N1c components of the auditory evoked potential (AEP) have been shown to be sensitive to remodeling of the auditory cortex by training at pitch discrimination in nonmusician subjects. Here, we investigated whether these neuroplastic components of the AEP are enhanced in musicians in accordance with their musical training histories. Highly skilled violinists and pianists and nonmusician controls listened under conditions of passive attention to violin tones, piano tones, and pure tones matched in fundamental frequency to the musical tones. Compared with nonmusician controls, both musician groups evidenced larger N1c (latency, 138 msec) and P2 (latency, 185 msec) responses to the three types of tonal stimuli. As in training studies with nonmusicians, N1c enhancement was expressed preferentially in the right hemisphere, where auditory neurons may be specialized for processing of spectral pitch. Equivalent current dipoles fitted to the N1c and P2 field patterns localized to spatially differentiable regions of the secondary auditory cortex, in agreement with previous findings. These results suggest that the tuning properties of neurons are modified in distributed regions of the auditory cortex in accordance with the acoustic training history (musical- or laboratory-based) of the subject. Enhanced P2 and N1c responses in musicians need not be considered genetic or prenatal markers for musical skill.

Audiovisual (AV) speech perception is robust to temporal asynchronies between visual and auditory stimuli. We investigated the neural mechanisms that facilitate tolerance for audiovisual stimulus onset asynchrony (AVOA) with EEG. Individuals were presented with AV words that were asynchronous in onsets of voice and mouth movement and judged whether they were synchronous or not. Behaviorally, individuals tolerated (perceived as synchronous) longer AVOAs when mouth movement preceded the speech (V-A) stimuli than when the speech preceded mouth movement (A-V). Neurophysiologically, the P1-N1-P2 auditory evoked potentials (AEPs), time-locked to sound onsets and known to arise in and surrounding the primary auditory cortex (PAC), were smaller for the in-sync than the out-of-sync percepts. Spectral power of oscillatory activity in the beta band (14–30 Hz) following the AEPs was larger during the in-sync than out-of-sync perception for both A-V and V-A conditions. However, alpha power (8–14 Hz), also following AEPs, was larger for the in-sync than out-of-sync percepts only in the V-A condition. These results demonstrate that AVOA tolerance is enhanced by inhibiting low-level auditory activity (e.g., AEPs representing generators in and surrounding PAC) that code for acoustic onsets. By reducing sensitivity to acoustic onsets, visual-to-auditory onset mapping is weakened, allowing for greater AVOA tolerance. In contrast, beta and alpha results suggest the involvement of higher-level neural processes that may code for language cues (phonetic, lexical), selective attention, and binding of AV percepts, allowing for wider neural windows of temporal integration, i.e., greater AVOA tolerance.

Speech reading enhances auditory perception in noise. One means by which this perceptual facilitation comes about is through information from visual networks reinforcing the encoding of the congruent speech signal by ignoring interfering acoustic signals. We tested this hypothesis neurophysiologically by acquiring EEG while individuals listened to words with a fixed portion of each word replaced by white noise. Congruent (meaningful) or incongruent (reversed frames) mouth movements accompanied the words. Individuals judged whether they heard the words as continuous (illusion) or interrupted (illusion failure) through the noise. We hypothesized that congruent, as opposed to incongruent, mouth movements should further enhance illusory perception by suppressing the auditory cortex's response to interruption onsets and offsets. Indeed, we found that the N1 auditory evoked potential (AEP) to noise onsets and offsets was reduced when individuals experienced the illusion during congruent, but not incongruent, audiovisual streams. This N1 inhibitory effect was most prominent at noise offsets, suggesting that visual influences on auditory perception are instigated to a greater extent during noisy periods. These findings suggest that visual context due to speech-reading disengages (inhibits) neural processes associated with interfering sounds (e.g., noisy interruptions) during speech perception.

A spin-probe method to study the surface hydration of sodium dodecyl sulfate (SDS) micelles (Bales, B. L.; Messina, L.; Vidal, A.; Peric, M.; Nascimento, O. R. J. Phys. Chem. 1998, 102, 10347; referred to as I) is applied to lithium dodecyl sulfate (LiDS) micelles in order to test both the method and a model of micelle hydration. The method is based on the fact that the hyperfine spacing between the low- and center-field resonance lines, A+, varies linearly with a polarity index, H(25 °C), which is the volume fraction occupied by water in a solvent mixture that contains only water as a source of OH dipoles. The model successfully employed in I predicts that H(25 °C) is determined only by the geometry of the micelle; the amount of water associated with the micelle is determined by the volume available to house the water. Thus, SDS and LiDS micelles of the same aggregation number, NA, ought to yield the same value of H(25 °C) (and, therefore A+) because, apart from their waters of hydration which are alread...

When speech is interrupted by noise, listeners often perceptually “fill-in” the degraded signal, giving an illusion of continuity and improving intelligibility. This phenomenon involves a neural process in which the auditory cortex (AC) response to onsets and offsets of acoustic interruptions is suppressed. Since meaningful visual cues behaviorally enhance this illusory filling-in, we hypothesized that during the illusion, lip movements congruent with acoustic speech should elicit a weaker AC response to interruptions relative to static (no movements) or incongruent visual speech. AC response to interruptions was measured as the power and inter-trial phase consistency of the auditory evoked theta band (4–8 Hz) activity of the electroencephalogram (EEG) and the N1 and P2 auditory evoked potentials (AEPs). A reduction in the N1 and P2 amplitudes and in theta phase-consistency reflected the perceptual illusion at the onset and/or offset of interruptions regardless of visual condition. These results suggest that the brain engages filling-in mechanisms throughout the interruption, which repairs degraded speech lasting up to ~ 250 ms following the onset of the degradation. Behaviorally, participants perceived speech continuity over longer interruptions for congruent compared to incongruent or static audiovisual streams. However, this specific behavioral profile was not mirrored in the neural markers of interest. We conclude that lip-reading enhances illusory perception of degraded speech not by altering the quality of the AC response, but by delaying it during degradations so that longer interruptions can be tolerated.

Does musical training affect our perception of speech? For example, does learning to play a musical instrument modify the neural circuitry for auditory processing in a way that improves one’s ability to perceive speech more clearly in noisy environments? If so, can speech perception in individuals with hearing loss, who struggle in noisy situations, benefit from musical training? While music and speech exhibit some specialization in neural processing, there is evidence suggesting that skills acquired through musical training for specific acoustical processes may transfer to, and thereby improve, speech perception. The neurophysiological mechanisms underlying the influence of musical training on speech processing and the extent of this influence remains a rich area to be explored. A prerequisite for such transfer is the facilitation of greater neurophysiological overlap between speech and music processing following musical training. This review first establishes a neurophysiological link between musical training and speech perception, and subsequently provides further hypotheses on the neurophysiological implications of musical training on speech perception in adverse acoustical environments and in individuals with hearing loss.

Objective Auditory training alters neural activity in humans but it is unknown if these alterations are specific to the trained cue. The objective of this study was to determine if enhanced cortical activity was specific to the trained voice-onset-time (VOT) stimuli ‘mba’ and ‘ba’, or whether it generalized to the control stimulus ‘a’ that did not contain the trained cue. Methods Thirteen adults were trained to identify a 10 ms VOT cue that differentiated the two experimental stimuli. We recorded event-related potentials (ERPs) evoked by three different speech sounds ‘ba’ ‘mba’ and ‘a’ before and after six days of VOT training. Results The P2 wave increased in amplitude after training for both control and experimental stimuli, but the effects differed between stimulus conditions. Whereas the effects of training on P2 amplitude were greatest in the left hemisphere for the trained stimuli, enhanced P2 activity was seen in both hemispheres for the control stimulus. In addition, subjects with enhanced pre-training N1 amplitudes were more responsive to training and showed the most perceptual improvement. Conclusion Both stimulus-specific and general effects of training can be measured in humans. An individual’s pre-training N1 response might predict their capacity for improvement. Significance N1 and P2 responses can be used to examine physiological correlates of human auditory perceptual learning.

The goal of this study was to measure the scalp topography of the event-related potentials (ERPs) during the detection of improbable auditory targets and to determine the intracerebral sources of these potentials. ERPs were recorded when subjects listened to a sequence of spoken words and detected occasional (p = 0.2) targets defined either by the gender of the speaker (male/female) or the meaning of the word (animate/inanimate). Waveforms were evaluated in relation to category (target versus standard) and task (voice versus semantic). Dipole source analysis was performed using locations obtained from fMRI. Fronto-central negative waves (N450 and N600 ms) generated by sources in both the auditory cortex and frontal regions were larger for semantic tasks and larger over the left hemisphere. A large parieto-occipital positive wave (P700) occurring with a peak latency about 150 ms before the reaction time was mainly generated in the left temporo-parietal regions for the semantic task and bilaterally for the voice task. About 300 ms after the P700, a highly lateralized right temporo-parietal positive wave P1000r occurred with a source in the right temporo-parietal area. These results indicate three distinct physiological processes underlie the detection of auditory targets. Perceptual discrimination is related to interactions between the frontal and temporal regions, stimulus-response association occurs in the temporo-parietal regions and post-perceptual processing in the right temporo-parietal region.

The human ear has been adapted by evolutionary processes to respond to sound frequencies that are present in the environment and convey information relevant to survival and reproductive fitness. However, the specific features of most sounds that we hear on a second-by-second basis (for example, the harmonic structure, loudness, and temporal shape of a particular voice, language, or musical note) and the meaning attached to these sounds are unique for each individual and cannot be anticipated by a genetic code. The evolutionary response to this limitation on natural selection has been the development of mechanisms that represent the detailed features of sensory input (sensory maps) and update those representations on a millisecond time scale (neural plasticity). We describe two experiments which used auditory evoked potentials (AEPs) to study these processes in the human brain.

Objective To examine how auditory brain responses change with increased spectral complexity of sounds in musicians and non-musicians. Methods Event-related potentials (ERPs) and fields (ERFs) to binaural piano tones were measured in musicians and non-musicians. The stimuli were C4 piano tones and a pure sine tone of the C4 fundamental frequency ( f 0 ). The first piano tone contained f 0 and the first eight harmonics, the second piano tone consisted of f 0 and the first two harmonics and the third piano tone consisted of f 0 . Results Subtraction of ERPs of the piano tone with only the fundamental from ERPs of the harmonically rich piano tones yielded positive difference waves peaking at 130ms (DP130) and 300ms (DP300). The DP130 was larger in musicians than non-musicians and both waves were maximally recorded over the right anterior scalp. ERP source analysis indicated anterior temporal sources with greater strength in the right hemisphere for both waves. Arbitrarily using these anterior sources to analyze the MEG signals showed a DP130m in musicians but not in non-musicians. Conclusions Auditory responses in the anterior temporal cortex to complex musical tones are larger in musicians than non-musicians. Significance Neural networks in the anterior temporal cortex are activated during the processing of complex sounds. Their greater activation in musicians may index either underlying cortical differences related to musical aptitude or cortical modification by acoustical training.

We investigated whether N1 and P2 auditory-evoked responses are modulated by the spectral complexity of musical sounds in pianists and non-musicians. Study participants were presented with three variants of a C4 piano tone equated for temporal envelope but differing in the number of harmonics contained in the stimulus. A fourth tone was a pure tone matched to the fundamental frequency of the piano tones. A simultaneous electroencephalographic/magnetoencephalographic recording was made. P2 amplitude was larger in musicians and increased with spectral complexity preferentially in this group, but N1 did not. The results suggest that P2 reflects the specific features of acoustic stimuli experienced during musical practice and point to functional differences in P2 and N1 that relate to their underlying mechanisms.

We investigated the morphology of the N1-P2 auditory evoked potential (AEP) to changes in amplitude rise time (ART) and rate of formant transition (RFT) of consonant-vowel pairs (CVs) in 4-6-year olds and adults. In the AEP session, individuals listened passively to the CVs /ba/, /wa/, and a /ba/ with a superimposed slower-rising /wa/ envelope (/ba/wa). In the behavioral session, individuals listened to the same stimuli and judged whether they heard a /ba/ or /wa/. In 6-year olds and adults, the N1-P2 amplitude reflected a change in RFT (/ba/wa and /wa/) but not in ART (/ba/ and /ba/wa). In contrast, in the 4-5-year olds, the poorly developed N1-P2 did not show specificity to changes in RFT or ART. Behaviorally, 6-year olds and adults relied more strongly on RFT cues (classified /ba/wa as /ba/) during phonetic judgments, as opposed to 4-5-year olds which utilized both cues equally (chance level). Our findings suggest that following age 4-5, the development of the N1-P2 AEP, representing maturation of syna...

The auditory cortex undergoes anatomical and functional development that reflects specialization for learned sounds. Auditory maturation is evident in the transient auditory evoked potentials (AEPs) and auditory phase‐locked oscillatory activity. Development of AEPs and the phase‐locking strength of oscillatory auditory responses to piano, violin, and pure tones were examined. The hypothesis was that if oscillatory activity in different frequency bands reflects different aspects of sound processing, then the development of phase‐locking at these frequencies will have different maturational trajectories. Phase‐locking for theta (4–8 Hz), alpha (8–14 Hz), lower‐to‐mid‐beta (14–25 Hz), and upper beta and gamma (25–70 Hz) bands strengthened with age and was stronger for musical than pure tones. The phase‐locking increase for gamma and upper beta bands mainly reflected the maturation of the spectral representations for sounds. In contrast, increase in phase‐locking for theta, alpha, and lower‐to‐mid‐beta was m...