Search

In multistability, perceptual interpretations (“percepts”) of ambiguous stimuli alternate over time. There is considerable debate as to whether similar regularities govern the first percept after stimulus onset and percepts during prolonged presentation. We address this question in a visual pattern-component rivalry paradigm by presenting two overlaid drifting gratings, which participants perceived as individual gratings passing in front of each other (“segregated”) or as a plaid (“integrated”). We varied the enclosed angle (“opening angle”) between the gratings (experiments 1 and 2) and stimulus orientation (experiment 2). The relative number of integrated percepts increased monotonically with opening angle. The point of equality, where half of the percepts were integrated, was at a smaller opening angle at onset than during prolonged viewing. The functional dependence of the relative number of integrated percepts on opening angle showed a steeper curve at onset than during prolonged viewing. Dominance durations of integrated percepts were longer at onset than during prolonged viewing and increased with opening angle. The general pattern persisted when stimuli were rotated (experiment 2), despite some perceptual preference for cardinal motion directions over oblique directions. Analysis of eye movements, specifically the slow phase of the optokinetic nystagmus (OKN), confirmed the veridicality of participants’ reports and provided a temporal characterization of percept formation after stimulus onset. Together, our results show that the first percept after stimulus onset exhibits a different dependence on stimulus parameters than percepts during prolonged viewing. This underlines the distinct role of the first percept in multistability.

A listener who focusses on a sound source of interest must continuously integrate the sounds emitted by the attended source and ignore the sounds emitted by the remaining sources in the auditory scene. Little is known about how the ignored sound sources in the background are mentally represented after the source of interest has formed the perceptual foreground. This is due to a key methodological challenge: the background representation is by definition not overtly reportable. Here we developed a paradigm based on event-related brain potentials (ERPs) to assess the mental representation of background sounds. Participants listened to sequences of three repeatedly presented tones arranged in an ascending order (low, middle, high frequency). They were instructed to detect intensity deviants in one of the tones, creating the perceptual foreground. The remaining two background tones contained timing and location deviants. Those deviants were set up such that mismatch negativity (MMN) components would be elicited in distinct ways if the background was decomposed into two separate sound streams (background segregation) or if it was not further decomposed (background integration). Results provide MMN-based evidence for background segregation and integration in parallel. This suggests that mental representations of background integration and segregation can be concurrently available, and that collecting empirical evidence for only one of these background organization alternatives might lead to erroneous conclusions.

In everyday listening environments, a main task for our auditory system is to follow one out of multiple speakers talking simultaneously. The present study was designed to find electrophysiological indicators of two central processes involved - segregating the speech mixture into distinct speech sequences corresponding to the two speakers, and then attending to one of the speech sequences. We generated multistable speech stimuli that were set up to create ambiguity as to whether only one or two speakers are talking. Thereby we were able to investigate three perceptual alternatives (no segregation, segregated - speaker A in the foreground, segregated - speaker B in the foreground) without any confounding stimulus changes. Participants listened to a continuously repeating sequence of syllables, which were uttered alternately by two human speakers, and indicated whether they perceived the sequence as an inseparable mixture or as originating from two separate speakers. In the latter case, they distinguished which speaker was in their attentional foreground. Our data show a long-lasting event-related potential (ERP) modulation starting at 130ms after stimulus onset, which can be explained by the perceptual organization of the two speech sequences into attended foreground and ignored background streams. Our paradigm extends previous work with pure-tone sequences toward speech stimuli and adds the possibility to obtain neural correlates of the difficulty to segregate a speech mixture into distinct streams.

The auditory environment typically comprises several simultaneously active sound sources. In contrast to the perceptual segregation of two concurrent sounds, the perception of three simultaneous sound objects has not yet been studied systematically. We conducted two experiments in which participants were presented with complex sounds containing sound segregation cues (mistuning, onset asynchrony, differences in frequency or amplitude modulation or in sound location), which were set up to promote the perceptual organization of the tonal elements into one, two, or three concurrent sounds. In Experiment 1, listeners indicated whether they heard one, two, or three concurrent sounds. In Experiment 2, participants watched a silent subtitled movie while EEG was recorded to extract the object-related negativity (ORN) component of the event-related potential. Listeners predominantly reported hearing two sounds when the segregation promoting manipulations were applied to the same tonal element. When two different tonal elements received manipulations promoting them to be heard as separate auditory objects, participants reported hearing two and three concurrent sounds objects with equal probability. The ORN was elicited in most conditions; sounds that included the amplitude- or the frequency-modulation cue generated the smallest ORN amplitudes. Manipulating two different tonal elements yielded numerically and often significantly smaller ORNs than the sum of the ORNs elicited when the same cues were applied on a single tonal element. These results suggest that ORN reflects the presence of multiple concurrent sounds, but not their number. The ORN results are compatible with the horse-race principle of combining different cues of concurrent sound segregation.

By encoding acoustic regularities present in the environment, the human brain can generate predictions of what is likely to occur next. Recent studies suggest that deviations from encoded regularities are detected within 10–50 ms after stimulus onset, as indicated by electrophysiological effects in the middle latency response (MLR) range. This is upstream of previously known long-latency (LLR) signatures of deviance detection such as the mismatch negativity (MMN) component. In the present study, we created predictable and unpredictable contexts to investigate MLR and LLR signatures of the encoding of spatial auditory regularities and the generation of predictions from these regularities. Chirps were monaurally delivered in an either regular (predictable: left–right–left–right) or a random (unpredictable left/right alternation or repetition) manner. Occasional stimulus omissions occurred in both types of sequences. Results showed that the Na component (peaking at 34 ms after stimulus onset) was attenuated for regular relative to random chirps, albeit no differences were observed for stimulus omission responses in the same latency range. In the LLR range, larger chirp-and omission-evoked responses were elicited for the regular than for the random condition, and predictability effects were more prominent over the right hemisphere. We discuss our findings in the framework of a hierarchical organization of spatial regularity encoding. This article is part of a Special Issue entitled SI: Prediction and Attention.

Recent electrophysiological studies have provided evidence that changes in multisensory processing in auditory cortex cannot only be observed following extensive hearing loss, but also in moderately hearing-impaired subjects. How the reduced auditory input affects audio-visual interactions is however largely unknown. Here we used a cross-modal distraction paradigm to investigate multisensory processing in elderly participants with an age-related high-frequency hearing loss as compared to young and elderly subjects with normal hearing. During the experiment, participants were simultaneously presented with independent streams of auditory and visual input and were asked to categorize either the auditory or visual information while ignoring the other modality. Unisensory sequences without any cross-modal input served as control conditions to assure that all participants were able to perform the task. While all groups performed similarly in these unisensory conditions, hearing-impaired participants showed significantly increased error rates when confronted with distracting cross-modal stimulation. This effect could be observed in both the auditory and the visual task. Supporting these findings, an additional regression analysis indicted that the degree of high-frequency hearing loss significantly modulates cross-modal visual distractibility in the auditory task. These findings provide new evidence that already a moderate sub-clinical hearing loss, a common phenomenon in the elderly population, affects the processing of audio-visual information.

Speech signals are often compromised by disruptions originating from external (e.g., masking noise) or internal (e.g., inaccurate articulation) sources. Speech comprehension thus entails detecting and replacing missing information based on predictive and restorative neural mechanisms. The present study targets predictive mechanisms by investigating the influence of a speech segment's predictability on early, modality-specific electrophysiological responses to this segment's omission. Predictability was manipulated in simple physical terms in a single-word framework (Experiment 1) or in more complex semantic terms in a sentence framework (Experiment 2). In both experiments, final consonants of the German words Lachs ([laks], salmon) or Latz ([lats], bib) were occasionally omitted, resulting in the syllable La ([la], no semantic meaning), while brain responses were measured with multi-channel electroencephalography (EEG). In both experiments, the occasional presentation of the fragment La elicited a larger omission response when the final speech segment had been predictable. The omission response occurred ∼125–165 msec after the expected onset of the final segment and showed characteristics of the omission mismatch negativity (MMN), with generators in auditory cortical areas. Suggestive of a general auditory predictive mechanism at work, this main observation was robust against varying source of predictive information or attentional allocation, differing between the two experiments. Source localization further suggested the omission response enhancement by predictability to emerge from left superior temporal gyrus and left angular gyrus in both experiments, with additional experiment-specific contributions. These results are consistent with the existence of predictive coding mechanisms in the central auditory system, and suggestive of the general predictive properties of the auditory system to support spoken word recognition.

This study applied an auditory distraction paradigm to investigate involuntary attention effects of unexpected deviations in speech and non-speech sounds on behavior (increase in response time and error rate) and event-related brain potentials (ΔN1/MMN and P3a). Our aim was to systematically compare identical speech sounds with physical vs. linguistic deviations and identical deviations (pitch) with speech vs. non-speech sounds in the same set of healthy volunteers. Sine tones and bi-syllabic pseudo-words were presented in a 2-alternative forced-choice paradigm with occasional phoneme deviants in pseudo-words, pitch deviants in pseudo-words, or pitch deviants in tones. Deviance-related ERP components were elicited in all conditions. Deviance-related negativities (ΔN1/MMN) differed in scalp distribution between phoneme and pitch deviants within phonemes, indicating that auditory deviance-detection partly operates in a deviance-specific manner. P3a as an indicator of attentional orienting was similar in all conditions, and was accompanied by behavioral indicators of distraction. Yet smaller behavioral effects and prolonged relative MMN-P3a latency were observed for pitch deviants within phonemes relative to the other two conditions. This suggests that the similarity and separability of task-relevant and task-irrelevant information is essential for the extent of attentional capture and distraction.

Auditory transients (such as sound onset or a frequency transition within a continuous sound) are assumed to parse the auditory input into smaller units enabling the formation of unitary sound representations separately for each segment. This was discovered by using the mismatch negativity (MMN) component of the event-related potential (ERP) that taps into auditory sensory memory representations. For unstructured sounds, MMN amplitude decreased or even vanished with increasing the temporal distance of an irregular feature (deviance, e.g. duration decrement) relative to the onset of an otherwise regularly occurring sound, whereas for sounds that were segmented by a transient, MMN persisted. It has been speculated that the P1–N1–P2 complex, indexing the sensory encoding of the transient, determines the temporal units of the acoustic input that are represented by the information processing system. To test this hypothesis, we utilized a previously reported asymmetry in the sensory encoding of physically identical but time-reversed transitions between segments of constant and gliding frequency. In separate blocks, we regularly presented 1400-ms sounds with a centered constant-to-glide or glide-to-constant transition. Occasionally and unpredictably, one of the regularly occurring sounds was shortened in duration to 910 ms. We found larger transition-related P1–N1–P2 potentials accompanied by larger deviance-related MMN amplitudes for sounds with constant-to-glide transition than for sounds with glide-to-constant transition. This provides evidence that it is the precise sensory encoding of the transition which is crucial for automatically parsing the auditory input into smaller units, thus enabling the formation of unitary sound representations even for late segments.

Objective Music perception with a cochlear implant (CI) can be unsatisfactory because current-day implants are primarily designed to enable speech discrimination. The present study aimed at evaluating electrophysiological correlates of musical sound perception in CI users to help achieve the long-term goal of improved restoration of hearing in those individuals. Methods Auditory discrimination accuracy in adult CI users ( n =12) and matched normal-hearing controls ( n =12) was measured by behavioral discrimination tasks and mismatch negativity (MMN) recordings. Discrimination profiles were obtained by using a set of clarinet sounds (original/vocoded) varying along different acoustic dimensions (frequency/intensity/duration) and deviation magnitudes (four levels). Results Behavioral results and MMN recordings revealed reduced auditory discrimination accuracy in CI users. An inverse relationship was found between MMN amplitudes and duration of profound deafness. Conclusions CI users have difficulties in discriminating small changes in the acoustic properties of musical sounds. The recently developed multi-feature MMN paradigm (Pakarinen et al., 2007) can be used to objectively evaluate discrimination abilities of CI users for musical sounds. Significance Measuring auditory discrimination functions by means of a multi-feature MMN paradigm could be of substantial clinical value by providing a comprehensive profile of the extent of restored hearing in CI users.

The separation of concurrently presented sounds into distinct auditory objects is accompanied by the elicitation of the object-related negativity (ORN) component of the event-related potential (ERP). In the present study, participants were exposed to tone sequences containing not only concurrent (harmonicity), but also sequential cues (similarity to previous stimuli in the series) for auditory object formation. Although ORN was consistently elicited by two-object relative to one-object sounds, it was modulated by the presentation rate. With increasing probability of sound segregation, ORN amplitude decreased, and its scalp topography shifted from a bilateral towards a unilateral (left-hemispheric) activation pattern. Probability dependence of the ORN matches previous behavioral data. The differential sensitivity of the left- and right-hemispheric response to this manipulation may explain why an ORN probability effect was not obtained in previous electrophysiological studies. The specific functional role of ORN for concurrent sound perception was confirmed by dissociating it from the mismatch negativity (MMN) component as a correlate of processing sequential cues. ORN probability dependence is argued to be functionally relevant for the interaction of concurrent and sequential mechanisms of auditory scene analysis.

Identifying novel or unexpected events which violate predictions based on the regularities extracted from our environment is crucially important for adaptive behavior. However, the exact dynamics of processing such events is not well understood. Furthermore, it is not known in which degree the process of deviant detection differs across contexts and how much it depends on the characteristics of deviant events. This issue was addressed by the present study which used event-related potentials (ERPs) in order to investigate the dynamics of identifying two types of deviants presented within the same visual setting. These events violated expectations based on two different types of information contained within each trial, either temporal order of stimulus presentation (sequential deviant) or physical attributes shared by the majority of individual stimuli (feature deviant). The obtained results indicate a certain degree of similarity in detecting two deviant types which, when task-relevant, both elicited N2 and P3b event-related potential components. However, significant differences across different stages of their processing were also identified. First, only feature, but not sequential deviants elicited an N1 enhancement. Furthermore, N2 and P3b responses elicited by sequential and feature deviants differed in their latency and topography and, in case of P3b, amplitude. Taken together, these results suggest that the dynamics of detecting different types of deviants strongly depends on the specific characteristics of such events. Furthermore, the identified differences in the topography of N2 and P3b indicate distinct mechanisms underlying several stages of their processing.

The present study aimed at investigating visual distraction in a serial, multi-deviant oddball paradigm with deviant stimuli occurring regularly (every third trial), having a larger overall probability (1/3), and low dimension-specific probability (1/9). Participants performed a categorization task (odd/even) on centrally presented digits. Task-irrelevant geometrical forms were presented concurrently in the left and right periphery of the target. These forms were green triangles that, in every third trial, contained a deviancy either in location, color, or shape at the left or right peripheral position. Behavioral performance and event-related potentials (ERPs) were measured during the multi-deviant blocks and during corresponding control blocks to compensate for physical differences. Results revealed prolonged reaction times for the categorization task in trials containing a deviant feature relative to the respective control condition. Furthermore, two negative shifts were observed in the ERPs for deviant compared to control stimuli, the early one at the ascending part of the N1 component, and the later one at the onset latency of the N2 component. Deviant displays violating a sequential regularity on one of the dimensions thus elicit respective posterior ERP components of change detection and a deterioration in task performance even when they occur as frequently as in every third trial of a sequence. In analogy to findings in audition, these results reveal the importance of regularity processing and its immediate consequences for adaptive behavior also in vision.