Search

Item does not contain fulltext OBJECTIVES: The purpose of this study was to improve bimodal benefit in listeners using a cochlear implant (CI) and a hearing aid (HA) in contralateral ears, by matching the time constants and the number of compression channels of the automatic gain control (AGC) of the HA to the CI. Equivalent AGC was hypothesized to support a balanced loudness for dynamically changing signals like speech and improve bimodal benefit for speech understanding in quiet and with noise presented from the side(s) at 90 degree. DESIGN: Fifteen subjects participated in the study, all using the same Advanced Bionics Harmony CI processor and HA (Phonak Naida S IX UP). In a 3-visit crossover design with 4 weeks between sessions, performance was measured using a HA with a standard AGC (syllabic multichannel compression with 1 ms attack time and 50 ms release time) or an AGC that was adjusted to match that of the CI processor (dual AGC broadband compression, 3 and 240 msec attack time, 80 and 1500 msec release time). In all devices, the AGC was activated above the threshold of 63 dB SPL. The authors balanced loudness across the devices for soft and loud input sounds in 3 frequency bands (0 to 548, 548 to 1000, and >1000 Hz). Speech understanding was tested in free field in quiet and in noise for three spatial speaker configurations, with target speech always presented from the front. Single-talker noise was either presented from the CI side or the HA side, or uncorrelated stationary speech-weighted noise or single-talker noise was presented from both sides. Questionnaires were administered to assess differences in perception between the two bimodal fittings. RESULTS: Significant bimodal benefit over the CI alone was only found for the AGC-matched HA for the speech tests with single-talker noise. Compared with the standard HA, matched AGC characteristics significantly improved speech understanding in single-talker noise by 1.9 dB when noise was presented from the HA side. AGC matching increased bimodal benefit insignificantly by 0.6 dB when noise was presented from the CI implanted side, or by 0.8 (single-talker noise) and 1.1 dB (stationary noise) in the more complex configurations with two simultaneous maskers from both sides. In questionnaires, subjects rated the AGC-matched HA higher than the standard HA for understanding of one person in quiet and in noise, and for the quality of sounds. Listening to a slightly raised voice, subjects indicated increased listening comfort with matched AGCs. At the end of the study, 9 of 15 subjects preferred to take home the AGC-matched HA, 1 preferred the standard HA and 5 subjects had no preference. CONCLUSION: For bimodal listening, the AGC-matched HA outperformed the standard HA in speech understanding in noise tasks using a single competing talker and it was favored in questionnaires and in a subjective preference test. When noise was presented from the HA side, AGC matching resulted in a 1.9 dB SNR additional benefit, even though the HA was at the least favorable SNR side in this speaker configuration. Our results possibly suggest better binaural processing for matched AGCs. 11 p.

The auditory system relies on binaural differences and spectral pinna cues to localize sounds in azimuth and elevation. However, the acoustic input can be unreliable, due to uncertainty about the environment, and neural noise. A possible strategy to reduce sound-location uncertainty is to integrate the sensory observations with sensorimotor information from previous experience, to infer where sounds are more likely to occur. We investigated whether and how human sound localization performance is affected by the spatial distribution of target sounds, and changes thereof. We tested three different open-loop paradigms, in which we varied the spatial range of sounds in different ways. For the narrowest ranges, target-response gains were highly idiosyncratic and deviated from an optimal gain predicted by error-minimization; in the horizontal plane the deviation typically consisted of a response overshoot. Moreover, participants adjusted their behavior by rapidly adapting their gain to the target range, both in elevation and in azimuth, yielding behavior closer to optimal for larger target ranges. Notably, gain changes occurred without any exogenous feedback about performance. We discuss how the findings can be explained by a sub-optimal model in which the motor-control system reduces its response error across trials to within an acceptable range, rather than strictly minimizing the error.

The brain estimates the two-dimensional direction of sounds from the pressure-induced displacements of the eardrums. Accurate localization along the horizontal plane (azimuth angle) is enabled by binaural difference cues in timing and intensity. Localization along the vertical plane (elevation angle), including frontal and rear directions, relies on spectral cues made possible by the elevation dependent filtering in the idiosyncratic pinna cavities. However, the problem of extracting elevation from the sensory input is ill-posed, since the spectrum results from a convolution between source spectrum and the particular head-related transfer function (HRTF) associated with the source elevation, which are both unknown to the system. It is not clear how the auditory system deals with this problem, or which implicit assumptions it makes about source spectra. By varying the spectral contrast of broadband sounds around the 6–9 kHz band, which falls within the human pinna’s most prominent elevation-related spectral notch, we here suggest that the auditory system performs a weighted spectral analysis across different frequency bands to estimate source elevation. We explain our results by a model, in which the auditory system weighs the different spectral bands, and compares the convolved weighted sensory spectrum with stored information about its own HRTFs, and spatial prior assumptions.

Single-unit recordings in head-restrained monkeys indicated that the population of saccade-related cells in the midbrain Superior Colliculus (SC) encodes the kinematics of desired straight saccade trajectories by the cumulative number of spikes. In addition, the nonlinear main sequence of saccades (their amplitude–peak velocity saturation) emerges from a spatial gradient of peak-firing rates of collicular neurons, rather than from neural saturation at brain-stem burst generators. We here extend this idea to eye-head gaze shifts and illustrate how the cumulative spike-count in head-unrestrained monkeys relates to the desired gaze trajectory and its kinematics. We argue that the output of the motor SC is an abstract desired gaze-motor signal, which drives in a feedforward way the instantaneous kinematics of ongoing gaze shifts, including the strong influence of initial eye position on gaze kinematics. We propose that the neural population acts as a vectorial gaze pulse-generator for eye-head saccades, which is sub-sequently decomposed into signals that drive both motor systems in appropriate craniocentric reference frames within a dynamic gaze-velocity feedback loop.

In dynamic visual or auditory gaze double-steps, a brief target flash or sound burst is presented in midflight of an ongoing eye-head gaze shift. Behavioral experiments in humans and monkeys have indicated that the subsequent eye and head movements to the target are goal-directed, regardless of stimulus timing, first gaze shift characteristics, and initial conditions. This remarkable behavior requires that the gaze-control system 1) has continuous access to accurate signals about eye-in-head position and ongoing eye-head movements, 2) that it accounts for different internal signal delays, and 3) that it is able to update the retinal ( TE) and head-centric ( TH) target coordinates into appropriate eye-centered and head-centered motor commands on millisecond time scales. As predictive, feedforward remapping of targets cannot account for this behavior, we propose that targets are transformed and stored into a stable reference frame as soon as their sensory information becomes available. We present a computational model, in which recruited cells in the midbrain superior colliculus drive eyes and head to the stored target location through a common dynamic oculocentric gaze-velocity command, which is continuously updated from the stable goal and transformed into appropriate oculocentric and craniocentric motor commands. We describe two equivalent, yet conceptually different, implementations that both account for the complex, but accurate, kinematic behaviors and trajectories of eye-head gaze shifts under a variety of challenging multisensory conditions, such as in dynamic visual-auditory multisteps.

F.C. Donders was born on May 27, 1818 in Tilburg, the Netherlands. This Summer, the Donders Institute for Brain, Cognition and Behaviour at Radboud University Nijmegen commemorated the 200th annive...

How the human auditory system learns to map complex pinna-induced spectral-shape cues onto veridical estimates of sound-source elevation in the median plane is still unclear. Earlier studies demonstrated considerable sound-localisation plasticity after applying pinna moulds, and to altered vision. Several factors may contribute to auditory spatial learning, like visual or motor feedback, or updated priors. We here induced perceptual learning for sounds with degraded spectral content, having weak, but consistent, elevation-dependent cues, as demonstrated by low-gain stimulus-response relations. During training, we provided visual feedback for only six targets in the midsagittal plane, to which listeners gradually improved their response accuracy. Interestingly, listeners’ performance also improved without visual feedback, albeit less strongly. Post-training results showed generalised improved response behaviour, also to non-trained locations and acoustic spectra, presented throughout the two-dimensional frontal hemifield. We argue that the auditory system learns to reweigh contributions from low-informative spectral bands to update its prior elevation estimates, and explain our results with a neuro-computational model.

Contains fulltext : 200557.pdf (Publisher’s version ) (Open Access) Sensory representations are typically endowed with intrinsic noise, leading to variability and inaccuracies in perceptual responses. The Bayesian framework accounts for an optimal strategy to deal with sensory-motor uncertainty, by combining the noisy sensory input with prior information regarding the distribution of stimulus properties. The maximum-a-posteriori (MAP) estimate selects the perceptual response from the peak (mode) of the resulting posterior distribution that ensure optimal accuracy-precision trade-off when the underlying distributions are Gaussians (minimal mean-squared error, with minimum response variability). We tested this model on human eye- movement responses toward broadband sounds, masked by various levels of background noise, and for head movements to sounds with poor spectral content. We report that the response gain (accuracy) and variability (precision) of the elevation response components changed systematically with the signal-to-noise ratio of the target sound: gains were high for high SNRs and decreased for low SNRs. In contrast, the azimuth response components maintained high gains for all conditions, as predicted by maximum-likelihood estimation. However, we found that the elevation data did not follow the MAP prediction. Instead, results were better described by an alternative decision strategy, in which the response results from taking a random sample from the posterior in each trial. We discuss two potential implementations of a simple posterior sampling scheme in the auditory system that account for the results and argue that although the observed response strategies for azimuth and elevation are sub-optimal with respect to their variability, it allows the auditory system to actively explore the environment in the absence of adequate sensory evidence.

Two synchronous sounds at different locations in the midsagittal plane induce a fused percept at a weighted-average position, with weights depending on relative sound intensities. In the horizontal plane, sound fusion (stereophony) disappears with a small onset asynchrony of 1–4 ms. The leading sound then fully determines the spatial percept (the precedence effect). Given that accurate localisation in the median plane requires an analysis of pinna-related spectral-shape cues, which takes ~25–30 ms of sound input to complete, we wondered at what time scale a precedence effect for elevation would manifest. Listeners localised the first of two sounds, with spatial disparities between 10–80 deg, and inter-stimulus delays between 0–320 ms. We demonstrate full fusion (averaging), and largest response variability, for onset asynchronies up to at least 40 ms for all spatial disparities. Weighted averaging persisted, and gradually decayed, for delays >160 ms, suggesting considerable backward masking. Moreover, response variability decreased with increasing delays. These results demonstrate that localisation undergoes substantial spatial blurring in the median plane by lagging sounds. Thus, the human auditory system, despite its high temporal resolution, is unable to spatially dissociate sounds in the midsagittal plane that co-occur within a time window of at least 160 ms.

After hearing a tone, the human auditory system becomes more sensitive to similar tones than to other tones. Current auditory models explain this phenomenon by a simple bandpass attention filter. Here, we demonstrate that auditory attention involves multiple pass-bands around octave-related frequencies above and below the cued tone. Intriguingly, this "octave effect" not only occurs for physically presented tones, but even persists for the missing fundamental in complex tones, and for imagined tones. Our results suggest neural interactions combining octave-related frequencies, likely located in nonprimary cortical regions. We speculate that this connectivity scheme evolved from exposure to natural vibrations containing octave-related spectral peaks, e.g., as produced by vocal cords. ispartof: Proceedings of the National Academy of Sciences of the United States of America vol:110 issue:38 pages:15225-15230 ispartof: location:United States status: published

Human sound localization in the mid-saggital plane (elevation) relies on an analysis of the idiosyncratic spectral shape cues provided by the head and pinnae. However, because the actual free-field stimulus spectrum is a-priori unknown to the auditory system, the problem of extracting the elevation angle from the sensory spectrum is ill-posed. Here we test different spectral localization models by eliciting head movements toward broad-band noise stimuli with randomly shaped, rippled amplitude spectra emanating from a speaker at a fixed location, while varying the ripple bandwidth between 1.5 and 5.0 cycles/octave. Six listeners participated in the experiments. From the distributions of localization responses toward the individual stimuli, we estimated the listeners' spectral-shape cues underlying their elevation percepts, by applying maximum-likelihood estimation. The reconstructed spectral cues resulted to be invariant to the considerable variation in ripple bandwidth, and for each listener they had a remarkable resemblance to the idiosyncratic head-related transfer functions (HRTFs). These results are not in line with models that rely on the detection of a single peak or notch in the amplitude spectrum, nor with a local analysis of first- and second-order spectral derivatives. Instead, our data support a model in which the auditory system performs a cross-correlation between the sensory input at the eardrum-auditory nerve, and stored representations of HRTF spectral shapes, to extract the perceived elevation angle.

To program a goal-directed response in the presence of multiple sounds, the audiomotor system should separate the sound sources. The authors examined whether the brain can segregate synchronous broadband sounds in the midsagittal plane, using amplitude modulations as an acoustic discrimination cue. To succeed in this task, the brain has to use pinna-induced spectral-shape cues and temporal envelope information. The authors tested spatial segregation performance in the midsagittal plane in two paradigms in which human listeners were required to localize, or distinguish, a target amplitude-modulated broadband sound when a non-modulated broadband distractor was played simultaneously at another location. The level difference between the amplitude-modulated and distractor stimuli was systematically varied, as well as the modulation frequency of the target sound. The authors found that participants were unable to segregate, or localize, the synchronous sounds. Instead, they invariably responded toward a level-weighted average of both sound locations, irrespective of the modulation frequency. An increased variance in the response distributions for double sounds of equal level was also observed, which cannot be accounted for by a segregation model, or by a probabilistic averaging model.

Single-unit recordings suggest that the midbrain superior colliculus (SC) acts as an optimal controller for saccadic gaze shifts. The SC is proposed to be the site within the visuomotor system where the nonlinear spatial-to-temporal transformation is carried out: the population encodes the intended saccade vector by its location in the motor map (spatial), and its trajectory and velocity by the distribution of firing rates (temporal). The neurons' burst profiles vary systematically with their anatomical positions and intended saccade vectors, to account for the nonlinear main-sequence kinematics of saccades. Yet, the underlying collicular mechanisms that could result in these firing patterns are inaccessible to current neurobiological techniques. Here, we propose a simple spiking neural network model that reproduces the spike trains of saccade-related cells in the intermediate and deep SC layers during saccades. The model assumes that SC neurons have distinct biophysical properties for spike generation that depend on their anatomical position in combination with a center-surround lateral connectivity. Both factors are needed to account for the observed firing patterns. Our model offers a basis for neuronal algorithms for spatiotemporal transformations and bio-inspired optimal controllers.

Contains fulltext : 117242.pdf (Publisher’s version ) (Closed access) It is unclear whether top-down processing in the auditory cortex (AC) interferes with its bottom-up analysis of sound. Recent studies indicated non-acoustic modulations of AC responses, and that attention changes a neuron's spectrotemporal tuning. As a result, the AC would seem ill-suited to represent a stable acoustic environment, which is deemed crucial for auditory perception. To assess whether top-down signals influence acoustic tuning in tasks without directed attention, we compared monkey single-unit AC responses to dynamic spectrotemporal sounds under different behavioral conditions. Recordings were mostly made from neurons located in primary fields (primary AC and area R of the AC) that were well tuned to pure tones, with short onset latencies. We demonstrated that responses in the AC were substantially modulated during an auditory detection task and that these modulations were systematically related to top-down processes. Importantly, despite these significant modulations, the spectrotemporal receptive fields of all neurons remained remarkably stable. Our results suggest multiplexed encoding of bottom-up acoustic and top-down task-related signals at single AC neurons. This mechanism preserves a stable representation of the acoustic environment despite strong non-acoustic modulations. 13 p.

Contains fulltext : 167811.pdf (Publisher’s version ) (Open Access) Conclusion In users of a cochlear implant (CI) and a hearing aid (HA) in contralateral ears, frequency-dependent loudness balancing between devices did, on average, not lead to improved speech understanding as compared to broadband balancing. However, nine out of 15 bimodal subjects showed significantly better speech understanding with either one of the fittings. Objectives Sub-optimal fittings and mismatches in loudness are possible explanations for the large individual differences seen in listeners using bimodal stimulation. Methods HA gain was adjusted for soft and loud input sounds in three frequency bands (0-548, 548-1000, and >1000 Hz) to match loudness with the CI. This procedure was compared to a simple broadband balancing procedure that reflected current clinical practice. In a three-visit cross-over design with 4 weeks between sessions, speech understanding was tested in quiet and in noise and questionnaires were administered to assess benefit in real world. Results Both procedures resulted in comparable HA gains. For speech in noise, a marginal bimodal benefit of 0.3 +/- 4 dB was found, with large differences between subjects and spatial configurations. Speech understanding in quiet and in noise did not differ between the two loudness balancing procedures. 7 p.

A world-fixed sound presented to a moving head produces changing sound-localization cues, from which the audiomotor system could infer sound movement relative to the head. When appropriately combined with self-motion signals, sound localization remains spatially accurate. Indeed, free-field orienting responses fully incorporate intervening eye-head movements under open-loop localization conditions. Here we investigate the default strategy of the audiomotor system when localizing sounds in the absence of efferent and proprioceptive head-movement signals. Head- and body-restrained listeners made saccades in total darkness toward brief (3, 10 or 100 ms) broadband noise bursts, while being rotated sinusoidally (f=1/9 Hz, V(peak) =112 deg/s) around the vertical body axis. As the loudspeakers were attached to the chair, the 100 ms sounds might be perceived as rotating along with the chair, and localized in head-centred coordinates. During 3 and 10 ms stimuli, however, the amount of chair rotation remained well below the minimum audible movement angle. These brief sounds would therefore be perceived as stationary in space and, as in open-loop gaze orienting, expected to be localized in world-centred coordinates. Analysis of the saccades shows, however, that all stimuli were accurately localized on the basis of imposed acoustic cues, but remained in head-centred coordinates. These results suggest that, in the absence of motor planning, the audio motor system keeps sounds in head-centred coordinates when unsure about sound motion relative to the head. To that end, it ignores vestibular canal signals of passive-induced head rotation, but incorporates intervening eye displacements from vestibular nystagmus during the saccade-reaction time.

We examined horizontal directional hearing in patients with acquired severe unilateral conductive hearing loss (UCHL). All patients (n = 12) had been fitted with a bone conduction device (BCD) to restore bilateral hearing. The patients were tested in the unaided (monaural) and aided (binaural) hearing condition. Five listeners without hearing loss were tested as a control group while listening with a monaural plug and earmuff, or with both ears (binaural). We randomly varied stimulus presentation levels to assess whether listeners relied on the acoustic head-shadow effect (HSE) for horizontal (azimuth) localization. Moreover, to prevent sound localization on the basis of monaural spectral shape cues from head and pinna, subjects were exposed to narrow band (1/3 octave) noises. We demonstrate that the BCD significantly improved sound localization in 8/12 of the UCHL patients. Interestingly, under monaural hearing (BCD off), we observed fairly good unaided azimuth localization performance in 4/12 of the patients. Our multiple regression analysis shows that all patients relied on the ambiguous HSE for localization. In contrast, acutely plugged control listeners did not employ the HSE. Our data confirm and further extend results of recent studies on the use of sound localization cues in chronic and acute monaural listening.

Orienting responses to audiovisual events in the environment can benefit markedly by the integration of visual and auditory spatial information. However, logically, audiovisual integration would only be considered successful for stimuli that are spatially and temporally aligned, as these would be emitted by a single object in space-time. As humans do not have prior knowledge about whether novel auditory and visual events do indeed emanate from the same object, such information needs to be extracted from a variety of sources. For example, expectation about alignment or misalignment could modulate the strength of multisensory integration. If evidence from previous trials would repeatedly favour aligned audiovisual inputs, the internal state might also assume alignment for the next trial, and hence react to a new audiovisual event as if it were aligned. To test for such a strategy, subjects oriented a head-fixed pointer as fast as possible to a visual flash that was consistently paired, though not always spatially aligned, with a co-occurring broadband sound. We varied the probability of audiovisual alignment between experiments. Reaction times were consistently lower in blocks containing only aligned audiovisual stimuli than in blocks also containing pseudorandomly presented spatially disparate stimuli. Results demonstrate dynamic updating of the subject's prior expectation of audiovisual congruency. We discuss a model of prior probability estimation to explain the results.

To generate an accurate saccade toward a sound in darkness requires a transformation of the head-centered sound location into an oculocentric motor command, which necessitates the use of an eye-in-head position signal. We tested whether this transformation uses a continuous representation of eye position by exploiting the property that the oculomotor neural integrator is leaky with a time constant of ∼20 s. Hence in complete darkness, the eyes tend to drift toward a neutral position. Alternatively, the spatial mapping stage could employ a sampled eye-position signal in which case drift will not be accounted for. Our data show that the sound location is accurately represented and that the transformation uses a dynamic eye-position signal. This signal, however, is slightly underestimated, leading to small systematic localization errors that tend to covary with the direction of eye position.

When a sound is presented in the free field at a location that remains fixed to the head during whole-body rotation in darkness, it is heard displaced in the direction opposing the rotation. This phenomenon is known as the audiogyral illusion. Consequently, the subjective auditory median plane (AMP) (the plane where the binaural difference cues for sound localization are perceived to be zero) shifts in the direction of body rotation. Recent experiments, however, have suggested opposite AMP results when using a fixation light that also moves with the head. Although in this condition the eyes remain stationary in the head, an ocular pursuit signal cancels the vestibulo-ocular reflex, which could induce an additional AMP shift. We tested whether the AMP is influenced by vestibular signals, eye position or eye velocity. We rotated subjects sinusoidally at different velocities, either in darkness or with a head-fixed fixation light, while they judged the laterality (left vs. right with respect to the midsagittal plane of the head) of broadband sounds presented over headphones. Subjects also performed the same task without vestibular stimulation while tracking a sinusoidally moving visual target, which mimicked the average eye-movement patterns of the vestibular experiments in darkness. Results show that whole-body rotation in darkness induces a shift of the AMP in the direction of body rotation. In contrast, we obtained no significant AMP change when a fixation light was used. The pursuit experiments showed a shift of the AMP in the direction of eccentric eye position but not at peak pursuit velocity. We therefore conclude that the vestibular-induced shift in average eye position underlies both the audiogyral illusion and the AMP shift.

Contains fulltext : 168177.pdf (Publisher’s version ) (Open Access) BACKGROUND: Speech understanding may rely not only on auditory, but also on visual information. Non-invasive functional neuroimaging techniques can expose the neural processes underlying the integration of multisensory processes required for speech understanding in humans. Nevertheless, noise (from functional MRI, fMRI) limits the usefulness in auditory experiments, and electromagnetic artifacts caused by electronic implants worn by subjects can severely distort the scans (EEG, fMRI). Therefore, we assessed audio-visual activation of temporal cortex with a silent, optical neuroimaging technique: functional near-infrared spectroscopy (fNIRS). METHODS: We studied temporal cortical activation as represented by concentration changes of oxy- and deoxy-hemoglobin in four, easy-to-apply fNIRS optical channels of 33 normal-hearing adult subjects and five post-lingually deaf cochlear implant (CI) users in response to supra-threshold unisensory auditory and visual, as well as to congruent auditory-visual speech stimuli. RESULTS: Activation effects were not visible from single fNIRS channels. However, by discounting physiological noise through reference channel subtraction (RCS), auditory, visual and audiovisual (AV) speech stimuli evoked concentration changes for all sensory modalities in both cohorts (p < 0.001). Auditory stimulation evoked larger concentration changes than visual stimuli (p < 0.001). A saturation effect was observed for the AV condition. CONCLUSIONS: Physiological, systemic noise can be removed from fNIRS signals by RCS. The observed multisensory enhancement of an auditory cortical channel can be plausibly described by a simple addition of the auditory and visual signals with saturation. 14 p.

Item does not contain fulltext Interaural differences in sound arrival time (ITD) and in level (ILD) enable us to localize sounds in the horizontal plane, and can support source segregation and speech understanding in noisy environments. It is uncertain whether these cues are also available to hearing-impaired listeners who are bimodally fitted, i.e. with a cochlear implant (CI) and a contralateral hearing aid (HA). Here, we assessed sound localization behavior of fourteen bimodal listeners, all using the same Phonak HA and an Advanced Bionics CI processor, matched with respect to loudness growth. We aimed to determine the availability and contribution of binaural (ILDs, temporal fine structure and envelope ITDs) and monaural (loudness, spectral) cues to horizontal sound localization in bimodal listeners, by systematically varying the frequency band, level and envelope of the stimuli. The sound bandwidth had a strong effect on the localization bias of bimodal listeners, although localization performance was typically poor for all conditions. Responses could be systematically changed by adjusting the frequency range of the stimulus, or by simply switching the HA and CI on and off. Localization responses were largely biased to one side, typically the CI side for broadband and high-pass filtered sounds, and occasionally to the HA side for low-pass filtered sounds. HA-aided thresholds better than 45 dB HL in the frequency range of the stimulus appeared to be a prerequisite, but not a guarantee, for the ability to indicate sound source direction. We argue that bimodal sound localization is likely based on ILD cues, even at frequencies below 1500 Hz for which the natural ILDs are small. These cues are typically perturbed in bimodal listeners, leading to a biased localization percept of sounds. The high accuracy of some listeners could result from a combination of sufficient spectral overlap and loudness balance in bimodal hearing. 11 p.

We extend the theory of linear systems analysis to nonlinear systems, by describing the Volterra formalism. The response of a finite memory, causal, and nonlinear system can be described by the Volterra functional series, in which each term captures the influence of higher-order interactions between past stimulus events to the current output through its Volterra kernel. As the Volterra functionals are mutually dependent, they are often rearranged into the Wiener functional series, in which each term is made orthogonal to, and independent from, all other functionals. Orthogonality is imposed by taking Gaussian white noise (GWN) as input and using its autocorrelation properties. The Wiener kernels are independently obtained from cross-correlating output and input. We end with an alternative approach to nonlinear systems identification by a trained artificial feed-forward neural network, from which the Volterra kernels can be independently extracted without the restriction to GWN input.

In this chapter we describe the different egocentric reference frames that are relevant for the control of orienting gaze shifts to brief sounds and lights: oculocentric, craniocentric, and world-centered coordinates. We then discuss how experiments and quantitative models of eye–head gaze orienting can dissociate the different coordinate systems. Neurophysiological experiments have identified two different mechanisms that could potentially cope with coordinate transformations: gain fields and predictive remapping. We describe the static and dynamic double-step paradigms as prime examples of studying the nature of spatial target updating, and discuss the limits of spatial updating performance in situations where the system lacks crucial information about stimulus motion. Finally, we show how eye- and head-position signals interact within the auditory system, and postulate that the sound-localization system aims to represent acoustic targets in a world-centered reference frame.

We discuss how a localized population of saccade-related cells in the superior colliculus (SC) encodes the vector for the upcoming gaze shift. We model the mechanism underlying the afferent complex-logarithmic SC motor map, and how the efferent projections of its cells to the brainstem contribute to encode the saccade, and the tuning characteristics of SC movement fields. We then present evidence that the firing rates of SC cells determine the saccade kinematics, such that the population effectively acts as the nonlinear vectorial pulse generator, hypothesized for the common-source control of eye–head gaze shifts. We show that a spatial gradient of peak firing rates along the rostral–caudal axis of the motor map, together with a fixed number of spikes in the burst, provide the nonlinear mechanism of the main sequence. Finally, we discuss the potential role of the midbrain inferior colliculus in spatial hearing, and how the tonotopic-to-spatial transformation in the auditory system might be understood.

We discuss the encoding of sound at the first stage of neural processing in the auditory system: the auditory nerve (AN). The tuning curves of AN fibers can be described by gammatone filters. AN fibers that are tuned to low frequencies show phase locking, which disappears at high-frequency cells as a result of rectification and low-pass filtering of the acoustic input by the inner hair cells. The phase locking is not perfect, as cell responses show considerable phase jittering, yielding vector strengths of about 0.65. Interestingly, the nonlinear properties of AN cells ensure that high-frequency cells can phase-lock to low-frequency envelopes of acoustic complexes. This interesting property can be used to reconstruct the vibrational patterns of the basilar membrane across it entire length, including the low-frequency apex, which is usually inaccessible to direct measurement.

This chapter summarizes the physical properties of sound waves that will be relevant in the later chapters. A simple elastic gas model can explain how the propagation speed of sound depends only on the bulk modulus and density of the air. We then introduce the homogeneous linear wave equation, from which the superposition of harmonic solutions (standing waves and traveling waves), and the concept of Fourier series are derived. As an example, we discuss the fundamental problem of spectral–temporal resolution. We then describe how inhomogeneities in the medium may change the wave equation and lead to nonharmonic spatial waves. We introduce the concept of acoustic impedance, and how this defines the conditions for transmission and reflection at boundaries between different media. We illustrate these ideas for the middle-ear transfer of acoustic energy from air to the fluid-filled cochlea. Finally, we discuss how the dispersion relation for homogeneous and inhomogeneous media relates to the phase and group velocities of the acoustic signals.

We describe the linear hydrodynamic model of the cochlear partition as developed by Von Bekesy and Zwislocki. The basilar membrane (BM) motion is driven by the instantaneous pressure difference across the organ of Corti, which induces a traveling wave along the BM, which peaks at a frequency-specific location. Although the linear model accounts for the basic tonotopy of the BM, it cannot explain the sharp tuning of auditory nerve fibers at low sound levels, and the strong sound-level dependent (nonlinear) tuning characteristics of the BM. The latter phenomenon requires an active amplifier, which is currently thought to be embodied by precise, extremely fast, and in phase mechanical feedback from outer hair cells. We highlight two nonlinear models that can account for the full dynamic tuning properties of the cochlea.

This final chapter describes the sound-localization abilities of hearing-impaired listeners. We distinguish listeners with a conductive hearing loss from sensory–neural impairments, and describe current technologies to restore impaired hearing: hearing aids, bone-conduction devices, middle-ear implants, and cochlear implants. Single-sided deaf listeners lack binaural hearing and may only use spectral cues from their hearing ear, and the head–shadow effect. Despite the fact that the latter cue is ambiguous, all SSD patients rely heavily on this cue. We argue that this behavior reflects a Bayesian strategy in which prior information is weighted more heavily than less reliable spectral cues. Listeners with a unilateral conductive hearing loss adopt a flexible localization strategy in which they weigh potentially remnant binaural cues, spectral cues, and the head–shadow effect to estimate the azimuth of a sound-source under varying acoustic conditions. Finally, we propose that modest age-related high-frequency hearing loss may be partially compensated by the acoustic effects of pinna growth. For certain sounds the elderly may thus outperform young listeners in localizing source elevation.

This chapter discusses the saccadic eye-movement system and the eye–head gaze-orienting system, as both are implicated in natural sound-localization behavior. Due to the highly inhomogeneous retina, foveate animals optimize their orienting behavior through rapid, accurate, and precise saccadic eye movements. The saccadic system is characterized by a prominent kinematic nonlinearity, which is implemented as a central vectorial pulse generator that provides a common eye–velocity drive to the horizontal and vertical eye–motor plants. We argue that the same vectorial motor command drives the eye- and head-motor systems, albeit that it is modified by appropriate coordinate transformations that let the eyes and head each move in the direction of auditory and visual targets. We discuss different quantitative models that explain the underlying circuitry of eye- and eye–head gaze shifts.

We describe the mechanisms by which the human auditory system extracts spatial information from the tonotopic encoded acoustic inputs. Three implicit acoustic cues are needed to determine the sound-source direction in the horizontal (azimuth) and vertical (elevation) planes: binaural difference cues arising from time (ITD) and level (ILD) differences, and spectral shape cues that are caused by complex acoustic interactions within the pinna cavities. We discuss two major hypotheses for the submillisecond ITD analysis in the auditory brainstem: Jeffress’ delay-line model versus precise-timed inhibition in (small) mammals. We also discuss the neural circuitries that embed the analysis of ILDs and spectral shape. We describe a simple model of the pinna, and point to the ill-posed nature of sound localization and a possible solution. Finally, we highlight how the auditory system might estimate the distance of a sound source in the free field and within enclosed reverberant spaces.

This chapter discusses some of the methods used to quantitatively assess spatial auditory behavior of human listeners. We distinguish discrete, relative, spatial judgments (eg, in lateralization experiments) from continuous, absolute, spatial pointing measures to determine acoustic spatial abilities. The former method is usually analyzed with a statistical modeling technique called Signal Detection Theory. Absolute localization is usually quantified through linear regression analyses. On the basis of earlier comparative studies it is argued that pointing with the eyes to peripheral sound locations provides a fast, natural, accurate, and precise probe to assess sound localization performance under a variety of acoustic conditions. We highlight the independence of azimuth and elevation extraction in sound-localization responses, the influence of temporal factors, and binaural integration of spectral cues in the sound-localization system, and discuss several models to explain the experimental observations. We finally describe the method of virtual acoustics that aims to simulate realistic free-field listening over headphones.

In this chapter we discuss plasticity of the human sound-localization system in response to a variety of acoustic manipulations. We distinguish explicit perceptual learning from implicit sensory–motor feedback learning, and describe experiments that illustrate both types of learning. We introduce the phenomenal plasticity of the barn owl’s auditory system, and describe the immediate localization effects of unilateral ear-canal plugging of human listeners. We discuss why spectral cues are essential to cope with this binaural perturbation. We then describe the adaptive response of humans to long-term unilateral plugging. When the pinna cues are perturbed by molds that change the pinna geometry, subjects relearn to map the new spectral cues to target elevation within a few weeks. Interestingly, these manipulations do not interfere with the original spectral cues, and there is no aftereffect. We argue that pattern-recognition learning (no aftereffects) differs in essential ways from parametric learning (with aftereffects). We demonstrate that the visual system is needed to fine-tune the spatial mapping of spectral cues, by comparing sound localization of congenital blind listeners with normal sighted listeners. Finally, we demonstrate that perturbed spatial vision will perturb the sound-localization mappings in a similar way.

This chapter describes a general approach to the study of linear systems (LS). Starting from the superposition principle, we derive a number of its important consequences: First, the response of a LS to a Dirac delta pulse (the system’s impulse response) completely characterizes the properties of the system. The convolution integral then allows one to predict the system’s response to arbitrary input signals. Second, it follows that the harmonic function (pure sine or cosine) is an Eigenfunction for all LS: the response of an LS to a harmonic input is a harmonic with the same frequency, of which only amplitude and phase may differ. How the latter two depend on frequency is quantified by the system’s transfer characteristic, which is the Fourier transform of the impulse response. Finally, we describe the Laplace transform as a convenient tool for analyzing the responses and transfer characteristics of LS in an intuitive way.

We discuss the mechanisms that guide multisensory integration. In particular, we focus on audiovisual and audio-vestibular interactions that influence spatial perception. We describe the spatial–temporal factors that modulate the strength of multisensory integration, and introduce the phenomenon of inverse effectiveness. We put forward three types of models to account for multisensory interactions on reaction times and response accuracy: race models, interactions within the superior collicular motor map, and Bayesian inference. We discuss the latter framework as a mathematical–statistical model to understand two prominent effects on response trajectories to multisensory stimuli: response averaging and decreased response variability. We show that audiovisual integration in cluttered audiovisual environments follow the spatial–temporal rules of multisensory integration, and provide strong support for inverse effectiveness at the behavioral level. We further show that the sensory–motor system integrates stimuli only when they are spatially and temporally congruent, and keeps track of the stimulus statistics in the environment to assess the probability of audiovisual alignment. Finally, we apply the Bayesian framework to explain the considerable mislocalizations of sounds in the world during head-tilts with respect to gravity.

So far, the double-magnetic induction (DMI) method has been successfully applied to record eye movements from head-restrained humans, monkeys and cats. An advantage of the DMI method, compared to the more widely used scleral search coil technique, is the absence of vulnerable lead wires on the eye. A disadvantage, however, is that the relationship between the eye-in-head orientation and the secondary induction signal is highly non-linear and non-monotonic. This limits the effective measuring range to maximum eye orientations of about +/-30 degrees . Here, we analyze and test two extensions required to record the full eye-head orienting range, well exceeding 90 degrees from straight-ahead in all directions. (1) The use of mutually perpendicular magnetic fields allows for the disambiguation of the non-monotonic signal from the ring. (2) The application of an artificial neural network for offline calibration of the signals. The theoretical predictions are tested for horizontal rotations with a gimbal system. Our results show that the method is a promising alternative to the search coil technique.

Saccades are rapid and ballistic eye-head gaze shifts between points of interest in the visual field. They are crucial for gathering high-resolution visual information. The midbrain superior colliculus (SC) generates saccadic eye-movement commands for downstream oculomotor circuits. It contains an eye-centered, gaze-motor map that relates the location of a Gaussian-shaped neural population to the intended movement vector. The gaze-motor map mediates the spatiotemporal transformation for eye-head orienting gaze shifts to peripheral targets [1]. Electrophysiological recordings have shown that SC neurons exhibit some remarkable activity properties that depend on both their anatomical position and the resulting saccade trajectory [2]. Here, we propose a biologically plausible spiking neural network model that is constrained by the observed firing patterns of real SC neurons for visually evoked saccades. The functional two-dimensional network model reproduces the spike trains of single neurons in recorded SC populations for saccades with different amplitudes and directions. The network model consists of a 2D grid of neurons, representing the gaze-motor map. The adaptive integrate-and-fire neurons [3] portray the observed site-dependent bursting profiles of individual SC neurons through distinct intrinsic biophysical properties, whereas Mexican-hat shaped lateral connections ensure the observed synchronized population activity by a soft winner-takes-all mechanism. We argue that our model offers a basis for neuronal algorithms of spatiotemporal transformations and bio-inspired optimal control signal generators.

Contains fulltext : 153995.PDF (Publisher’s version ) (Open Access) So far, most studies of core auditory cortex (AC) have characterized the spectral and temporal tuning properties of cells in non-awake, anesthetized preparations. As experiments in awake animals are scarce, we here used dynamic spectral-temporal broadband ripples to study the properties of the spectrotemporal receptive fields (STRFs) of AC cells in awake monkeys. We show that AC neurons were typically most sensitive to low ripple densities (spectral) and low velocities (temporal), and that most cells were not selective for a particular spectrotemporal sweep direction. A substantial proportion of neurons preferred amplitude-modulated sounds (at zero ripple density) to dynamic ripples (at non-zero densities). The vast majority (>93%) of modulation transfer functions were separable with respect to spectral and temporal modulations, indicating that time and spectrum are independently processed in AC neurons. We also analyzed the linear predictability of AC responses to natural vocalizations on the basis of the STRF. We discuss our findings in the light of results obtained from the monkey midbrain inferior colliculus by comparing the spectrotemporal tuning properties and linear predictability of these two important auditory stages.

Contains fulltext : 32926.pdf (Publisher’s version ) (Open Access) Human sound localization results primarily from the processing of binaural differences in sound level and arrival time for locations in the horizontal plane (azimuth) and of spectral shape cues generated by the head and pinnae for positions in the vertical plane (elevation). The latter mechanism incorporates two processing stages: a spectral-to-spatial mapping stage and a binaural weighting stage that determines the contribution of each ear to perceived elevation as function of sound azimuth. We demonstrated recently that binaural pinna molds virtually abolish the ability to localize sound-source elevation, but, after several weeks, subjects regained normal localization performance. It is not clear which processing stage underlies this remarkable plasticity, because the auditory system could have learned the new spectral cues separately for each ear (spatial-mapping adaptation) or for one ear only, while extending its contribution into the contralateral hemifield (binaural-weighting adaptation). To dissociate these possibilities, we applied a long-term monaural spectral perturbation in 13 subjects. Our results show that, in eight experiments, listeners learned to localize accurately with new spectral cues that differed substantially from those provided by their own ears. Interestingly, five subjects, whose spectral cues were not sufficiently perturbed, never yielded stable localization performance. Our findings indicate that the analysis of spectral cues may involve a correlation process between the sensory input and a stored spectral representation of the subject's ears and that learning acts predominantly at a spectral-to-spatial mapping level rather than at the level of binaural weighting.

Item does not contain fulltext Auditory and visual target locations are encoded differently in the brain, but must be co-calibrated to maintain cross-sensory concordance. Mechanisms that adjust spatial calibration across modalities have been described (for example, prism adaptation in owls), though rudimentarily in humans. We quantified the adaptation of human sound localization in response to spatially compressed vision (0.5x lenses for 2-3 days). This induced a corresponding compression of auditory localization that was most pronounced for azimuth (minimal for elevation) and was restricted to the visual field of the lenses. Sound localization was also affected outside the field of visual-auditory interaction (shifted centrally, not compressed). These results suggest that spatially modified vision induces adaptive changes in adult human sound localization, including novel mechanisms that account for spatial compression. Findings are consistent with a model in which the central processing of sound location is encoded by recruitment rather than by a place code.

This paper describes the effect of spectro-temporal factors on human sound localization performance in two dimensions (2D). Subjects responded with saccadic eye movements to acoustic stimuli presented in the frontal hemisphere. Both the horizontal (azimuth) and vertical (elevation) stimulus location were varied randomly. Three types of stimuli were used, having different spectro-temporal patterns, but identically shaped broadband averaged power spectra: noise bursts, frequency-modulated tones, and trains of short noise bursts. In all subjects, the elevation components of the saccadic responses varied systematically with the different temporal parameters, whereas the azimuth response components remained equally accurate for all stimulus conditions. The data show that the auditory system does not calculate a final elevation estimate from a long-term (order 100 ms) integration of sensory input. Instead, the results suggest that the auditory system may apply a “multiple-look” strategy in which the final estimate is calculated from consecutive short-term (order few ms) estimates. These findings are incorporated in a conceptual model that accounts for the data and proposes a scheme for the temporal processing of spectral sensory information into a dynamic estimate of sound elevation.

My Ph.D. thesis on human sound localization with perturbed cues.

This is a poster presented at the 2012 Midwinter Meeting of the Association for Research in Otolaryngology.

Contains fulltext : 128121.pdf (Publisher’s version ) (Closed access) We characterised task-related top-down signals in monkey auditory cortex cells by comparing single-unit activity during passive sound exposure with neuronal activity during a predictable and unpredictable reaction-time task for a variety of spectral-temporally modulated broadband sounds. Although animals were not trained to attend to particular spectral or temporal sound modulations, their reaction times demonstrated clear acoustic spectral-temporal sensitivity for unpredictable modulation onsets. Interestingly, this sensitivity was absent for predictable trials with fast manual responses, but re-emerged for the slower reactions in these trials. Our analysis of neural activity patterns revealed a task-related dynamic modulation of auditory cortex neurons that was locked to the animal's reaction time, but invariant to the spectral and temporal acoustic modulations. This finding suggests dissociation between acoustic and behavioral signals at the single-unit level. We further demonstrated that single-unit activity during task execution can be described by a multiplicative gain modulation of acoustic-evoked activity and a task-related top-down signal, rather than by linear summation of these signals.

How the brain maintains an accurate and stable representation of visual target locations despite the occurrence of saccadic gaze shifts is a classical problem in oculomotor research. Here we test and dissociate the predictions of different conceptual models for head-unrestrained gaze-localization behavior of macaque monkeys. We adopted the double-step paradigm with rapid eye-head gaze shifts to measure localization accuracy in response to flashed visual stimuli in darkness. We presented the second target flash either before (static), or during (dynamic) the first gaze displacement. In the dynamic case the brief visual flash induced a small retinal streak of up to about 20 deg at an unpredictable moment and retinal location during the eye-head gaze shift, which provides serious challenges for the gaze-control system. However, for both stimulus conditions, monkeys localized the flashed targets with accurate gaze shifts, which rules out several models of visuomotor control. First, these findings exclude the possibility that gaze-shift programming relies on retinal inputs only. Instead, they support the notion that accurate eye-head motor feedback updates the gaze-saccade coordinates. Second, in dynamic trials the visuomotor system cannot rely on the coordinates of the planned first eye-head saccade either, which rules out remapping on the basis of a predictive corollary gaze-displacement signal. Finally, because gaze-related head movements were also goal-directed, requiring continuous access to eye-in-head position, we propose that our results best support a dynamic feedback scheme for spatial updating in which visuomotor control incorporates accurate signals about instantaneous eye- and head positions rather than relative eye- and head displacements.

Item does not contain fulltext Several studies have attributed deterioration of sound localization in the horizontal (azimuth) and vertical (elevation) planes to an age-related decline in binaural processing and high-frequency hearing loss (HFHL). The latter might underlie decreased elevation performance of older adults. However, as the pinnae keep growing throughout life, we hypothesized that larger ears might enable older adults to localize sounds in elevation on the basis of lower frequencies, thus (partially) compensating their HFHL. In addition, it is not clear whether sound localization has already matured at a very young age, when the body is still growing, and the binaural and monaural sound-localization cues change accordingly. The present study investigated sound-localization performance of children (7-11 years), young adults (20-34 years), and older adults (63-80 years) under open-loop conditions in the two-dimensional frontal hemifield. We studied the effect of age-related hearing loss and ear size on localization responses to brief broadband sound bursts with different bandwidths. We found similar localization abilities in azimuth for all listeners, including the older adults with HFHL. Sound localization in elevation for the children and young adult listeners with smaller ears improved when stimuli contained frequencies above 7 kHz. Subjects with larger ears could also judge the elevation of sound sources restricted to lower frequency content. Despite increasing ear size, sound localization in elevation deteriorated in older adults with HFHL. We conclude that the binaural localization cues are successfully used well into later stages of life, but that pinna growth cannot compensate the more profound HFHL with age.

Sound localization in the horizontal (azimuth) plane relies mainly on interaural time differences (ITDs) and interaural level differences (ILDs). Both are distorted in listeners with acquired unilateral conductive hearing loss (UCHL), reducing their ability to localize sound. Several studies demonstrated that UCHL listeners had some ability to localize sound in azimuth. To test whether listeners with acquired UCHL use strongly perturbed binaural difference cues, we measured localization while they listened with a sound-attenuating earmuff over their impaired ear. We also tested the potential use of monaural pinna-induced spectral-shape cues for localization in azimuth and elevation, by filling the cavities of the pinna of their better-hearing ear with a mould. These conditions were tested while a bone-conduction device (BCD), fitted to all UCHL listeners in order to provide hearing from the impaired side, was turned off. We varied stimulus presentation levels to investigate whether UCHL listeners were using sound level as an azimuth cue. Furthermore, we examined whether horizontal sound-localization abilities improved when listeners used their BCD. Ten control listeners without hearing loss demonstrated a significant decrease in their localization abilities when they listened with a monaural plug and muff. In 4/13 UCHL listeners we observed good horizontal localization of 65 dB SPL broadband noises with their BCD turned off. Localization was strongly impaired when the impaired ear was covered with the muff. The mould in the good ear of listeners with UCHL deteriorated the localization of broadband sounds presented at 45 dB SPL. This demonstrates that they used pinna cues to localize sounds presented at low levels. Our data demonstrate that UCHL listeners have learned to adapt their localization strategies under a wide variety of hearing conditions and that sound-localization abilities improved with their BCD turned on.

An important aspect of binaural hearing is the proper detection of interaural sound level differences and interaural timing differences. Assessments of binaural hearing were made in patients with acquired unilateral conductive hearing loss (UCHL, n = 11) or congenital UCHL (n = 10) after unilateral application of a bone conduction device (BCD), and in patients with bilateral conductive or mixed hearing loss after bilateral BCD application. Benefit (bilateral versus unilateral listening) was assessed by measuring directional hearing, compensation of the acoustic head shadow, binaural summation and binaural squelch. Measurements were performed after an acclimatization time of at least 10 weeks. Unilateral BCD application was beneficial, but there was less benefit in the patients with congenital UCHL as compared to patients with acquired UCHL. In adults with bilateral hearing loss, bilateral BCD application was clearly beneficial as compared to unilateral BCD application. Binaural summation was present, but binaural squelch could not be proven. To explain the poor results in the patients with congenital UCHL, two factors seemed to be important. First, a critical period in the development of binaural hearing might affect the binaural hearing abilities. Second, crossover stimulation, referring to additional stimulation of the cochlea contralateral to the BCD side, might deteriorate binaural hearing in patients with UCHL.