Search

We extend the theory of linear systems analysis to nonlinear systems, by describing the Volterra formalism. The response of a finite memory, causal, and nonlinear system can be described by the Volterra functional series, in which each term captures the influence of higher-order interactions between past stimulus events to the current output through its Volterra kernel. As the Volterra functionals are mutually dependent, they are often rearranged into the Wiener functional series, in which each term is made orthogonal to, and independent from, all other functionals. Orthogonality is imposed by taking Gaussian white noise (GWN) as input and using its autocorrelation properties. The Wiener kernels are independently obtained from cross-correlating output and input. We end with an alternative approach to nonlinear systems identification by a trained artificial feed-forward neural network, from which the Volterra kernels can be independently extracted without the restriction to GWN input.

In this chapter we describe the different egocentric reference frames that are relevant for the control of orienting gaze shifts to brief sounds and lights: oculocentric, craniocentric, and world-centered coordinates. We then discuss how experiments and quantitative models of eye–head gaze orienting can dissociate the different coordinate systems. Neurophysiological experiments have identified two different mechanisms that could potentially cope with coordinate transformations: gain fields and predictive remapping. We describe the static and dynamic double-step paradigms as prime examples of studying the nature of spatial target updating, and discuss the limits of spatial updating performance in situations where the system lacks crucial information about stimulus motion. Finally, we show how eye- and head-position signals interact within the auditory system, and postulate that the sound-localization system aims to represent acoustic targets in a world-centered reference frame.

We discuss how a localized population of saccade-related cells in the superior colliculus (SC) encodes the vector for the upcoming gaze shift. We model the mechanism underlying the afferent complex-logarithmic SC motor map, and how the efferent projections of its cells to the brainstem contribute to encode the saccade, and the tuning characteristics of SC movement fields. We then present evidence that the firing rates of SC cells determine the saccade kinematics, such that the population effectively acts as the nonlinear vectorial pulse generator, hypothesized for the common-source control of eye–head gaze shifts. We show that a spatial gradient of peak firing rates along the rostral–caudal axis of the motor map, together with a fixed number of spikes in the burst, provide the nonlinear mechanism of the main sequence. Finally, we discuss the potential role of the midbrain inferior colliculus in spatial hearing, and how the tonotopic-to-spatial transformation in the auditory system might be understood.

We discuss the encoding of sound at the first stage of neural processing in the auditory system: the auditory nerve (AN). The tuning curves of AN fibers can be described by gammatone filters. AN fibers that are tuned to low frequencies show phase locking, which disappears at high-frequency cells as a result of rectification and low-pass filtering of the acoustic input by the inner hair cells. The phase locking is not perfect, as cell responses show considerable phase jittering, yielding vector strengths of about 0.65. Interestingly, the nonlinear properties of AN cells ensure that high-frequency cells can phase-lock to low-frequency envelopes of acoustic complexes. This interesting property can be used to reconstruct the vibrational patterns of the basilar membrane across it entire length, including the low-frequency apex, which is usually inaccessible to direct measurement.

This chapter summarizes the physical properties of sound waves that will be relevant in the later chapters. A simple elastic gas model can explain how the propagation speed of sound depends only on the bulk modulus and density of the air. We then introduce the homogeneous linear wave equation, from which the superposition of harmonic solutions (standing waves and traveling waves), and the concept of Fourier series are derived. As an example, we discuss the fundamental problem of spectral–temporal resolution. We then describe how inhomogeneities in the medium may change the wave equation and lead to nonharmonic spatial waves. We introduce the concept of acoustic impedance, and how this defines the conditions for transmission and reflection at boundaries between different media. We illustrate these ideas for the middle-ear transfer of acoustic energy from air to the fluid-filled cochlea. Finally, we discuss how the dispersion relation for homogeneous and inhomogeneous media relates to the phase and group velocities of the acoustic signals.

We describe the linear hydrodynamic model of the cochlear partition as developed by Von Bekesy and Zwislocki. The basilar membrane (BM) motion is driven by the instantaneous pressure difference across the organ of Corti, which induces a traveling wave along the BM, which peaks at a frequency-specific location. Although the linear model accounts for the basic tonotopy of the BM, it cannot explain the sharp tuning of auditory nerve fibers at low sound levels, and the strong sound-level dependent (nonlinear) tuning characteristics of the BM. The latter phenomenon requires an active amplifier, which is currently thought to be embodied by precise, extremely fast, and in phase mechanical feedback from outer hair cells. We highlight two nonlinear models that can account for the full dynamic tuning properties of the cochlea.

This final chapter describes the sound-localization abilities of hearing-impaired listeners. We distinguish listeners with a conductive hearing loss from sensory–neural impairments, and describe current technologies to restore impaired hearing: hearing aids, bone-conduction devices, middle-ear implants, and cochlear implants. Single-sided deaf listeners lack binaural hearing and may only use spectral cues from their hearing ear, and the head–shadow effect. Despite the fact that the latter cue is ambiguous, all SSD patients rely heavily on this cue. We argue that this behavior reflects a Bayesian strategy in which prior information is weighted more heavily than less reliable spectral cues. Listeners with a unilateral conductive hearing loss adopt a flexible localization strategy in which they weigh potentially remnant binaural cues, spectral cues, and the head–shadow effect to estimate the azimuth of a sound-source under varying acoustic conditions. Finally, we propose that modest age-related high-frequency hearing loss may be partially compensated by the acoustic effects of pinna growth. For certain sounds the elderly may thus outperform young listeners in localizing source elevation.

This chapter discusses the saccadic eye-movement system and the eye–head gaze-orienting system, as both are implicated in natural sound-localization behavior. Due to the highly inhomogeneous retina, foveate animals optimize their orienting behavior through rapid, accurate, and precise saccadic eye movements. The saccadic system is characterized by a prominent kinematic nonlinearity, which is implemented as a central vectorial pulse generator that provides a common eye–velocity drive to the horizontal and vertical eye–motor plants. We argue that the same vectorial motor command drives the eye- and head-motor systems, albeit that it is modified by appropriate coordinate transformations that let the eyes and head each move in the direction of auditory and visual targets. We discuss different quantitative models that explain the underlying circuitry of eye- and eye–head gaze shifts.

We describe the mechanisms by which the human auditory system extracts spatial information from the tonotopic encoded acoustic inputs. Three implicit acoustic cues are needed to determine the sound-source direction in the horizontal (azimuth) and vertical (elevation) planes: binaural difference cues arising from time (ITD) and level (ILD) differences, and spectral shape cues that are caused by complex acoustic interactions within the pinna cavities. We discuss two major hypotheses for the submillisecond ITD analysis in the auditory brainstem: Jeffress’ delay-line model versus precise-timed inhibition in (small) mammals. We also discuss the neural circuitries that embed the analysis of ILDs and spectral shape. We describe a simple model of the pinna, and point to the ill-posed nature of sound localization and a possible solution. Finally, we highlight how the auditory system might estimate the distance of a sound source in the free field and within enclosed reverberant spaces.

This chapter discusses some of the methods used to quantitatively assess spatial auditory behavior of human listeners. We distinguish discrete, relative, spatial judgments (eg, in lateralization experiments) from continuous, absolute, spatial pointing measures to determine acoustic spatial abilities. The former method is usually analyzed with a statistical modeling technique called Signal Detection Theory. Absolute localization is usually quantified through linear regression analyses. On the basis of earlier comparative studies it is argued that pointing with the eyes to peripheral sound locations provides a fast, natural, accurate, and precise probe to assess sound localization performance under a variety of acoustic conditions. We highlight the independence of azimuth and elevation extraction in sound-localization responses, the influence of temporal factors, and binaural integration of spectral cues in the sound-localization system, and discuss several models to explain the experimental observations. We finally describe the method of virtual acoustics that aims to simulate realistic free-field listening over headphones.

In this chapter we discuss plasticity of the human sound-localization system in response to a variety of acoustic manipulations. We distinguish explicit perceptual learning from implicit sensory–motor feedback learning, and describe experiments that illustrate both types of learning. We introduce the phenomenal plasticity of the barn owl’s auditory system, and describe the immediate localization effects of unilateral ear-canal plugging of human listeners. We discuss why spectral cues are essential to cope with this binaural perturbation. We then describe the adaptive response of humans to long-term unilateral plugging. When the pinna cues are perturbed by molds that change the pinna geometry, subjects relearn to map the new spectral cues to target elevation within a few weeks. Interestingly, these manipulations do not interfere with the original spectral cues, and there is no aftereffect. We argue that pattern-recognition learning (no aftereffects) differs in essential ways from parametric learning (with aftereffects). We demonstrate that the visual system is needed to fine-tune the spatial mapping of spectral cues, by comparing sound localization of congenital blind listeners with normal sighted listeners. Finally, we demonstrate that perturbed spatial vision will perturb the sound-localization mappings in a similar way.

This chapter describes a general approach to the study of linear systems (LS). Starting from the superposition principle, we derive a number of its important consequences: First, the response of a LS to a Dirac delta pulse (the system’s impulse response) completely characterizes the properties of the system. The convolution integral then allows one to predict the system’s response to arbitrary input signals. Second, it follows that the harmonic function (pure sine or cosine) is an Eigenfunction for all LS: the response of an LS to a harmonic input is a harmonic with the same frequency, of which only amplitude and phase may differ. How the latter two depend on frequency is quantified by the system’s transfer characteristic, which is the Fourier transform of the impulse response. Finally, we describe the Laplace transform as a convenient tool for analyzing the responses and transfer characteristics of LS in an intuitive way.

We discuss the mechanisms that guide multisensory integration. In particular, we focus on audiovisual and audio-vestibular interactions that influence spatial perception. We describe the spatial–temporal factors that modulate the strength of multisensory integration, and introduce the phenomenon of inverse effectiveness. We put forward three types of models to account for multisensory interactions on reaction times and response accuracy: race models, interactions within the superior collicular motor map, and Bayesian inference. We discuss the latter framework as a mathematical–statistical model to understand two prominent effects on response trajectories to multisensory stimuli: response averaging and decreased response variability. We show that audiovisual integration in cluttered audiovisual environments follow the spatial–temporal rules of multisensory integration, and provide strong support for inverse effectiveness at the behavioral level. We further show that the sensory–motor system integrates stimuli only when they are spatially and temporally congruent, and keeps track of the stimulus statistics in the environment to assess the probability of audiovisual alignment. Finally, we apply the Bayesian framework to explain the considerable mislocalizations of sounds in the world during head-tilts with respect to gravity.