Frequency transfer of the ventriloquism aftereffect

The ventriloquism aftereffect (VAE) describes the persistent shift of perceived sound location after having been adapted to a ventriloquism condition, in which the sound was repeatedly paired with a displaced visual stimulus. In the latter case, participants consistently mislocalize the sound in the direction of the visual stimulus (ventriloquism effect, VE). Previous studies provide conflicting reports regarding the strength of the VAE, ranging from 0 to nearly 100%. Moreover, there is controversy about its generalization to different sounds than the one inducing the VE, ranging from no transfer at all, to full transfer across different sound spectra. Here, we imposed the VE for three different sounds: a low-frequency and a high-frequency narrow-band noise, and a broadband Gaussian white noise (GWN). In the adaptation phase, listeners generated fast goal-directed head movements to localize the sound, presented across a 70 deg range in the horizontal plane, while ignoring a visual distracter that was consistently displaced 10 deg to the right of the sound. In the post-adaptation phase, participants localized narrow-band sounds with center frequencies from 0.5 to 8 kHz, as well as GWN, without the visual distracter. Our results show that the VAE amounted to approximately 40% of the VE and generalized well across the entire frequency domain. We also found that the strength of the VAE correlated with the pre-adaptation sound-localization performance. We compare our results with previous reports and discuss different hypotheses regarding optimal audio-visual cue integration.Significance statementWhere we believe sounds to emanate from is heavily influenced by what we see. We experience this illusion - the ventriloquism effect - regularly. For example, when watching movies, we perceive the voices of actors to originate from the actors’ lips rather than from the speakers. Long exposure to a spatial mismatch between what we hear and what we see, leads to a recalibration of our localization of sounds - the ventriloquism aftereffect. Unclear was whether exposure to an audiovisual mismatch for one sound type would also lead to adaptive changes for other sounds. The present study shows that the ventriloquism aftereffect generalizes, implying that it does not act at an early auditory stage, but at a late multisensory stage.