Search

Studies on multisensory perception often focus on simplistic conditions where one signal is presented per modality. Yet, in everyday life we encounter multiple signals per modality that may be associated with each other. Using the spatial ventriloquism paradigm as model, we tested how the presence of two visual stimuli shapes sound location judgements. Across three experiments we systematically paired one sound with either one or two visual stimuli that differed in relative timing and spatial offset to the sound. We focused on two biases typically studied in this paradigm, the ventriloquism effect and the trial-wise aftereffect. Our results show that these are not shaped by a single visual stimulus in a winner-takes-all manner but rather by their collective influence. In particular, the contribution of individual visual stimuli to the ventriloquism bias depends on their own relative timing to the sound and the temporal context provided by the other visual signal. Depending on this context, two visual signals influence the collective bias with comparable or asymmetric weights, which we attribute to the temporal evolvement of evidence for sensory causal relations during the individual trials. Concerning the aftereffect, we show that this emerges following both auditory and visual judgements in the preceding AV trials and following asynchronous AV stimulus pairs, confirming the notion that this reflects an automatic adjustment to spatial discrepancies.

Multisensory integration and recalibration are two processes by which perception deals with discrepant multisensory signals. In the lab, integration can be probed via the presentation of simultaneous spatially discrepant audio-visual stimuli, i.e. in a spatial ventriloquism paradigm. Recalibration here manifests as an aftereffect bias in unisensory judgements following immediate or long-term exposure to discrepant audio-visual stimuli. Despite many studies exploring the ventriloquism effect (VE) and the immediate aftereffect (VAE), it remains unclear whether the VAE is a direct consequence of the integration of preceding discrepant multisensory signals. We here provide evidence that these two biases are not strictly related. First, we analysed data from ten experiments probing the dependence of each bias on audio-visual discrepancies experienced in multiple preceding trials. This revealed a seven-fold stronger dependence of the VAE on preceding discrepancies compared to the VE. Second, we analysed data from an experiment deviating from the typical trial context in which these biases are probed and found that the VAE can vanish despite the VE being present. We argue that integration may help maintaining a stable percept by reducing immediate sensory discrepancies, whereas recalibration may help maintaining an accurate percept by accounting for consistent discrepancies. Hence, the immediate VAE is not a direct and necessary consequence of the integration of discrepant signals and these two well-studied multisensory response biases can be experimentally dissociated.