Search

This study describes data on auditory-visual integration and visually-guided adaptation of auditory distance perception using the ventriloquism effect (VE) and ventriloquism aftereffect (VAE). In an experiment, participants judged egocentric distance of interleaved auditory or auditory-visual stimuli with the auditory component located from 0.7 to 2.04 m in front of listeners in a real reverberant environment. The visual component of auditory-visual stimuli was displaced 30% closer (V-closer), 30% farther (V-farther), or aligned (V-aligned) with respect to the auditory component. The VE and VAE were measured in auditory and auditory-visual trials, respectively. Both effects were approximately independent of target distance when expressed in logarithmic units. The VE strength, defined as a difference of V-misaligned and V-aligned response bias, was approximately 72% of the auditory-visual disparity regardless of the visual-displacement direction, while the VAE was stronger in the V-farther (44%) than the V-closer (31%) condition. The VAE persisted to post-adaptation auditory-only blocks of trials, although it was diminished. The rates of build-up/break-down of the VAE were asymmetrical, with slower adaptation in the V-closer condition. These results suggest that auditory-visual distance integration is independent of the direction of induced shift, while the re-calibration is stronger and faster when evoked by more distant visual stimuli.

Objetivo: O objetivo do estudo é avaliar a ansiedade da criança antes e após o tratamento por meio da técnica de distração visual auditiva em crianças. Metodologia: É um estudo quantitativo, descritivo e transversal. O estudo foi realizado entre 2016 e 2017 em duas clínicas de estomatologia pediátrica. A ansiedade foi avaliada antes e após o tratamento pela técnica audiovisual. Resultados: Verifica-se que a ansiedade leve aumenta após o tratamento de 18% para 47%. A ansiedade moderada a severa diminui de 16% para 3%, observa-se que nenhuma criança após o tratamento apresenta ansiedade severa. Esses resultados são estatisticamente significativos. Conclusões: Em conclusão, a técnica de distração auditiva visual pode ser um método eficaz para reduzir os níveis de ansiedade em pacientes pediátricos durante o tratamento odontológico invasivo.

Children with autism spectrum disorder (ASD) often display impaired listener skills, and few studies have evaluated procedures for establishing initial auditory-visual conditional discrimination skills. We developed and evaluated a treatment package for training initial auditory-visual conditional discriminations based on the extant research on training such discriminations in children with ASD with at least some preexisting skills in this area. The treatment package included (a) conditional-only training, (b) prompting the participant to echo the sample stimulus as a differential observing response, (c) prompting correct selection responses using an identity-match prompt, (d) using progressively delayed prompts, and (e) repeating trials until the participant emitted an independent correct response. Results indicated all participants mastered all listener targets, and the two participants for whom we tested the emergence of corresponding tacts showed mastery of most tacts without direct training. We discuss these results relative to prior research on listener skills and tacts.

Some classical studies on temporal order judgments (TOJ) suggested a single central process comparing stimulus onsets across modalities. The prevalent current view suggests that there is modality-specific timing estimation followed by a cross-modal stage. If the latter view is correct, TOJ's may vary depending on stimulus modality. Further, if TOJ is based only on onsets, stimulus duration should be irrelevant. To address these issues, we used both unisensory and multisensory stimuli to test whether unisensory duration processing influences cross-modal TOJ's. The stimuli were auditory noise bursts, visual squares, and their cross-modal combinations presented at 10, 40 and 500 ms durations, and various stimulus onset asynchronies. Psychometric functions were measured with an identical task in all conditions: On each trial, two stimuli were presented, one to the left, the other to the right of fixation. The participants judged which one started first. TOJ's were little affected by stimulus duration, implying that they are mainly determined by stimulus onsets. Throughout, the cross-modal just noticeable differences were larger than the unisensory ones. In accordance with the current view, our results suggest that cross-modal TOJ's require a comparison of timing after modality-specific estimations.

Aim: The present study was carried out to evaluate whether there exists a relation between two scoring procedures (score and span) to evaluate memory as well as sequencing within three modality con...

This study examined the effects of age and hearing loss on recognition of speech presented when the auditory and visual speech information was misaligned in time (i.e., asynchronous). Prior research suggests that older listeners are less sensitive than younger listeners in detecting the presence of asynchronous speech for auditory-lead conditions, but recognition of speech in auditory-lead conditions has not yet been examined. Recognition performance was assessed for sentences and words presented in the auditory-visual modalities with varying degrees of auditory lead and lag. Detection of auditory-visual asynchrony for sentences was assessed to verify that listeners detected these asynchronies. The listeners were younger and older normal-hearing adults and older hearing-impaired adults. Older listeners (regardless of hearing status) exhibited a significant decline in performance in auditory-lead conditions relative to visual lead, unlike younger listeners whose recognition performance was relatively stable across asynchronies. Recognition performance was not correlated with asynchrony detection. However, one of the two cognitive measures assessed, processing speed, was identified in multiple regression analyses as contributing significantly to the variance in auditory-visual speech recognition scores. The findings indicate that, particularly in auditory-lead conditions, listener age has an impact on the ability to recognize asynchronous auditory-visual speech signals.

Visual capture and the ventriloquism aftereffect resolve spatial disparities of incongruent auditory visual (AV) objects by shifting auditory spatial perception to align with vision. Here, we demonstrated the distinct temporal characteristics of visual capture and the ventriloquism aftereffect in response to brief AV disparities. In a set of experiments, subjects localized either the auditory component of AV targets (A within AV) or a second sound presented at varying delays (1-20 s) after AV exposure (A2 after AV). AV targets were trains of brief presentations (1 or 20), covering a ±30° azimuthal range, and with ±8° (R or L) disparity. We found that the magnitude of visual capture generally reached its peak within a single AV pair and did not dissipate with time, while the ventriloquism aftereffect accumulated with repetitions of AV pairs and dissipated with time. Additionally, the magnitude of the auditory shift induced by each phenomenon was uncorrelated across listeners and visual capture was unaffected by subsequent auditory targets, indicating that visual capture and the ventriloquism aftereffect are separate mechanisms with distinct effects on auditory spatial perception. Our results indicate that visual capture is a 'sample-and-hold' process that binds related objects and stores the combined percept in memory, whereas the ventriloquism aftereffect is a 'leaky integrator' process that accumulates with experience and decays with time to compensate for cross-modal disparities.

Simple Summary Comparative investigations on infants’ and dogs’ social and communicative skills revealed striking similarity, which can be attributed to convergent evolutionary and domestication processes. Using a suitable experimental method that allows systematic and direct comparisons of dogs and humans is essential. In the current study, we used non-invasive eye-tracking technology in order to investigate looking behaviour of dogs and human infants in an auditory–visual matching task. We found a similar gazing pattern in the two species when they were presented with pictures and vocalisations of a dog and a female human, that is, both dogs and infants looked longer at the dog portrait during the dog’s bark, while matching human speech with the human face was less obvious. Our results suggested different mechanisms underlying this analogous behaviour and highlighted the importance of future investigations into cross-modal cognition in dogs and humans. Abstract We tested whether dogs and 14–16-month-old infants are able to integrate intersensory information when presented with conspecific and heterospecific faces and vocalisations. The looking behaviour of dogs and infants was recorded with a non-invasive eye-tracking technique while they were concurrently presented with a dog and a female human portrait accompanied with acoustic stimuli of female human speech and a dog’s bark. Dogs showed evidence of both con- and heterospecific intermodal matching, while infants’ looking preferences indicated effective auditory–visual matching only when presented with the audio and visual stimuli of the non-conspecifics. The results of the present study provided further evidence that domestic dogs and human infants have similar socio-cognitive skills and highlighted the importance of comparative examinations on intermodal perception.

Introduction Enhanced auditory, tactile, and visual feedback systems have been researched as alternative means of sensory feedback for people with severe to profound hearing loss to improve articulation and intelligibility of speech. The general consensus is that children with hearing loss are on average 20% intelligible, which translates to one word out of every five is understood by the listener. As children enter school, their speech intelligibility is often considered an indication of their abilities, meaning that low intelligibility could result in underestimation of potential. For adults who experience hearing loss, articulation has been shown to dramatically decrease over time, resulting in lowered intelligibility. Purpose The purpose of this research is to compare each form of feedback to determine which form of sensory feedback provides the highest levels of efficacy for preschool aged children (3 to 5 years), school aged children (6 to 17 years), and adults (18+ years). Conclusion A combination of biofeedback types is suggested for school aged children and adults, as each form demonstrated improvement for differing areas of speech. Visual and tactile are not recommended for the preschool children, as it is assumed these forms of feedback are too complex to be translated at this age. Tactile is not recommended for school aged children as there were inconclusive findings. For adults, each of the feedback systems demonstrated positive effects, indicating the three systems should be paired for optimal and most efficient results.

Acquired auditory-visual synesthesia (AVS) is a rare neurological sign, in which specific auditory stimulation triggers visual experience. In this study, we used event-related fMRI to explore the brain regions correlated with acquired monocular sound-induced phosphenes, which occurred 2 months after unilateral visual loss due to an ischemic optic neuropathy. During the fMRI session, 1-s pure tones at various pitches were presented to the patient, who was asked to report occurrence of sound-induced phosphenes by pressing one of the two buttons (yes/no). The brain activation during phosphene-experienced trials was contrasted with non-phosphene trials and compared to results obtained in one healthy control subject who underwent the same fMRI protocol. Our results suggest, for the first time, that acquired AVS occurring after visual impairment is associated with bilateral activation of primary and secondary visual cortex, possibly due to cross-wiring between auditory and visual sensory modalities.

This paper proposes a novel approach to assess audiovisual integration for both congruent and incongruent speech stimuli using reaction times (RT). The experiments are based on the McGurk effect, in which a listener is presented with incongruent audiovisual speech signals. A typical example involves the auditory consonant/b/combined with a visually articulated/g/, often yielding a perception of/d/. We quantify the amount of integration relative to the predictions of a parallel independent model as a function of attention and congruency between auditory and visual signals. We assessed RT distributions for congruent and incongruent auditory and visual signals in a within-subjects signal detection paradigm under conditions of divided versus focused attention. Results showed that listeners often received only minimal benefit from congruent auditory visual stimuli, even when such information could have improved performance. Incongruent stimuli adversely affected performance in divided and focused attention conditions. Our findings support a parallel model of auditory-visual integration with interactions between auditory and visual channels.

This study aimed to investigate how individuals with bipolar disorder integrate auditory and visual speech information compared to healthy individuals. Furthermore, we wanted to see whether there were any differences between manic and depressive episode bipolar disorder patients with respect to auditory and visual speech integration. It was hypothesized that the bipolar group&rsquo, s auditory&ndash, visual speech integration would be weaker than that of the control group. Further, it was predicted that those in the manic phase of bipolar disorder would integrate visual speech information more robustly than their depressive phase counterparts. To examine these predictions, a McGurk effect paradigm with an identification task was used with typical auditory&ndash, visual (AV) speech stimuli. Additionally, auditory-only (AO) and visual-only (VO, lip-reading) speech perceptions were also tested. The dependent variable for the AV stimuli was the amount of visual speech influence. The dependent variables for AO and VO stimuli were accurate modality-based responses. Results showed that the disordered and control groups did not differ in AV speech integration and AO speech perception. However, there was a striking difference in favour of the healthy group with respect to the VO stimuli. The results suggest the need for further research whereby both behavioural and physiological data are collected simultaneously. This will help us understand the full dynamics of how auditory and visual speech information are integrated in people with bipolar disorder.

The focus of this study was to investigate how individuals with bipolar disorder integrate auditory and visual speech information compared to non-disordered individuals and whether there were any differences in auditory and visual speech integration in the manic and depressive episodes in bipolar disorder patients. It was hypothesized that bipolar groups’ auditory-visual speech integration would be less robust than the control group. Further, it was predicted that those in the manic phase of bipolar disorder would integrate visual speech information more than their depressive phase counterparts. To examine these, the McGurk effect paradigm was used with typical auditory-visual speech (AV) as well as auditory-only (AO) speech perception on visual-only (VO) stimuli. Results. Results showed that the disordered and non-disordered groups did not differ on auditory-visual speech (AV) integration and auditory-only (AO) speech perception but on visual-only (VO) stimuli. The results are interpreted to pave the way for further research whereby both behavioural and physiological data are collected simultaneously. This will allow us understand the full dynamics of how, actually, the auditory and visual (relatively impoverished in bipolar disorder) speech information are integrated in people with bipolar disorder.

In dynamic visual or auditory gaze double-steps, a brief target flash or sound burst is presented in midflight of an ongoing eye-head gaze shift. Behavioral experiments in humans and monkeys have indicated that the subsequent eye and head movements to the target are goal-directed, regardless of stimulus timing, first gaze shift characteristics, and initial conditions. This remarkable behavior requires that the gaze-control system 1) has continuous access to accurate signals about eye-in-head position and ongoing eye-head movements, 2) that it accounts for different internal signal delays, and 3) that it is able to update the retinal ( TE) and head-centric ( TH) target coordinates into appropriate eye-centered and head-centered motor commands on millisecond time scales. As predictive, feedforward remapping of targets cannot account for this behavior, we propose that targets are transformed and stored into a stable reference frame as soon as their sensory information becomes available. We present a computational model, in which recruited cells in the midbrain superior colliculus drive eyes and head to the stored target location through a common dynamic oculocentric gaze-velocity command, which is continuously updated from the stable goal and transformed into appropriate oculocentric and craniocentric motor commands. We describe two equivalent, yet conceptually different, implementations that both account for the complex, but accurate, kinematic behaviors and trajectories of eye-head gaze shifts under a variety of challenging multisensory conditions, such as in dynamic visual-auditory multisteps.

Author(s): Gabouer, Allison; Oghalai, John; Bortfeld, Heather | Abstract: Parent-child dyads in which the child is deaf but the parent is hearing present a unique opportunity to examine parents’ use of non-auditory cues, particularly vision and touch, to establish communicative intent. This study examines the multimodal communication patterns of hearing parents during a free play task with their hearing (N=9) or deaf (N=9) children. Specifically, we coded parents’ use of multimodal cues in the service of establishing joint attention with their children. Dyad types were compared for overall use of multimodal – auditory, visual, and tactile – attention-establishing cues, and for the overall number of successful and failed bids by a parent for a child’s attention. The relationship between multimodal behaviors on the part of the parent were tracked for whether they resulted in successful or failed initiation of joint attention. We focus our interpretation of the results on how hearing parents differentially accommodate their hearing and deaf children to engage them in joint attention. Findings can inform the development of recommendations for hearing parents of deaf children who are candidates for cochlear implantation regarding communication strategies to use prior to a child’s implantation. Moreover, these findings expand our understanding of how joint attention is established between parents and their preverbal children, regardless of children’s hearing status.

We continuously receive the external information from multiple sensors simultaneously. The brain must judge a source event of these sensory informations and integrate them. It is thought that judging the simultaneity of such multisensory stimuli is an important cue when we discriminate whether the stimuli are derived from one event or not. Although previous studies have investigated the correspondence between an auditory-visual (AV) simultaneity perceptions and the neural responses, there are still few studies of this. Electrophysiological studies have reported that ongoing oscillations in human cortex affect perception. Especially, the phase resetting of ongoing oscillations has been examined as it plays an important role in multisensory integration. The aim of this study was to investigate the relationship of phase resetting for the judgment of AV simultaneity judgement tasks. The subjects were successively presented with auditory and visual stimuli with intervals that were controlled as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\mathrm{SOA_{50\%}}$$\end{document}SOA50% and they were asked to report whether they perceived them simultaneously or not. We investigated the effects of the phase of ongoing oscillations on simultaneity judgments with AV stimuli with SOAs in which the detection rate of asynchrony was 50 %. It was found that phase resetting at the beta frequency band in the brain area that related to the modality of the following stimulus occurred after preceding stimulus onset only when the subjects perceived AV stimuli as simultaneous. This result suggested that beta phase resetting occurred in areas that are related to the subsequent stimulus, supporting perception multisensory stimuli as simultaneous.

Conditionally automated driving systems may soon be available on the market. Even though these systems exempt drivers from the driving task for extended periods of time, drivers are expected to take back control when the automation issues a so-called take-over request. This study investigated the interaction between take-over request modality and type of non-driving task, regarding the driver's reaction time. It was hypothesized that reaction times are higher when the non-driving task and the take-over request use the same modality. For example, auditory take-over requests were expected to be relatively ineffective in situations in which the driver is making a phone call. 101 participants, divided into three groups, performed one of three non-driving tasks, namely reading (i.e., visual task), calling (auditory task), or watching a video (visual/auditory task). Results showed that auditory and tactile take-over requests yielded overall faster reactions than visual take-over requests. The expected interaction between takeover modality and the dominant modality of the non-driving task was not found. As for self-reported usefulness, auditory and tactile take-over requests yielded higher scores than visual ones. In conclusion, it seems that auditory and tactile stimuli are equally effective as take-over requests, regardless of the non-driving task. Further study into the effects of realistic non-driving tasks is needed to identify which non-driving tasks are detrimental to safety in automated driving.

Children with autism spectrum disorder are typically taught conditional discriminations using a match-to-sample arrangement. Consideration should be given to the temporal order in which antecedent stimuli (the sample and comparison stimuli) are presented during match-to-sample trials, as various arrangements have been used in the extant literature. The purpose of the current study was to compare the effects of four stimulus presentation orders on the acquisition of auditory-visual conditional discriminations. The study included participants from a clinically relevant population (three children with autism spectrum disorder), employed clinically relevant teaching procedures, and included two presentation formats not included in previous comparison evaluations (simultaneous and sample-first with re-presentation conditions). Results were found to be learner-specific; that is, a different stimulus presentation format was most efficient for each participant. We provide suggestions to evaluate stimulus control topographies and enhance experimental control in match-to-sample arrangements.

Visual distractions are present in real-world listening environments, such as conversing in a crowded restaurant. This study examined the impact of visual distractors on younger and older adults' ability to understand auditory-visual (AV) speech in noise. AV speech stimuli were presented with one competing talker and with three different types of visual distractors. SNR50 thresholds for both listener groups were affected by visual distraction; the poorest performance for both groups was the AV + Video condition, and differences across groups were noted for some conditions. These findings suggest that older adults may be more susceptible to irrelevant auditory and visual competition in a real-world environment.

Objective The study evaluated whether there exists a difference/relation in the way four different memory skills (memory score, sequencing score, memory span, & sequencing span) are processed through the auditory modality, visual modality and combined modalities. Methods Four memory skills were evaluated on 30 typically developing children aged 7 years and 8 years across three modality conditions (auditory, visual, & auditory-visual). Analogous auditory and visual stimuli were presented to evaluate the three modality conditions across the two age groups. Results The children obtained significantly higher memory scores through the auditory modality compared to the visual modality. Likewise, their memory scores were significantly higher through the auditory-visual modality condition than through the visual modality. However, no effect of modality was observed on the sequencing scores as well as for the memory and the sequencing span. A good agreement was seen between the different modality conditions that were studied (auditory, visual, & auditory-visual) for the different memory skills measures (memory scores, sequencing scores, memory span, & sequencing span). A relatively lower agreement was noted only between the auditory and visual modalities as well as between the visual and auditory-visual modality conditions for the memory scores, measured using Bland-Altman plots. Conclusions The study highlights the efficacy of using analogous stimuli to assess the auditory, visual as well as combined modalities. The study supports the view that the performance of children on different memory skills was better through the auditory modality compared to the visual modality.

Space is a dimension shared by different modalities, but at what stage spatial encoding is affected by multisensory processes is unclear. Early studies observed attenuation of N1/P2 auditory evoked responses following repetition of sounds from the same location. Here, we asked whether this effect is modulated by audiovisual interactions. In two experiments, using a repetition-suppression paradigm, we presented pairs of tones in free field, where the test stimulus was a tone presented at a fixed lateral location. Experiment 1 established a neural index of auditory spatial sensitivity, by comparing the degree of attenuation of the response to test stimuli when they were preceded by an adapter sound at the same location versus 30° or 60° away. We found that the degree of attenuation at the P2 latency was inversely related to the spatial distance between the test stimulus and the adapter stimulus. In Experiment 2, the adapter stimulus was a tone presented from the same location or a more medial location than the test stimulus. The adapter stimulus was accompanied by a simultaneous flash displayed orthogonally from one of the two locations. Sound-flash incongruence reduced accuracy in a same-different location discrimination task (i.e., the ventriloquism effect) and reduced the location-specific repetition-suppression at the P2 latency. Importantly, this multisensory effect included topographic modulations, indicative of changes in the relative contribution of underlying sources across conditions. Our findings suggest that the auditory response at the P2 latency is affected by spatially selective brain activity, which is affected crossmodally by visual information.

Purpose Five experiments probed auditory-visual (AV) understanding of sentences by users of cochlear implants (CIs). Method Sentence material was presented in auditory (A), visual (V), and AV test conditions to listeners with normal hearing and CI users. Results (a) Most CI users report that most of the time, they have access to both A and V information when listening to speech. (b) CI users did not achieve better scores on a task of speechreading than did listeners with normal hearing. (c) Sentences that are easy to speechread provided 12 percentage points more gain to speech understanding than did sentences that were difficult. (d) Ease of speechreading for sentences is related to phrase familiarity. (e) Users of bimodal CIs benefit from low-frequency acoustic hearing even when V cues are available, and a second CI adds to the benefit of a single CI when V cues are available. (f) V information facilitates lexical segmentation by improving the recognition of the number of syllables produced and the relative strength of these syllables. Conclusions Our data are consistent with the view that V information improves CI users' ability to identify syllables in the acoustic stream and to recognize their relative juxtaposed strengths. Enhanced syllable resolution allows better identification of word onsets, which, when combined with place-of-articulation information from visible consonants, improves lexical access.

The perceived intensity for happy and sad emotions in three sensory domains-AV (auditory-visual), VO (visual-only) and AO (auditory-only)-was compared with 58 undergraduate college students using a within-subject design. Both happy and sad emotions were rated higher in intensity in the AV (auditory-visual) condition compared to the VO (visual-only) and AO (auditory-only) conditions separately. There was no significant difference observed between VO (visual-only) and AO (auditory-only) conditions for either happy or sad emotional stimuli. The implications for these findings are discussed especially with respect to the possibly decreased impact of intensity of emotions in different modes of communication like phone or email.

This study examined the effect of auditory-visual (AV) speech stimuli on working memory in older adults with poorer-hearing (PH) in comparison to age- and education-matched older adults with better hearing (BH). Participants completed a working memory n-back task (0- to 2-back) in which sequences of digits were presented in visual-only (i.e., speech-reading), auditory-only (A-only), and AV conditions. Auditory event-related potentials (ERP) were collected to assess the relationship between perceptual and working memory processing. The behavioral results showed that both groups were faster in the AV condition in comparison to the unisensory conditions. The ERP data showed perceptual facilitation in the AV condition, in the form of reduced amplitudes and latencies of the auditory N1 and/or P1 components, in the PH group. Furthermore, a working memory ERP component, the P3, peaked earlier for both groups in the AV condition compared to the A-only condition. In general, the PH group showed a more robust AV benefit; however, the BH group showed a dose-response relationship between perceptual facilitation and working memory improvement, especially for facilitation of processing speed. Two measures, reaction time and P3 amplitude, suggested that the presence of visual speech cues may have helped the PH group to counteract the demanding auditory processing, to the level that no group differences were evident during the AV modality despite lower performance during the A-only condition. Overall, this study provides support for the theory of an integrated perceptual-cognitive system. The practical significance of these findings is also discussed.

To provide a comprehensive investigation of goal-oriented saccades, experiments are designed to trigger fast eye movements by different types of targets, including visual, auditory, and auditory-visual bisensory stimuli. Human saccades are recorded using a high speed eye tracking system. Data is analyzed to produce parameter estimates using system identification technique for a 3rd-order linear horizontal saccadic eye movement model. Saccade characteristics are explored and compared. The auditory-visual stimuli provide the greatest saccade accuracy. Auditory saccades show lower peak velocity and longer duration. Saccade latent period is relatively independent of saccade amplitude, but there is a significant reduction in the bisensory saccades. Neural inputs of saccades are estimated and analyzed as well. Auditory saccades exhibit lower agonist pulse magnitude and longer agonist pulse duration. Antagonist onset delay is longer in auditory saccades. Post saccade phenomena are caused by the post-inhibitory rebound burst of the antagonist motoneurons. There is a higher incidence of dynamic overshoot in auditory saccades than visual saccades, with more in the abducting direction than the adducting direction.

Статья рассматривает исторические аспекты развития идеи использования свето-звуковой стимуляции мозга для влияния на разные виды нарушений психических состояний.

Through a series of investigations involving different levels of contextual fidelity we devel- oped scales of perceived urgency for several dimensions of the auditory, visual, and tactile modalities. Psychophysical ratings of perceived urgency, annoyance, and acceptability as well as behavioral re- sponses to signals in each modality were obtained and analyzed using Steven's Power Law to allow comparison across modalities. Obtained results and their implications for use as in-vehicle alerts and warnings are discussed.

From birth, infants detect associations between the locations of static visual objects and sounds they emit, but there is limited evidence regarding their sensitivity to the dynamic equivalent when a sound-emitting object moves. In 4 experiments involving thirty-six 2-month-olds, forty-eight 5-month-olds, and forty-eight 8-month-olds, we investigated infants' ability to process this form of spatial colocation. Whereas there was no evidence of spontaneous sensitivity, all age groups detected a dynamic colocation during habituation and looked longer at test trials in which sound and sight were dislocated. Only 2-month-olds showed clear sensitivity to the dislocation relation, although 8-month-olds did so following additional habituation. These results are discussed relative to the intersensory redundancy hypothesis and work suggesting increasing specificity in processing with age.

Speech understanding is improved when the observer can both see and hear the talker. This study compared the effects of reduced visual acuity on auditory-visual (AV) speech-recognition in noise among younger and older adults. Two groups of participants performed a closed-set sentence-recognition task in one auditory-alone (A-alone) condition and under three AV conditions: normal visual acuity (6/6), and with blurred vision to simulate a 6/30 and 6/60 visual impairment. The results showed that (1) the addition of visual speech cues improved speech-perception relative to the A-alone condition, (2) under the AV conditions, performance declined as the amount of blurring increased, (3) even under the AV condition that simulated a visual acuity of 6/60, the speech recognition scores were significantly higher than those obtained under the A-alone condition, and (4) generally, younger adults obtained higher scores than older adults under all conditions. Our results demonstrate the benefits of visual cues to enhance speech understanding even when visual acuity is not optimal.

In auditory–visual synesthesia, sounds automatically elicit conscious and reliable visual experiences. It is presently unknown whether this reflects early or late processes in the brain. It is also unknown whether adult audiovisual synesthesia resembles auditory-induced visual illusions that can sometimes occur in the general population or whether it resembles the electrophysiological deflection over occipital sites that has been noted in infancy and has been likened to synesthesia. Electrical brain activity was recorded from adult synesthetes and control participants who were played brief tones and required to monitor for an infrequent auditory target. The synesthetes were instructed to attend either to the auditory or to the visual (i.e., synesthetic) dimension of the tone, whereas the controls attended to the auditory dimension alone. There were clear differences between synesthetes and controls that emerged early (100 msec after tone onset). These differences tended to lie in deflections of the auditory-evoked potential (e.g., the auditory N1, P2, and N2) rather than the presence of an additional posterior deflection. The differences occurred irrespective of what the synesthetes attended to (although attention had a late effect). The results suggest that differences between synesthetes and others occur early in time, and that synesthesia is qualitatively different from similar effects found in infants and certain auditory-induced visual illusions in adults. In addition, we report two novel cases of synesthesia in which colors elicit sounds, and vice versa.

Introduction. It is well established that the number of people diagnosed and suffering from depression is on the increase. Many of these patients are not responsive to first-line pharmacological intervention or simply cannot use medications for other reasons. As such, there has been a growing need for nonmedication approaches to treatment. The purpose of this study was to examine the use of auditory-visual EEG entrainment (AVE) at a 14 Hz (beta) frequency to decrease symptoms of depression with corresponding changes in neurophysiology. Method. Sixteen participants ranged in age from 20 to 67 years and were screened utilizing the Beck Depression Inventory–II (BDI–II) and broken into two groups of 8 (simulated, AVE treatment groups), with a cross-over design. Both groups were given the BDI–II and QEEG testing at baseline, 4 weeks following either AVE or simulated treatment, and then again after an additional 4 weeks and a switch in treatment in the cross-over design. Results. Results revealed signi...

Distractibility with auditory, visual, and bimodal stimulus changes was investigated using an audio-visual distraction paradigm. Participants were asked to discriminate between equiprobable short and long audio-visual stimuli. Infrequently, the auditory, the visual, or both parts of the stimuli changed. These rare deviations (deviants) were irrelevant for the actual task. The influence of the three types of deviant stimuli on the processing of task-relevant information was assessed with behavioral and event-related potential (ERP) measures assuming that bimodal deviants would lead to an increase in distraction. Behavioral and ERP results did not support this assumption, as reaction time (RT) prolongation and components amplitudes did not differ significantly for auditory and bimodal deviants. It is suggested that a maximal threshold of distraction accounts for these results. In addition, the processing of bimodal deviations was assessed. Audio-visual interactions were found following modality-specific deviance detection suggesting that integration only occurs with involuntary attention switching to task-irrelevant changes.

This study investigated whether ERPs from an inter-modal oddball task could distinguish between adults with Attention-Deficit/Hyperactivity Disorder (AD/HD) and controls. Two age-matched groups of young adult males (18 AD/HD, 18 controls) were presented with an inter-modal oddball task in which a counter-phasing checkerboard was the non-target visual stimulus (randomly presented on 80% of trials), and a 2000 Hz tone was the auditory target (20% of trials). Stimuli were presented at a fixed rate (stimulus-onset asynchrony 1.03 s) and participants were required to silently count all targets. The AD/HD group showed globally enhanced P2 and reduced N2 amplitudes to auditory targets, with no differences in target P3, together with topographic differences in N1 to auditory targets, and P1, N1, P2, N2 and P3 to visual non-targets, compared with controls. These results were interpreted in terms of early sensory-processing impairments in adults with AD/HD, which may be partially overcome through effortful processing, as reflected in the later endogenous ERP components.

The researchers evaluated the contribution of cochlear implants (CIs) to speech perception by a sample of prelingually deaf individuals implanted after age 8 years. This group was compared with a group with profound hearing impairment (HA-P), and with a group with severe hearing impairment (HA-S), both of which used hearing aids. Words and sentences were presented to the auditory channel alone, the visual channel alone, and the combined auditory-visual channel. Some of the results indicated better performance of the CI group than the HA-P group, thus indicating the advantage of CIs over hearing aids for "late" implantees, especially under difficult listening conditions. In addition, all participants relied on visual information under difficult auditory conditions. These last outcomes suggest that intervention with CI users should include exposure to visual as well as auditory information and should emphasize auditory-visual integration.

It is known that the brain adaptively recalibrates itself to small (approximately 100 ms) auditory-visual (AV) temporal asynchronies so as to maintain intersensory temporal coherence. Here we explored whether spatial disparity between a sound and light affects AV temporal recalibration. Participants were exposed to a train of asynchronous AV stimulus pairs (sound-first or light-first) with sounds and lights emanating from either the same or a different location. Following a short exposure phase, participants were tested on an AV temporal order judgement (TOJ) task. Temporal recalibration manifested itself as a shift of subjective simultaneity in the direction of the adapted audiovisual lag. The shift was equally big when exposure and test stimuli were presented from the same or different locations. These results provide strong evidence for the idea that spatial co-localisation is not a necessary constraint for intersensory pairing to occur.