Search

Autistic youth demonstrate differences in processing multisensory information, particularly in temporal processing of multisensory speech. Extensive research has identified several key brain regions for multisensory speech processing in non-autistic adults, including the superior temporal sulcus (STS) and insula, but it is unclear to what extent these regions are involved in temporal processing of multisensory speech in autistic youth. As a first step in exploring the neural substrates of multisensory temporal processing in this clinical population, we employed functional magnetic resonance imaging (fMRI) with a simultaneity-judgment audiovisual speech task. Eighteen autistic youth and a comparison group of 20 non-autistic youth matched on chronological age, biological sex, and gender participated. Results extend prior findings from studies of non-autistic adults, with non-autistic youth demonstrating responses in several similar regions as previously implicated in adult temporal processing of multisensory speech. Autistic youth demonstrated responses in fewer of the multisensory regions identified in adult studies; responses were limited to visual and motor cortices. Group responses in the middle temporal gyrus significantly interacted with age; younger autistic individuals showed reduced MTG responses whereas older individuals showed comparable MTG responses relative to non-autistic controls. Across groups, responses in the precuneus covaried with task accuracy, and anterior temporal and insula responses covaried with nonverbal IQ. These preliminary findings suggest possible differences in neural mechanisms of audiovisual processing in autistic youth while highlighting the need to consider participant characteristics in future, larger-scale studies exploring the neural basis of multisensory function in autism.

OBJECTIVE In drug-resistant temporal lobe epilepsy, automated tools for seizure onset zone (SOZ) localization that use brief interictal recordings could supplement presurgical evaluations and improve care. Thus, the authors sought to localize SOZs by training a multichannel convolutional neural network on stereoelectroencephalography (SEEG) cortico-cortical evoked potentials. METHODS The authors performed single-pulse electrical stimulation in 10 drug-resistant temporal lobe epilepsy patients implanted with SEEG. Using 500,000 unique poststimulation SEEG epochs, the authors trained a multichannel 1-dimensional convolutional neural network to determine whether an SOZ had been stimulated. RESULTS SOZs were classified with mean sensitivity of 78.1% and specificity of 74.6% according to leave-one-patient-out testing. To achieve maximum accuracy, the model required a 0- to 350-msec poststimulation time period. Post hoc analysis revealed that the model accurately classified unilateral versus bilateral mesial temporal lobe seizure onset, as well as neocortical SOZs. CONCLUSIONS This was the first demonstration, to the authors’ knowledge, that a deep learning framework can be used to accurately classify SOZs with single-pulse electrical stimulation–evoked responses. These findings suggest that accurate classification of SOZs relies on a complex temporal evolution of evoked responses within 350 msec of stimulation. Validation in a larger data set could provide a practical clinical tool for the presurgical evaluation of drug-resistant epilepsy.

Background Autism spectrum disorder (ASD) affects many aspects of life, from social interactions to (multi)sensory processing. Similarly, the condition expresses at a variety of levels of description, from genetics to neural circuits and interpersonal behavior. We attempt to bridge between domains and levels of description by detailing the behavioral, electrophysiological, and putative neural network basis of peripersonal space (PPS) updating in ASD during a social context, given that the encoding of this space relies on appropriate multisensory integration, is malleable by social context, and is thought to delineate the boundary between the self and others. Methods Fifty (20 male/30 female) young adults, either diagnosed with ASD or age- and sex-matched individuals, took part in a visuotactile reaction time task indexing PPS, while high-density electroencephalography was continuously recorded. Neural network modeling was performed in silico. Results Multisensory psychophysics demonstrates that while PPS in neurotypical individuals shrinks in the presence of others—as to “give space”—this does not occur in ASD. Likewise, electroencephalography recordings suggest that multisensory integration is altered by social context in neurotypical individuals but not in individuals with ASD. Finally, a biologically plausible neural network model shows, as a proof of principle, that PPS updating may be inflexible in ASD owing to the altered excitatory/inhibitory balance that characterizes neural circuits in animal models of ASD. Conclusions Findings are conceptually in line with recent statistical inference accounts, suggesting diminished flexibility in ASD, and further these observations by suggesting within an example relevant for social cognition that such inflexibility may be due to excitatory/inhibitory imbalances.

Visual cues are especially vital for hearing impaired individuals such as cochlear implant (CI) users to understand speech in noise. Functional Near Infrared Spectroscopy (fNIRS) is a light-based imaging technology that is ideally suited for measuring the brain activity of CI users due to its compatibility with both the ferromagnetic and electrical components of these implants. In a preliminary step toward better elucidating the behavioral and neural correlates of audiovisual (AV) speech integration in CI users, we designed a speech-in-noise task and measured the extent to which 24 normal hearing individuals could integrate the audio of spoken monosyllabic words with the corresponding visual signals of a female speaker. In our behavioral task, we found that audiovisual pairings provided average improvements of 103% and 197% over auditory-alone listening conditions in −6 and −9 dB signal-to-noise ratios consisting of multi-talker background noise. In an fNIRS task using similar stimuli, we measured activity during auditory-only listening, visual-only lipreading, and AV listening conditions. We identified cortical activity in all three conditions over regions of middle and superior temporal cortex typically associated with speech processing and audiovisual integration. In addition, three channels active during the lipreading condition showed uncorrected correlations associated with behavioral measures of audiovisual gain as well as with the McGurk effect. Further work focusing primarily on the regions of interest identified in this study could test how AV speech integration may differ for CI users who rely on this mechanism for daily communication.

Looking to the mouth of a talker early in life predicts expressive communication. We hypothesized that looking at a talker’s mouth may signal that infants are ready for increased supported joint engagement and that it subsequently facilitates prelinguistic vocal development and translates to broader gains in expressive communication. We tested this hypothesis in 50 infants aged 6–18 months with heightened and general population-level likelihood of autism diagnosis (Sibs-autism and Sibs-NA; respectively). We measured infants’ gaze to a speaker’s face using an eye tracking task, supported joint engagement during parent-child free play sessions, vocal complexity during a communication sample, and broader expressive communication. Looking at the mouth was indirectly associated with expressive communication via increased higher-order supported joint engagement and vocal complexity. This indirect effect did not vary according to sibling status. This study provides preliminary insights into the mechanisms by which looking at the mouth may influence expressive communication development.

Why are people with focal epilepsy not continuously having seizures? Previous neuronal signalling work has implicated gamma-aminobutyric acid balance as integral to seizure generation and termination, but is a high-level distributed brain network involved in suppressing seizures? Recent intracranial electrographic evidence has suggested that seizure-onset zones have increased inward connectivity that could be associated with interictal suppression of seizure activity. Accordingly, we hypothesize that seizure-onset zones are actively suppressed by the rest of the brain network during interictal states.Full testing of this hypothesis would require collaboration across multiple domains of neuroscience. We focused on partially testing this hypothesis at the electrographic network level within 81 individuals with drug-resistant focal epilepsy undergoing presurgical evaluation. We used intracranial electrographic resting-state and neurostimulation recordings to evaluate the network connectivity of seizure onset, early propagation and non-involved zones. We then used diffusion imaging to acquire estimates of white-matter connectivity to evaluate structure–function coupling effects on connectivity findings. Finally, we generated a resting-state classification model to assist clinicians in detecting seizure-onset and propagation zones without the need for multiple ictal recordings.Our findings indicate that seizure onset and early propagation zones demonstrate markedly increased inwards connectivity and decreased outwards connectivity using both resting-state (one-way ANOVA, P-value = 3.13 × 10−13) and neurostimulation analyses to evaluate evoked responses (one-way ANOVA, P-value = 2.5 × 10−3). When controlling for the distance between regions, the difference between inwards and outwards connectivity remained stable up to 80 mm between brain connections (two-way repeated measures ANOVA, group effect P-value of 2.6 × 10−12). Structure–function coupling analyses revealed that seizure-onset zones exhibit abnormally enhanced coupling (hypercoupling) of surrounding regions compared to presumably healthy tissue (two-way repeated measures ANOVA, interaction effect P-value of 9.76 × 10−21). Using these observations, our support vector classification models achieved a maximum held-out testing set accuracy of 92.0 ± 2.2% to classify early propagation and seizure-onset zones.These results suggest that seizure-onset zones are actively segregated and suppressed by a widespread brain network. Furthermore, this electrographically observed functional suppression is disproportionate to any observed structural connectivity alterations of the seizure-onset zones. These findings have implications for the identification of seizure-onset zones using only brief electrographic recordings to reduce patient morbidity and augment the presurgical evaluation of drug-resistant epilepsy. Further testing of the interictal suppression hypothesis can provide insight into potential new resective, ablative and neuromodulation approaches to improve surgical success rates in those suffering from drug-resistant focal epilepsy.

Children with autism show alterations in multisensory integration that have been theoretically and empirically linked with the core and related features of autism. It is unclear, however, to what extent multisensory integration maps onto features of autism within children with and without autism. This study, thus, evaluates relations between audiovisual integration and core and related autism features across children with and without autism. Thirty-six children reported perceptions of the McGurk illusion during a psychophysical task. Parents reported on participants' autistic features. Increased report of illusory percepts tended to covary with reduced autistic features and greater communication skill. Some relations, though, were moderated by group. This work suggests that associations between multisensory integration and higher-order skills are present, but in some instances vary according to diagnostic group.

Background Individuals on the autism spectrum are reported to display alterations in interoception, the sense of the internal state of the body. The Interoception Sensory Questionnaire (ISQ) is a 20-item self-report measure of interoception specifically intended to measure this construct in autistic people. The psychometrics of the ISQ, however, have not previously been evaluated in a large sample of autistic individuals. Methods Using confirmatory factor analysis, we evaluated the latent structure of the ISQ in a large online sample of adults on the autism spectrum and found that the unidimensional model fit the data poorly. Using misspecification analysis to identify areas of local misfit and item response theory to investigate the appropriateness of the seven-point response scale, we removed redundant items and collapsed the response options to put forth a novel eight-item, five-response choice ISQ. Results The revised, five-response choice ISQ (ISQ-8) showed much improved fit while maintaining high internal reliability. Differential item functioning (DIF) analyses indicated that the items of the ISQ-8 were answered in comparable ways by autistic adolescents and adults and across multiple other sociodemographic groups. Limitations Our results were limited by the fact that we did not collect data for typically developing controls, preventing the analysis of DIF by diagnostic status. Additionally, while this study proposes a new 5-response scale for the ISQ-8, our data were not collected using this method; thus, the psychometric properties for the revised version of this instrument require further investigation. Conclusion The ISQ-8 shows promise as a reliable and valid measure of interoception in adolescents and adults on the autism spectrum, but additional work is needed to examine its psychometrics in this population. A free online score calculator has been created to facilitate the use of ISQ-8 latent trait scores for further studies of autistic adolescents and adults (available at https://asdmeasures.shinyapps.io/ISQ_score/).

Differences in audiovisual integration are commonly observed in autism. Temporal binding windows (TBWs) of audiovisual speech can be trained (i.e., narrowed) in non-autistic adults; this study evaluated a computer-based perceptual training in autistic youth and assessed whether treatment outcomes varied according to individual characteristics. Thirty autistic youth aged 8-21 were randomly assigned to a brief perceptual training (n = 15) or a control condition (n = 15). At post-test, the perceptual training group did not differ, on average, on TBWs for trained and untrained stimuli and perception of the McGurk illusion compared to the control group. The training benefited youth with higher language and nonverbal IQ scores; the training caused widened TBWs in youth with co-occurring cognitive and language impairments.

Why are people with focal epilepsy not constantly seizing? Previous molecular work has implicated gamma-aminobutyric acid balance as integral to seizure generation and termination, but is the high-level distributed brain network involved in suppressing seizures? Recent intracranial electrographic evidence has suggested that seizure onset zones have an increased inward connectivity. Accordingly, we hypothesize that seizure onset zones are actively suppressed by the rest of the brain network during interictal states.We tested this hypothesis on 81 subjects with drug resistant focal epilepsy undergoing presurgical evaluation. We utilized intracranial electrographic resting-state and neurostimulation recordings to evaluate the network connectivity of seizure onset, propagative, and non-involved regions. We then utilized diffusion imaging to acquire estimates of white matter connectivity to evaluate structure-function coupling effects on connectivity findings. Finally, using our observations, we generated a resting-state classification model to assist clinicians in detecting seizure onset and propagative zones without the need for multiple ictal recordings.Our findings indicate that seizure onset and propagative zones demonstrate markedly increased inward connectivity and decreased outward connectivity on both resting-state and neurostimulation analyses. When controlling for distance between regions, the difference between inward vs. outward connectivity remained stable up to 80 mm between brain connections. Structure-function coupling analyses revealed that seizure onset zones exhibit abnormally enhanced coupling (hypercoupling) of surrounding regions compared to presumably healthy tissue. Using these observations, our classification models achieved a maximum held-out testing set accuracy of 92.0±2.2%.These results indicate that seizure onset zones are actively segregated and suppressed by a widespread brain network. Furthermore, this electrographically observed functional suppression is disproportionate to any observed structural connectivity alterations of the seizure onset zones. These findings have implications for the identification of seizure onset zones using only brief resting-sate recordings to reduce patient morbidity and augment the presurgical evaluation of drug resistant epilepsy. Furthermore, testing of the interictal suppression hypothesis can provide insight into potential new resective, ablative, and neuromodulation approaches to improve surgical success rates in those suffering from drug resistant focal epilepsy.

In a naturalistic environment, auditory cues are often accompanied by information from other senses, which can be redundant with or complementary to the auditory information. Although the multisensory interactions derived from this combination of information and that shape auditory function are seen across all sensory modalities, our greatest body of knowledge to date centers on how vision influences audition. In this review, we attempt to capture the state of our understanding at this point in time regarding this topic. Following a general introduction, the review is divided into 5 sections. In the first section, we review the psychophysical evidence in humans regarding vision’s influence in audition, making the distinction between vision’s ability to enhance versus alter auditory performance and perception. Three examples are then described that serve to highlight vision’s ability to modulate auditory processes: spatial ventriloquism, cross-modal dynamic capture, and the McGurk effect. The final part of this section discusses models that have been built based on available psychophysical data and that seek to provide greater mechanistic insights into how vision can impact audition. The second section reviews the extant neuroimaging and far-field imaging work on this topic, with a strong emphasis on the roles of feedforward and feedback processes, on imaging insights into the causal nature of audiovisual interactions, and on the limitations of current imaging-based approaches. These limitations point to a greater need for machine-learning-based decoding approaches toward understanding how auditory representations are shaped by vision. The third section reviews the wealth of neuroanatomical and neurophysiological data from animal models that highlights audiovisual interactions at the neuronal and circuit level in both subcortical and cortical structures. It also speaks to the functional significance of audiovisual interactions for two critically important facets of auditory perception—scene analysis and communication. The fourth section presents current evidence for alterations in audiovisual processes in three clinical conditions: autism, schizophrenia, and sensorineural hearing loss. These changes in audiovisual interactions are postulated to have cascading effects on higher-order domains of dysfunction in these conditions. The final section highlights ongoing work seeking to leverage our knowledge of audiovisual interactions to develop better remediation approaches to these sensory-based disorders, founded in concepts of perceptual plasticity in which vision has been shown to have the capacity to facilitate auditory learning.

In drug resistant temporal lobe epilepsy, automated tools for seizure onset zone (SOZ) localization using brief interictal recordings would supplement presurgical evaluations and improve care. Thus, we sought to localize SOZs by training a multi-channel convolutional neural network on stereo-EEG (SEEG) cortico-cortical evoked potentials. We performed single pulse electrical stimulation with 10 drug resistant temporal lobe epilepsy patients implanted with SEEG. Using the 500,000 unique post-stimulation SEEG epochs, we trained a multi-channel one-dimensional convolutional neural network to determine whether an SOZ was stimulated. SOZs were classified with a mean leave-one-patient-out testing sensitivity of 78.1% and specificity of 74.6%. To achieve maximum accuracy, the model requires a 0-350 ms post stimulation time period. Post-hoc analysis revealed that the model accurately classified unilateral vs bilateral mesial temporal lobe seizure onset, and neocortical SOZs. This is the first demonstration, to our knowledge, that a deep learning framework can be used to accurately classify SOZs using cortico-cortical evoked potentials. Our findings suggest accurate classification of SOZs relies on a complex temporal evolution of evoked potentials within 350 ms of stimulation. Validation in a larger dataset could provide a practical clinical tool for the presurgical evaluation of drug resistant epilepsy.

Background Sensory hypersensitivity, defined as heightened awareness of and reactivity to external stimuli, is a bothersome symptom that affects up to 80% of adults with Tourette syndrome (TS). Such widespread prevalence suggests sensory hypersensitivity is a core feature of the disorder, but its severity and association with other clinical features of TS remain largely unexplored. Complicating matters, sensory hypersensitivity has been observed in two neurodevelopmental disorders commonly comorbid with TS: obsessive-compulsive disorder (OCD) and attention deficit hyperactivity disorder (ADHD). Objective We sought to measure sensory hypersensitivity in TS patients relative to healthy controls and to investigate the relationship of sensory hypersensitivity with OCD and ADHD symptoms in the context of TS. Methods We recruited 34 adults with TS or chronic tic disorder to undergo evaluation with the Yale Global Tic Severity Scale (YGTSS) and a battery of validated self-report instruments assessing sensory hypersensitivity (Sensory Gating Inventory, SGI; Sensory Perception Quotient, SPQ), premonitory urge (Premonitory Urge to Tic Scale, PUTS), OCD (Dimensional Obsessive-Compulsive Scale, DOCS), and ADHD (Adult ADHD Self-Report Screening Scale for DSM-5, ASRS-V). Age- and sex-matched healthy controls were recruited to complete SGI and psychiatric measures. Results SGI and SPQ scores strongly correlated (rs = -0.73, p < 0.0001) within patients. SGI total score was significantly higher in patients versus controls (119.0 vs 67.6, U =-5.3, p < 0.0001), indicating greater sensory hypersensitivity in the tic disorder group. SGI score correlated modestly with PUTS, DOCS, and ASRS-V scores but not with YGTSS total tic score. Hierarchical linear regression analysis revealed that, of the tested variables, only DOCS score contributed significantly to mean SGI score, with β ranging from 1.03 (p = 0.044) to 1.41 (p = 0.001). A simple linear regression model with DOCS as the independent variable accounted for 31.9% of SGI score variance. Conclusion Sensory hypersensitivity is prominent in adults with tic disorder and is independently associated with obsessive-compulsive symptom severity.

Abnormal sensory responses are a DSM-5 symptom of autism spectrum disorder (ASD), and research findings demonstrate altered sensory processing in ASD. Beyond difficulties with processing information within single sensory domains, including both hypersensitivity and hyposensitivity, difficulties in multisensory processing are becoming a core issue of focus in ASD. These difficulties may be targeted by treatment approaches such as "sensory integration," which is frequently applied in autism treatment but not yet based on clear evidence. Recently, psychophysical data have emerged to demonstrate multisensory deficits in some children with ASD. Unlike deficits in social communication, which are best understood in humans, sensory and multisensory changes offer a tractable marker of circuit dysfunction that is more easily translated into animal model systems to probe the underlying neurobiological mechanisms. Paralleling experimental paradigms that were previously applied in humans and larger mammals, we and others have demonstrated that multisensory function can also be examined behaviorally in rodents. Here, we review the sensory and multisensory difficulties commonly found in ASD, examining laboratory findings that relate these findings across species. Next, we discuss the known neurobiology of multisensory integration, drawing largely on experimental work in larger mammals, and extensions of these paradigms into rodents. Finally, we describe emerging investigations into multisensory processing in genetic mouse models related to autism risk. By detailing findings from humans to mice, we highlight the advantage of multisensory paradigms that can be easily translated across species, as well as the potential for rodent experimental systems to reveal opportunities for novel treatments. LAY SUMMARY: Sensory and multisensory deficits are commonly found in ASD and may result in cascading effects that impact social communication. By using similar experiments to those in humans, we discuss how studies in animal models may allow an understanding of the brain mechanisms that underlie difficulties in multisensory integration, with the ultimate goal of developing new treatments. Autism Res 2020, 13: 1430-1449. © 2020 International Society for Autism Research, Wiley Periodicals, Inc.

Autism Spectrum Disorder (ASD) is a highly heterogeneous neurodevelopmental disorder characterized by deficits in social communication and by patterns of restricted interests and/or repetitive behaviors. The Simons Foundation Autism Research Initiative’s Human Gene and CNV Modules now list over 1,000 genes implicated in ASD and over 2,000 copy number variant loci reported in individuals with ASD. Given this ever-growing list of genetic changes associated with ASD, it has become evident that there is likely not a single genetic cause of this disorder nor a single neurobiological basis of this disorder. Instead, it is likely that many different neurobiological perturbations (which may represent subtypes of ASD) can result in the set of behavioral symptoms that we called ASD. One such of possible subtype of ASD may be associated with dopamine dysfunction. Precise regulation of synaptic dopamine (DA) is required for reward processing and behavioral learning, behaviors which are disrupted in ASD. Here we review evidence for DA dysfunction in ASD and in animal models of ASD. Further, we propose that these studies provide a scaffold for scientists and clinicians to consider subcategorizing the ASD diagnosis based on the genetic changes, neurobiological difference, and behavioral features identified in individuals with ASD.

Humans are able to recognize objects under a variety of noisy conditions, so models of the human visual system must account for how this feat is accomplished. In this study, we investigated how image perturbations, specifically reducing images to their low spatial frequency (LSF) components, affected correspondence between convolutional neural networks (CNNs) and brain signals recorded using magnetoencephalography (MEG). Using the high temporal resolution of MEG, we found that CNN-Brain correspondence for deeper and more complex layers across CNN architectures emerged earlier for LSF images than for their unfiltered broadband counterparts. The early emergence of LSF components is consistent with the coarse-to-fine theoretical framework for visual image processing, but surprisingly shows that LSF signals from images are more prominent when high spatial frequencies are removed. In addition, we decomposed MEG signals into oscillatory components and found correspondence varied based on frequency bands, painting a full picture of how CNN-Brain correspondence varies with time, frequency, and MEG sensor locations. Finally, we varied image properties of CNN training sets, and found marked changes in CNN processing dynamics and correspondence to brain activity. In sum, we show that image perturbations affect CNN-Brain correspondence in unexpected ways, as well as provide a rich methodological framework for assessing CNN-Brain correspondence across space, time, and frequency.

It has been challenging to elucidate the differences in brain structure that underlie behavioral features of autism. Prior studies have begun to identify patterns of changes in autism across multiple structural indices, including cortical thickness, local gyrification, and sulcal depth. However, common approaches to local gyrification indexing used in prior studies have been limited by low spatial resolution relative to functional brain topography. In this study, we analyze the aforementioned structural indices, utilizing a new method of local gyrification indexing that quantifies this index adaptively in relation to specific sulci/gyri, improving interpretation with respect to functional organization. Our sample included n = 115 autistic and n = 254 neurotypical participants aged 5–54, and we investigated structural patterns by group, age, and autism-related behaviors. Differing structural patterns by group emerged in many regions, with age moderating group differences particularly in frontal and limbic regions. There were also several regions, particularly in sensory areas, in which one or more of the structural indices of interest either positively or negatively covaried with autism-related behaviors. Given the advantages of this approach, future studies may benefit from its application in hypothesis-driven examinations of specific brain regions and/or longitudinal studies to assess brain development in autism.

Most evidence on the neural and perceptual correlates of sensory processing derives from studies that have focused on only a single sensory modality and averaged the data from groups of participants. Although valuable, such studies ignore the substantial interindividual and intraindividual differences that are undoubtedly at play. Such variability plays an integral role in both the behavioral/perceptual realms and in the neural correlates of these processes, but substantially less is known when compared with group-averaged data. Recently, it has been shown that the presentation of stimuli from two or more sensory modalities (i.e., multisensory stimulation) not only results in the well-established performance gains but also gives rise to reductions in behavioral and neural response variability. To better understand the relationship between neural and behavioral response variability under multisensory conditions, this study investigated both behavior and brain activity in a task requiring participants to discriminate moving versus static stimuli presented in either a unisensory or multisensory context. EEG data were analyzed with respect to intraindividual and interindividual differences in RTs. The results showed that trial-by-trial variability of RTs was significantly reduced under audiovisual presentation conditions as compared with visual-only presentations across all participants. Intraindividual variability of RTs was linked to changes in correlated activity between clusters within an occipital to frontal network. In addition, interindividual variability of RTs was linked to differential recruitment of medial frontal cortices. The present findings highlight differences in the brain networks that support behavioral benefits during unisensory versus multisensory motion detection and provide an important view into the functional dynamics within neuronal networks underpinning intraindividual performance differences.

The integration of information across distinct modalities enhances perceptual abilities. An ecologically important role of multisensory integration is in scaffolding verbal communication, which relies upon the precise temporal integration of auditory and visual cues. However, the role of (multi)sensory function in supporting another important aspect of communication, namely, nonverbal communication, is unknown. Here, individuals with autism spectrum disorder (ASD) and a group of typically developing (TD) participants performed a simultaneity judgment task to index their audiovisual temporal acuity for speech stimuli. Further, under a naturalistic scenario, nonverbal synchrony between the participant and a naive experimenter was measured. Automated motion analysis was performed to quantify movements of different body-parts. Results demonstrate a wider window of audiovisual temporal integration for ASD participants in comparison to their TD counterparts. Moreover, ASD individuals performed less complex movements and demonstrated less nonverbal synchrony during the interactive exchange. Lastly, multisensory temporal acuity significantly predicted the synchrony in hand and head movements between TD participants and the experimenter, but not between the ASD participants and the experimenter. Taken together, the results suggest an important role for multisensory perceptual abilities in shaping nonverbal communication between dyads and highlight the important role of perceptual systems in supporting social interactive skills.

An ever-growing literature has aimed to determine how individuals with autism spectrum disorder (ASD) differ from their typically developing (TD) peers on measures of multisensory integration (MSI) and to ascertain the degree to which differences in MSI are associated with the broad range of symptoms associated with ASD. Findings, however, have been highly variable across the studies carried out to date. The present work systematically reviews and quantitatively synthesizes the large literature on audiovisual MSI in individuals with ASD to evaluate the cumulative evidence for (a) group differences between individuals with ASD and TD peers, (b) correlations between MSI and autism symptoms in individuals with ASD and (c) study level factors that may moderate findings (i.e., explain differential effects) observed across studies. To identify eligible studies, a comprehensive search strategy was employed using the ProQuest search engine, PubMed database, forwards and backwards citation searches, direct author contact, and hand-searching of select conference proceedings. A significant between-group difference in MSI was evident in the literature, with individuals with ASD demonstrating worse audiovisual integration on average across studies compared to TD controls. This effect was moderated by mean participant age, such that between-group differences were more pronounced in younger samples. The mean correlation between MSI and autism and related symptomatology was also significant, indicating that increased audiovisual integration in individuals with ASD is associated with better language/communication abilities and/or reduced autism symptom severity in the extant literature. This effect was moderated by whether the stimuli were linguistic versus non-linguistic in nature, such that correlation magnitudes tended to be significantly greater when linguistic stimuli were utilized in the measure of MSI. Limitations and future directions for primary and meta-analytic research are discussed.

The reduction in spectral resolution by cochlear implants oftentimes requires complementary visual speech cues to aid in understanding. Despite substantial clinical characterization of auditory-only speech outcome measures, relatively little is known about the audiovisual integrative abilities that most cochlear implant (CI) users rely on for daily speech comprehension. In this study, we tested audiovisual integration in 63 CI users and 69 normal-hearing (NH) controls using the McGurk and sound-induced flash illusions. This study is the largest to-date measuring the McGurk effect in this population and the first to test the sound-induced flash illusion. When presented with conflicting audiovisual speech stimuli (i.e., the phoneme “ba” dubbed onto the viseme “ga”), we found that 55 CI users (87%) reported a fused percept of “da” or “tha” on at least one trial. However, overall, we found that CI users experienced the McGurk effect less often than controls—a result that was concordant with results with the sound-induced flash illusion where the pairing of a single circle flashing on the screen with multiple beeps resulted in fewer illusory flashes for CI users. While illusion perception in these two tasks appears to be uncorrelated among CI users, we identified a negative correlation in the NH group. Because neither illusion appears to provide further explanation of variability in CI outcome measures, further research is needed to determine how these findings relate to CI users’ speech understanding, particularly in ecological listening conditions that are naturally multisensory.

Autism spectrum disorder (ASD) is a heterogenous disorder predominantly characterized by social and communicative differences, but increasingly recognized to also alter (multi)sensory function. To face the heterogeneity and ubiquity of ASD, researchers have proposed models of statistical inference operating at the level of ‘computations’. Here, we attempt to bridge both across domains – from social to sensory – and levels of description – from behavioral computations to neural ensemble activity to a biologically-plausible artificial neural network – in furthering our understanding of autism. We do so by mapping visuo-tactile peri-personal space (PPS), and examining its electroencephalography (EEG) correlates, in individuals with ASD and neurotypical individuals during both a social and non-social context given that (i) the sensory coding of PPS is well understood, (ii) this space is thought to distinguish between self and other, and (iii) PPS is known to remap during social interactions. In contrast to their neurotypical counterparts, psychophysical and EEG evidence suggested that PPS does not remap in ASD during a social context. To account for this observation, we then employed a neural network model of PPS and demonstrate that PPS remapping may be driven by changes in neural gain operating at the level of multisensory neurons. Critically, under the anomalous excitation-inhibition (E/I) regime of ASD, this gain modulation does not result in PPS resizing. Overall, our findings are in line with recent statistical inference accounts suggesting diminished flexibility in ASD, and further these accounts by demonstrating within an example relevant for social cognition that such inflexibility may be due to E/I imbalances.

Autism is a neurodevelopmental condition defined by differences in social communication and by the presence of restricted and repetitive patterns of behavior, interests, and activities (RRBs). Individuals with autism also commonly present with atypical patterns of sensory responsiveness (i.e., hyporesponsiveness, hyperresponsiveness, and sensory seeking), which are theorized to produce cascading effects across other domains of development. The purpose of this study was to examine differences in sensory responsiveness in children with and without autism (ages 8&ndash, 18 years), as well as relations between patterns of sensory responsiveness and core and related features of autism. Participants were 50 children with autism and 50 non-autistic peers matched on age and sex. A comprehensive clinical battery included multiple measures of sensory responsiveness, core features of autism, adaptive behavior, internalizing behaviors, cognitive ability, and language ability. Groups significantly differed on all three patterns of sensory responsiveness. Some indices of core and related autism features were robustly associated with all three patterns of sensory responsiveness (e.g., RRBs), while others were more strongly associated with discrete patterns of sensory responsiveness (i.e., internalizing problem behaviors and hyperresponsiveness, language and sensory seeking). This study extends prior work to show that differences in sensory responsiveness that are linked with core and related features of autism persist in older children and adolescents on the spectrum.

The human brain retains a striking degree of plasticity into adulthood. Recent studies have demonstrated that a short period of altered visual experience (via monocular deprivation) can change the dynamics of binocular rivalry in favor of the deprived eye, a compensatory action thought to be mediated by an upregulation of cortical gain control mechanisms. Here, we sought to better understand the impact of monocular deprivation on multisensory abilities, specifically examining audiovisual temporal perception. Using an audiovisual simultaneity judgment task, we discovered that 90 minutes of monocular deprivation produced opposing effects on the temporal binding window depending on the eye used in the task. Thus, in those who performed the task with their deprived eye there was a narrowing of the temporal binding window, whereas in those performing the task with their nondeprived eye there was a widening of the temporal binding window. The effect was short lived, being observed only in the first 10 minutes of postdeprivation testing. These findings indicate that changes in visual experience in the adult can rapidly impact multisensory perceptual processes, a finding that has important clinical implications for those patients with adult-onset visual deprivation and for therapies founded on monocular deprivation.

Background Sensory hypersensitivity, defined as heightened awareness of and reactivity to external stimuli, is a bothersome symptom that affects up to 80% of adults with Tourette syndrome (TS). Such widespread prevalence suggests sensory hypersensitivity is a core feature of the disorder, but its severity and association with other clinical features of TS remain largely unexplored. Complicating matters, sensory hypersensitivity has been observed in two neurodevelopmental disorders commonly comorbid with TS: obsessive-compulsive disorder (OCD) and attention deficit hyperactivity disorder (ADHD). Objective We sought to measure sensory hypersensitivity in TS patients relative to healthy controls and to investigate the relationship of sensory hypersensitivity with OCD and ADHD symptoms in the context of TS. Methods We recruited 34 adults with TS or chronic tic disorder to undergo evaluation with the Yale Global Tic Severity Scale (YGTSS) and a battery of validated self-report instruments assessing sensory hypersensitivity (Sensory Gating Inventory, SGI; Sensory Perception Quotient, SPQ), premonitory urge (Premonitory Urge to Tic Scale, PUTS), OCD (Dimensional Obsessive-Compulsive Scale, DOCS), and ADHD (Adult ADHD Self-Report Screening Scale for DSM-5, ASRS-V). Age- and sex-matched healthy controls were recruited to complete SGI and psychiatric measures. Results SGI and SPQ scores strongly correlated (rs = −0.73, p < 0.0001) within patients. SGI total score was significantly higher in patients versus controls (119.0 vs 67.6, U =−5.3, p < 0.0001), indicating greater sensory hypersensitivity in the tic disorder group. SGI score correlated modestly with PUTS, DOCS, and ASRS-V scores but not with YGTSS total tic score. Hierarchical linear regression analysis revealed that, of the tested variables, only DOCS score contributed significantly to mean SGI score, with β ranging from 1.03 (p = 0.044) to 1.41 (p = 0.001). A simple linear regression model with DOCS as the independent variable accounted for 31.9% of SGI score variance. Conclusion Sensory hypersensitivity is prominent in adults with tic disorder and is independently associated with obsessive-compulsive symptom severity.

Multisensory processes include the capacity to combine information from the different senses, often improving stimulus representations and behavior. The extent to which multisensory processes are an innate capacity or instead require experience with environmental stimuli remains debated. We addressed this knowledge gap by studying multisensory processes in prematurely born and full-term infants. We recorded 128-channel event-related potentials (ERPs) from a cohort of 55 full-term and 61 preterm neonates (at an equivalent gestational age) in response to auditory, somatosensory, and combined auditory-somatosensory multisensory stimuli. Data were analyzed within an electrical neuroimaging framework, involving unsupervised topographic clustering of the ERP data. Multisensory processing in full-term infants was characterized by a simple linear summation of responses to auditory and somatosensory stimuli alone, which furthermore shared common ERP topographic features. We refer to the ERP topography observed in full-term infants as “typical infantile processing” (TIP). In stark contrast, preterm infants exhibited non-linear responses and topographies less-often characterized by TIP; there were distinct patterns of ERP topographies to multisensory and summed unisensory conditions. We further observed that the better TIP characterized an infant’s ERPs, independently of prematurity, the more typical was the score on the Infant/Toddler Sensory Profile (ITSP) at 12 months of age and the less likely was the child to the show internalizing tendencies at 24 months of age. Collectively, these results highlight striking differences in the brain’s responses to multisensory stimuli in children born prematurely; differences that relate to later sensory and internalizing functions. Electronic supplementary material The online version of this article (10.1007/s10548-020-00791-4) contains supplementary material, which is available to authorized users.

The brain is comprised of neural circuits that are able to flexibly represent the complexity of the external world. In accomplishing this feat, one of the first attributes the brain must code for is whether a stimulus is present and subsequently what sensory information that stimulus contains. One of the core characteristics of that information is which sensory modality(ies) are being represented. How information regarding both the presence and modal identity of a given stimulus is represented and transformed within the brain remains poorly understood. In this study, we investigated how the brain represents the presence and modal identity of a given stimulus while tactile, audio, and audio-tactile stimuli were passively presented to non-human primates. We recorded spiking activity from primary somatosensory (S1) and ventral pre-motor (PMv) cortices, two areas known to be instrumental in transforming sensory information into motor commands for action. Using multivariate analyses to decode stimulus presence and identity, we found that information regarding stimulus presence and modal identity were found in both S1 and PMv and extended beyond the duration of significant evoked spiking activity, and that this information followed different time-courses in these two areas. Further, we combined time-generalization decoding with cross-area decoding to demonstrate that while signaling the presence of a stimulus involves a feedforward-feedback coupling between S1-PMv, the processing of modal identity is largely restricted to S1. Together, these results highlight the differing spatiotemporal dynamics of information flow regarding stimulus presence and modal identity in two nodes of an important cortical sensorimotor circuit.Significance StatementIt is unclear how the structure and function of the brain support differing sensory functions, such as detecting the presence of a stimulus in the environment vs. identifying it. Here, we used multivariate decoding methods on monkey neuronal data to track how information regarding stimulus presence and modal identity flow within a sensorimotor circuit. Results demonstrate that while neural patterns in both primary somatosensory (S1) and ventral pre-motor (PMv) cortices can be used to detect and discriminate between stimuli, they follow different time-courses. Importantly, findings suggest that while information regarding the presence of a stimulus flows reciprocally between S1 and PMv, information regarding stimulus identity is largely contained in S1.

Everyday environments, such as classrooms or the High Street, typically stimulate multiple senses at once (i.e., are multisensory). Evidence from the last 40 years has characterized how multisensory perception unfolds and manifests neurophysiologically. However, a large majority of this research has focused on the effects of multisensory processes on instantaneous perception. This omission is important inasmuch as learning in multisensory settings is a rule rather than an exception. Furthermore, it is increasingly recognized that object representations are inherently multisensory in nature. This chapter reviews efforts to understand when and how multisensory information improves learning and memory, particularly in the context of single-trial encoding that emulates real-world settings. This chapter bridges multisensory findings and the more traditional, unisensory literature on object representations and provides a tentative roadmap for more scientifically based utilization of multisensory processes toward improving two crucial domains of everyday life, i.e., education and rehabilitation of sensory and cognitive functions.

Multisensory processing changes over the course of both typical and atypical human development. The complex behavioral profiles seen in clinical groups with altered development, such as autism and schizophrenia, may in part reflect differences in sensory processing. Both groups exhibit social deficits that unfold with a characteristic developmental course, alongside other symptoms. Current research converges on the presence of multisensory processing deficits that, in both groups, may become more pronounced as stimuli become more complex. Often impaired multisensory processing accompanies intact unisensory processing. In the auditory and visual domains, these deficits are linked to deficits in verbal communication, while integration between these senses and more proximal somatic senses (touch, proprioception, interoception) may be important for more nonverbal social-communicative development such as self–other distinction and perspective-taking. Animal models of both disorders also provide evidence for deficits in multisensory integration in the physiology of integrative cortex and performance on behavioral tasks of multisensory processing. Effective, neuroscience-informed behavioral treatments for both groups are still lacking and none address the growing consensus that higher-order symptoms cascade from more basic perceptual differences that play out over the course of development. Multisensory behavioral neuroscience has an important role to play in addressing this translational challenge: to sustain and expand rigorous multidisciplinary investigation of sensory integration by incorporating basic science, human perception, developmental, and clinical perspectives to understand and impact the role of altered multisensory integration in these disorders.

Research shows that children with autism spectrum disorder (ASD) differ in their behavioral patterns of responding to sensory stimuli (i.e., sensory responsiveness) and in various other aspects of sensory functioning relative to typical peers. This study explored relations between measures of sensory responsiveness and multisensory speech perception and integration in children with and without ASD. Participants were 8-17 year old children, 18 with ASD and 18 matched typically developing controls. Participants completed a psychophysical speech perception task, and parents reported on children's sensory responsiveness. Psychophysical measures (e.g., audiovisual accuracy, temporal binding window) were associated with patterns of sensory responsiveness (e.g., hyporesponsiveness, sensory seeking). Results indicate that differences in multisensory speech perception and integration covary with atypical patterns of sensory responsiveness.

Self-perception is scaffolded upon the integration of multisensory cues on the body, the space surrounding the body (i.e., the peri-personal space; PPS), and from within the body. We asked whether reducing information available from external space would change: PPS, interoceptive accuracy, and self-experience. Twenty participants were exposed to 15 min of audio-visual deprivation and performed: (i) a visuo-tactile interaction task measuring their PPS; (ii) a heartbeat perception task measuring interoceptive accuracy; and (iii) a series of questionnaires related to self-perception and mental illness. These tasks were carried out in two conditions: while exposed to a standard sensory environment and under a condition of audio-visual deprivation. Results suggest that while PPS becomes ill defined after audio-visual deprivation, interoceptive accuracy is unaltered at a group-level, with some participants improving and some worsening in interoceptive accuracy. Interestingly, correlational individual differences analyses revealed that changes in PPS after audio-visual deprivation were related to interoceptive accuracy and self-reports of “unusual experiences” on an individual subject basis. Taken together, the findings argue for a relationship between the malleability of PPS, interoceptive accuracy, and an inclination toward aberrant ideation often associated with mental illness.

Binding across sensory modalities yields substantial perceptual benefits, including enhanced speech intelligibility. The coincidence of sensory inputs across time is a fundamental cue for this integration process. Recent work has suggested that individuals with diagnoses of schizophrenia (SZ) and autism spectrum disorder (ASD) will characterize auditory and visual events as synchronous over larger temporal disparities than their neurotypical counterparts. Namely, these clinical populations possess an enlarged temporal binding window (TBW). Although patients with SZ and ASD share aspects of their symptomatology, phenotypic similarities may result from distinct etiologies. To examine similarities and variances in audiovisual temporal function in these two populations, individuals diagnosed with ASD (n = 46; controls n = 40) and SZ (n = 16, controls = 16) completed an audiovisual simultaneity judgment task. In addition to standard psychometric analyses, synchrony judgments were assessed using Bayesian causal inference modeling. This approach permits distinguishing between distinct causes of an enlarged TBW: an a priori bias to bind sensory information and poor fidelity in the sensory representation. Findings indicate that both ASD and SZ populations show deficits in multisensory temporal acuity. Importantly, results suggest that while the wider TBWs in ASD most prominently results from atypical priors, the wider TBWs in SZ results from a trend toward changes in prior and weaknesses in the sensory representations. Results are discussed in light of current ASD and SZ theories and highlight that different perceptual training paradigms focused on improving multisensory integration may be most effective in these two clinical populations and emphasize that similar phenotypes may emanate from distinct mechanistic causes.

Multisensory integration of visual mouth movements with auditory speech is known to offer substantial perceptual benefits, particularly under challenging (i.e., noisy) acoustic conditions. Previous work characterizing this process has found that ERPs to auditory speech are of shorter latency and smaller magnitude in the presence of visual speech. We sought to determine the dependency of these effects on the temporal relationship between the auditory and visual speech streams using EEG. We found that reductions in ERP latency and suppression of ERP amplitude are maximal when the visual signal precedes the auditory signal by a small interval and that increasing amounts of asynchrony reduce these effects in a continuous manner. Time–frequency analysis revealed that these effects are found primarily in the theta (4–8 Hz) and alpha (8–12 Hz) bands, with a central topography consistent with auditory generators. Theta effects also persisted in the lower portion of the band (3.5–5 Hz), and this late activity was more frontally distributed. Importantly, the magnitude of these late theta oscillations not only differed with the temporal characteristics of the stimuli but also served to predict participants' task performance. Our analysis thus reveals that suppression of single-trial brain responses by visual speech depends strongly on the temporal concordance of the auditory and visual inputs. It further illustrates that processes in the lower theta band, which we suggest as an index of incongruity processing, might serve to reflect the neural correlates of individual differences in multisensory temporal perception.

Sensory symptoms, including auditory processing deficits, are common in autism spectrum disorder (ASD). Processing of temporal aspects of auditory input is understudied; yet, deficits in this domain could contribute to language-related impairments. In children with ASD and well-matched controls, this study examined electrophysiological response to silent gaps in auditory stimuli. Results revealed attenuated amplitude of the P2 event-related potential (ERP) component in ASD. The P2 amplitude reduction was also associated with sensory, language, and diagnostic features. These results suggest that neural response during auditory gap detection is a promising ASD biomarker that could be useful for stratifying subgroups and evaluating treatment response.

The temporal relationship between individual pieces of information from the different sensory modalities is one of the stronger cues to integrate such information into a unified perceptual gestalt, conveying numerous perceptual and behavioral advantages. Temporal acuity, however, varies greatly over the life span. It has previously been hypothesized that changes in temporal acuity in both development and healthy aging may thus play a key role in integrative abilities. This study tested the temporal acuity of 138 individuals ranging in age from 5 to 80. Temporal acuity and multisensory integration abilities were tested both within and across modalities (audition and vision) with simultaneity judgment and temporal order judgment tasks. We observed that temporal acuity, both within and across modalities, improved throughout development into adulthood and subsequently declined with healthy aging, as did the ability to integrate multisensory speech information. Of importance, throughout development, temporal acuity of simple stimuli (i.e., flashes and beeps) predicted individuals' abilities to integrate more complex speech information. However, in the aging population, although temporal acuity declined with healthy aging and was accompanied by declines in integrative abilities, temporal acuity was not able to predict integration at the individual level. Together, these results suggest that the impact of temporal acuity on multisensory integration varies throughout the life span. Although the maturation of temporal acuity drives the rise of multisensory integrative abilities during development, it is unable to account for changes in integrative abilities in healthy aging. The differential relationships between age, temporal acuity, and multisensory integration suggest an important role for experience in these processes. (PsycINFO Database Record