Search

To fluidly engage with the world, our brains must simultaneously represent both the scene in front of us and our memory of the immediate surrounding environment (i.e., local visuospatial context). How does the brain’s functional architecture enable sensory and mnemonic representations to closely interface, while also avoiding sensory-mnemonic interference? Here, we asked this question using first-person, head-mounted virtual reality (VR) and fMRI. Using VR, human participants of both sexes learned a set of immersive, real-world visuospatial environments in which we systematically manipulated the extent of visuospatial context associated with a scene image in memory across three learning conditions, spanning from a single field-of-view to a city street. We used fine-grained individual subject fMRI to determine which brain areas support memory of the visuospatial context associated with a scene during recall (Exp. 1) and recognition (Exp. 2). Across the whole brain, activity in three patches of cortex scaled with the amount of known visuospatial context, each located immediately anterior to one of the three scene perception areas of high-level visual cortex. Individual subject analyses revealed that these anterior patches corresponded to three functionally-defined place memory areas, which selectively respond when visually recalling personally familiar places. In addition to showing activity levels that scaled with the amount of visuospatial context, multivariate analyses showed that these anterior areas represented the identity of the specific environment being recalled. Together, these results suggest a convergence zone for scene perception and memory of the local visuospatial context at the anterior edge of high-level visual cortex.Significance statementAs we move through the world, the visual scene around us is integrated with our memory of the wider visuospatial context. Here, we sought to understand how the functional architecture of the brain enables coexisting representations of the current visual scene and memory of the surrounding environment. Using a combination of immersive virtual reality and fMRI, we show that memory of visuospatial context outside the current field-of-view is represented in a distinct set of brain areas immediately anterior adjacent to the perceptually-oriented scene-selective areas of high-level visual cortex. This functional architecture would allow efficient interaction between immediately adjacent mnemonic and perceptual areas, while also minimizing mnemonic-perceptual interference.

Individuals with autism spectrum conditions (ASC) describe differences in both social cognition and sensory processing, but little is known about the causal relationship between these disparate functional domains. In the present study, we sought to understand how a core characteristic of autism-reduced social attention-is impacted by the complex multisensory signals present in real-world environments. We tested the hypothesis that reductions in social attention associated with autism would be magnified by increasing perceptual load (e.g., motion, multisensory cues). Adult participants (N = 40; 19 ASC) explored a diverse set of 360° real-world scenes in a naturalistic, active viewing paradigm (immersive virtual reality + eyetracking). Across three conditions, we systematically varied perceptual load while holding the social and semantic information present in each scene constant. We demonstrate that reduced social attention is not a static signature of the autistic phenotype. Rather, group differences in social attention emerged with increasing perceptual load in naturalistic environments, and the susceptibility of social attention to perceptual load predicted continuous measures of autistic traits across groups. Crucially, this pattern was specific to the social domain: we did not observe differential impacts of perceptual load on attention directed toward nonsocial semantic (i.e., object, place) information or low-level fixation behavior (i.e., overall fixation frequency or duration). This study provides a direct link between social and sensory processing in autism. Moreover, reduced social attention may be an inaccurate characterization of autism. Instead, our results suggest that social attention in autism is better explained by "social vulnerability," particularly to the perceptual load of real-world environments.

Significance Color is a foundational aspect of visual experience that aids in segmenting objects, identifying food sources, and signaling emotions. Intuitively, it feels that we are immersed in a colorful world that extends to the farthest limits of our periphery. How accurate is our intuition? Here, we used gaze-contingent rendering in immersive VR to reveal the limits of color awareness during naturalistic viewing. Observers explored 360° real-world environments, which we altered so that only the regions where observers looked were in color, while their periphery was black-and-white. Overall, we found that observers routinely failed to notice when color vanished from the majority of their visual world. These results show that our intuitive sense of a rich, colorful world is largely incorrect., Color ignites visual experience, imbuing the world with meaning, emotion, and richness. As soon as an observer opens their eyes, they have the immediate impression of a rich, colorful experience that encompasses their entire visual world. Here, we show that this impression is surprisingly inaccurate. We used head-mounted virtual reality (VR) to place observers in immersive, dynamic real-world environments, which they naturally explored via saccades and head turns. Meanwhile, we monitored their gaze with in-headset eye tracking and then systematically altered the visual environments such that only the parts of the scene they were looking at were presented in color and the rest of the scene (i.e., the visual periphery) was entirely desaturated. We found that observers were often completely unaware of these drastic alterations to their visual world. In the most extreme case, almost a third of observers failed to notice when less than 5% of the visual display was presented in color. This limitation on perceptual awareness could not be explained by retinal neuroanatomy or previous studies of peripheral visual processing using more traditional psychophysical approaches. In a second study, we measured color detection thresholds using a staircase procedure while a set of observers intentionally attended to the periphery. Still, we found that observers were unaware when a large portion of their field of view was desaturated. Together, these results show that during active, naturalistic viewing conditions, our intuitive sense of a rich, colorful visual world is largely incorrect.

Visual search is a universal human activity in naturalistic environments. Traditionally, visual search is investigated under tightly controlled conditions, where head-restricted participants locate a minimalistic target in a cluttered array presented on a computer screen. Do classic findings of visual search extend to naturalistic settings, where participants actively explore complex, real-world scenes? Here, we leverage advances in virtual reality (VR) technology to relate individual differences in classic visual search paradigms to naturalistic search behavior. In a naturalistic visual search task, participants looked for an object within their environment via a combination of head-turns and eye-movements using a head-mounted display. Then, in a classic visual search task, participants searched for a target within a simple array of colored letters using only eye-movements. We tested how set size, a property known to limit visual search within computer displays, predicts the efficiency of search behavior inside immersive, real-world scenes that vary in levels of visual clutter. We found that participants’ search performance was impacted by the level of visual clutter within real-world scenes. Critically, we also observed that individual differences in vi1–3sual search efficiency in classic search predicted efficiency in real-world search, but only when the comparison was limited to the forward-facing field of view for real-world search. These results demonstrate that set size is a reliable predictor of individual performance across computer-based and active, real-world visual search behavior.

Purpose Heating of gradient coils and passive shim components is a common cause of instability in the B0 field, especially when gradient intensive sequences are used. The aim of the study was to set a benchmark for typical drift encountered during MR spectroscopy (MRS) to assess the need for real-time field-frequency locking on MRI scanners by comparing field drift data from a large number of sites. Method A standardized protocol was developed for 80 participating sites using 99 3T MR scanners from 3 major vendors. Phantom water signals were acquired before and after an EPI sequence. The protocol consisted of: minimal preparatory imaging; a short pre-fMRI PRESS; a ten-minute fMRI acquisition; and a long post-fMRI PRESS acquisition. Both pre- and post-fMRI PRESS were non-water suppressed. Real-time frequency stabilization/adjustment was switched off when appropriate. Sixty scanners repeated the protocol for a second dataset. In addition, a three-hour post-fMRI MRS acquisition was performed at one site to observe change of gradient temperature and drift rate. Spectral analysis was performed using MATLAB. Frequency drift in pre-fMRI PRESS data were compared with the first 5:20 minutes and the full 30:00 minutes of data after fMRI. Median (interquartile range) drifts were measured and showed in violin plot. Paired t-tests were performed to compare frequency drift pre- and post-fMRI. A simulated in vivo spectrum was generated using FID-A to visualize the effect of the observed frequency drifts. The simulated spectrum was convolved with the frequency trace for the most extreme cases. Impacts of frequency drifts on NAA and GABA were also simulated as a function of linear drift. Data from the repeated protocol were compared with the corresponding first dataset using Pearson's and intraclass correlation coefficients (ICC). Results Of the data collected from 99 scanners, 4 were excluded due to various reasons. Thus, data from 95 scanners were ultimately analyzed. For the first 5:20 min (64 transients), median (interquartile range) drift was 0.44 (1.29) Hz before fMRI and 0.83 (1.29) Hz after. This increased to 3.15 (4.02) Hz for the full 30 min (360 transients) run. Average drift rates were 0.29 Hz/min before fMRI and 0.43 Hz/min after. Paired t-tests indicated that drift increased after fMRI, as expected (p < 0.05). Simulated spectra convolved with the frequency drift showed that the intensity of the NAA singlet was reduced by up to 26%, 44 % and 18% for GE, Philips and Siemens scanners after fMRI, respectively. ICCs indicated good agreement between datasets acquired on separate days. The single site long acquisition showed drift rate was reduced to 0.03 Hz/min approximately three hours after fMRI. Discussion This study analyzed frequency drift data from 95 3T MRI scanners. Median levels of drift were relatively low (5-min average under 1 Hz), but the most extreme cases suffered from higher levels of drift. The extent of drift varied across scanners which both linear and nonlinear drifts were observed.

The neural systems supporting scene-perception and spatial-memory systems of the human brain are well-described. But how do these neural systems interact? Here, using fine-grained individual-subject fMRI, we report three cortical areas of the human brain, each lying immediately anterior to a region of the scene perception network in posterior cerebral cortex, that selectively activate when recalling familiar real-world locations. Despite their close proximity to the scene-perception areas, network analyses show that these regions constitute a distinct functional network that interfaces with spatial memory systems during naturalistic scene understanding. These "place-memory areas" offer a new framework for understanding how the brain implements memory-guided visual behaviors, including navigation.

Bumetanide has received much interest as a potential pharmacological modulator of the putative imbalance in excitatory/inhibitory (E/I) signaling that is thought to characterize autism spectrum conditions. Yet, currently, no studies of bumetanide efficacy have used an outcome measure that is modeled to depend on E/I balance in the brain. In this manuscript, we present the first causal study of the effect of bumetanide on an objective marker of E/I balance in the brain, binocular rivalry, which we have previously shown to be sensitive to pharmacological manipulation of GABA. Using a within-subjects placebo-control crossover design study, we show that, contrary to expectation, acute administration of bumetanide does not alter binocular rivalry dynamics in neurotypical adult individuals. Neither changes in response times nor response criteria can account for these results. These results raise important questions about the efficacy of acute bumetanide administration for altering E/I balance in the human brain, and highlight the importance of studies using objective markers of the underlying neural processes that drugs hope to target.

Vision is an active process. Humans actively sample their sensory environment via saccades, head turns, and body movements. Yet, little is known about active visual processing in real-world environments. Here, we exploited recent advances in immersive virtual reality (VR) and in-headset eye-tracking to show that active viewing conditions impact how humans process complex, real-world scenes. Specifically, we used quantitative, model-based analyses to compare which visual features participants prioritize over others while encoding a novel environment in two experimental conditions: active and passive. In the active condition, participants used head-mounted VR displays to explore 360º scenes from a first-person perspective via self-directed motion (saccades and head turns). In the passive condition, 360º scenes were passively displayed to participants within the VR headset while they were head-restricted. Our results show that signatures of top-down attentional guidance increase in active viewing conditions: active viewers disproportionately allocate their attention to semantically relevant scene features, as compared with passive viewers. We also observed increased signatures of exploratory behavior in eye movements, such as quicker, more entropic fixations during active as compared with passive viewing conditions. These results have broad implications for studies of visual cognition, suggesting that active viewing influences every aspect of gaze behavior – from the way we move our eyes to what we choose to attend to – as we construct a sense of place in a real-world environment.Significance StatementEye-tracking in immersive virtual reality offers an unprecedented opportunity to study human gaze behavior under naturalistic viewing conditions without sacrificing experimental control. Here, we advanced this new technique to show how humans deploy attention as they encode a diverse set of 360º, real-world scenes, actively explored from a first-person perspective using head turns and saccades. Our results build on classic studies in psychology, showing that active, as compared with passive, viewing conditions fundamentally alter perceptual processing. Specifically, active viewing conditions increase information-seeking behavior in humans, producing faster, more entropic fixations, which are disproportionately deployed to scene areas that are rich in semantic meaning. In addition, our results offer key benchmark measurements of gaze behavior in 360°, naturalistic environments.

Models of healthy brain function and psychiatric conditions assume that excitatory and inhibitory activity are balanced in the human brain at multiple spatial and temporal scales. In human neuroimaging, concentrations of the major excitatory (glutamate) and inhibitory (γ-aminobutyric acid, GABA) neurotransmitters are measured in vivo using magnetic resonance spectroscopy (MRS). However, despite the central importance of E/I balance to theories of brain function, a relationship between regional glutamate and GABA levels in the human brain has not been shown. We addressed this question in a large corpus of edited MRS data collected at 19 different sites (n ​= ​220). Consistent with the notion of E/I balance, we found that levels of glutamate+glutamine (Glx) and GABA+ were highly correlated (R ​= ​0.52, p ​= ​2.86 x 10−14). This relationship held when controlling for site, scanner vendor, and demographics. Controlling for neurochemicals associated with neuronal density and metabolism (i.e. N-acetylaspartate and creatine) significantly reduced the correlation between GABA+ and Glx, suggesting that the levels of GABA+ and Glx may be critically linked to regional metabolism. These results are consistent with the notion that excitation and inhibition are balanced in the human brain.

Autism is a complex neurodevelopmental condition, and little is known about its neurobiology. Much of autism research has focused on the social, communication and cognitive difficulties associated with the condition. However, the recent revision of the diagnostic criteria for autism has brought another key domain of autistic experience into focus: sensory processing. Here, we review the properties of sensory processing in autism and discuss recent computational and neurobiological insights arising from attention to these behaviours. We argue that sensory traits have important implications for the development of animal and computational models of the condition. Finally, we consider how difficulties in sensory processing may relate to the other domains of behaviour that characterize autism.

Summary We experience our visual environment as a seamless, immersive panorama. Yet, each view is discrete and fleeting, separated by expansive eye movements and discontinuous views of our spatial surroundings. How are discrete views of a panoramic environment knit together into a broad, unified memory representation? Regions of the brain's "scene network" are well poised to integrate retinal input and memory [1]: they are visually driven [2, 3] but also densely interconnected with memory structures in the medial temporal lobe [4]. Further, these regions harbor memory signals relevant for navigation [5–8] and adapt across overlapping shifts in scene viewpoint [9, 10]. However, it is unknown whether regions of the scene network support visual memory for the panoramic environment outside of the current field of view and, further, how memory for the surrounding environment influences ongoing perception. Here, we demonstrate that specific regions of the scene network—the retrosplenial complex (RSC) and occipital place area (OPA)—unite discrete views of a 360° panoramic environment, both current and out of sight, in a common representational space. Further, individual scene views prime associated representations of the panoramic environment in behavior, facilitating subsequent perceptual judgments. We propose that this dynamic interplay between memory and perception plays an important role in weaving the fabric of continuous visual experience.

An imbalance between excitatory/inhibitory neurotransmission has been posited as a central characteristic of the neurobiology of autism [1], inspired in part by the striking prevalence of seizures among individuals with the disorder [2]. Evidence supporting this hypothesis has specifically implicated the signaling pathway of the inhibitory neurotransmitter, γ-aminobutyric acid (GABA), in this putative imbalance: GABA receptor genes have been associated with autism in linkage and copy number variation studies [3-7], fewer GABA receptor subunits have been observed in the post-mortem tissue of autistic individuals [8, 9], and GABAergic signaling is disrupted across heterogeneous mouse models of autism [10]. Yet, empirical evidence supporting this hypothesis in humans is lacking, leaving a gulf between animal and human studies of the condition. Here, we present a direct link between GABA signaling and autistic perceptual symptomatology. We first demonstrate a robust, replicated autistic deficit in binocular rivalry [11], a basic visual function that is thought to rely on the balance of excitation/inhibition in visual cortex [12-15]. Then, using magnetic resonance spectroscopy, we demonstrate a tight linkage between binocular rivalry dynamics in typical participants and both GABA and glutamate levels in the visual cortex. Finally, we show that the link between GABA and binocular rivalry dynamics is completely and specifically absent in autism. These results suggest a disruption in inhibitory signaling in the autistic brain and forge a translational path between animal and human models of the condition.

Individuals with autism are often characterized as 'seeing the trees, but not the forest'-attuned to individual details in the visual world at the expense of the global percept they compose. Here, we tested the extent to which global processing deficits in autism reflect impairments in (i) primary visual processing; or (ii) decision-formation, using an archetypal example of global perception, coherent motion perception. In an event-related functional MRI experiment, 43 intelligence quotient and age-matched male participants (21 with autism, age range 15-27 years) performed a series of coherent motion perception judgements in which the amount of local motion signals available to be integrated into a global percept was varied by controlling stimulus viewing duration (0.2 or 0.6 s) and the proportion of dots moving in the correct direction (coherence: 4%, 15%, 30%, 50%, or 75%). Both typical participants and those with autism evidenced the same basic pattern of accuracy in judging the direction of motion, with performance decreasing with reduced coherence and shorter viewing durations. Critically, these effects were exaggerated in autism: despite equal performance at the long duration, performance was more strongly reduced by shortening viewing duration in autism (P0.015) and decreasing stimulus coherence (P0.008). To assess the neural correlates of these effects we focused on the responses of primary visual cortex and the middle temporal area, critical in the early visual processing of motion signals, as well as a region in the intraparietal sulcus thought to be involved in perceptual decision-making. The behavioural results were mirrored in both primary visual cortex and the middle temporal area, with a greater reduction in response at short, compared with long, viewing durations in autism compared with controls (both P0.018). In contrast, there was no difference between the groups in the intraparietal sulcus (P0.574). These findings suggest that reduced global motion perception in autism is driven by an atypical response early in visual processing and may reflect a fundamental perturbation in neural circuitry.

Autism has traditionally been regarded as a disorder of the social brain. Recent reports of differences in visual perception have challenged this notion, but little evidence for altered visual processing in the autistic brain exists. We have previously observed slower behaviorally reported rates of a basic visual phenomenon, binocular rivalry, in autism [1, 2]. During rivalry, two images-one presented to each eye-vie for awareness, alternating back and forth in perception. This competition is modeled to rely, in part, on the balance of excitation and inhibition in visual cortex [3-8], which may be altered in autism [2, 9-14]. Yet direct neural evidence for this potential marker of excitation/inhibition (E/I) balance in autism is lacking. Here, we report a striking alteration in the neural dynamics of binocular rivalry in individuals with autism. Participants viewed true and simulated frequency-tagged binocular rivalry displays while steady-state visually evoked potentials (SSVEPs) were measured over occipital cortex using electroencephalography (EEG). First, we replicate our prior behavioral findings of slower rivalry and reduced perceptual suppression in individuals with autism compared with controls. Second, we provide direct neural evidence for slower rivalry in autism compared with controls, which strongly predicted individuals' behavioral switch rates. Finally, using neural data alone, we were able to predict autism symptom severity (ADOS) and correctly classify individuals' diagnostic status (autistic versus control; 87% accuracy). These findings clearly implicate atypical visual processing in the neurobiology of autism. Down the road, this paradigm may serve as a non-verbal marker of autism for developmental and cross-species research.

Enhanced perception of detail has long been regarded a hallmark of autism spectrum conditions (ASC), but its origins are unknown. Normal sensitivity on all fundamental perceptual measures—visual acuity, contrast discrimination, and flicker detection—is strongly established in the literature. If individuals with ASC do not have superior low-level vision, how is perception of detail enhanced? We argue that this apparent paradox can be resolved by considering visual attention, which is known to enhance basic visual sensitivity, resulting in greater acuity and lower contrast thresholds. Here, we demonstrate that the focus of attention and concomitant enhancement of perception are sharper in human individuals with ASC than in matched controls. Using a simple visual acuity task embedded in a standard cueing paradigm, we mapped the spatial and temporal gradients of attentional enhancement by varying the distance and onset time of visual targets relative to an exogenous cue, which obligatorily captures attention. Individuals with ASC demonstrated a greater fall-off in performance with distance from the cue than controls, indicating a sharper spatial gradient of attention. Further, this sharpness was highly correlated with the severity of autistic symptoms in ASC, as well as autistic traits across both ASC and control groups. These findings establish the presence of a form of “tunnel vision” in ASC, with far-reaching implications for our understanding of the social and neurobiological aspects of autism.

Enhanced spatial processing of local visual details has been reported in individuals with autism spectrum conditions (ASC), and crowding is postulated to be a mechanism that may produce this ability. However, evidence for atypical crowding in ASC is mixed, with some studies reporting a complete lack of crowding in autism and others reporting a typical magnitude of crowding between individuals with and without ASC. Here, we aim to disambiguate these conflicting results by testing both the magnitude and the spatial extent of crowding in individuals with ASC (N = 25) and age- and IQ-matched controls (N = 23) during an orientation discrimination task. We find a strong crowding effect in individuals with and without ASC, which falls off as the distance between target and flanker is increased. Both the magnitude and the spatial range of this effect were comparable between individuals with and without ASC. We also find typical (uncrowded) orientation discrimination thresholds in individuals with ASC. These findings suggest that the spatial extent of crowding is unremarkable in ASC, and is therefore unlikely to account for the visual symptoms reported in individuals with the diagnosis.

An imbalance between cortical excitation and inhibition is a central component of many models of autistic neurobiology. We tested a potential behavioral footprint of this proposed imbalance using binocular rivalry, a visual phenomenon in which perceptual experience is thought to mirror the push and pull of excitatory and inhibitory cortical dynamics. In binocular rivalry, two monocularly presented images compete, leading to a percept that alternates between them. In a series of trials, we presented separate images of objects (e.g., a baseball and a broccoli) to each eye using a mirror stereoscope and asked human participants with autism and matched control subjects to continuously report which object they perceived, or whether they perceived a mixed percept. Individuals with autism demonstrated a slower rate of binocular rivalry alternations than matched control subjects, with longer durations of mixed percepts and an increased likelihood to revert to the previously perceived object when exiting a mixed percept. Critically, each of these findings was highly predictive of clinical measures of autistic symptomatology. Control “playback” experiments demonstrated that differences in neither response latencies nor response criteria could account for the atypical dynamics of binocular rivalry we observed in autistic spectrum conditions. Overall, these results may provide an index of atypical cortical dynamics that may underlie both the social and nonsocial symptoms of autism.

The primary agent of inhibitory neurotransmission in visual cortex, GABA, has been linked to autism etiology in animal, genetic, and post-mortem studies. Specifically, GABA is thought to be down-regulated in the autistic brain, producing an overabundance of excitatory neurotransmission. Yet, perturbations in GABAergic signaling have never been experimentally associated with behavioral symptoms in people with autism. We recently demonstrated a striking autistic deficit in binocular rivalry (Robertson et al., 2013), a visual behavior that is governed by Excitatory/Inhibitory dynamics in the brain. Here, we tested whether the dynamics of binocular rivalry in typical adults are correlated with levels of excitatory (glutamate) and inhibitory (GABA) neurotransmitters measured in vivo. We measured binocular rivalry dynamics in 18 control participants, as well as the concentration of excitatory (glutamate) and inhibitory (GABA) neurotransmitters in the primary visual (test) and motor (control) areas of each individual's brain using Magnetic Resonance Spectroscopy (MRS). GABA in V1 strongly predicted the dynamics of binocular rivalry in control individuals (Proportion of mixed percepts: rho = -0.61, p 0.19). Here, we demonstrate that a robust behavioral alteration in individuals with ASD strongly predicts GABA concentration measured in controls. This research may lead to the first experimental demonstration of a link between a robust autistic symptom and a prominent theory of autistic neural circuitry. Meeting abstract presented at VSS 2015.

Vision in Autism Spectrum Conditions (ASC) is characterized by enhanced perception of local elements, but impaired perception of global percepts. Deficits in coherent motion perception seem to support this characterization, but the roots and robustness of such deficits remain unclear. We aimed to investigate the dynamics of the perceptual decision-making network known to support coherent motion perception. In a series of forced-choice coherent motion perception tests, we parametrically varied a single stimulus dimension, viewing duration, to test whether the rate at which evidence is accumulated towards a global decision is atypical in ASC. 40 adult participants (20 ASC) performed a classic motion discrimination task, manually indicating the global direction of motion in a random-dot kinematogram across a range of coherence levels (2-75%) and stimulus-viewing durations (200-1500 ms). We report a deficit in global motion perception at short viewing durations in ASC. Critically, however, we found that increasing the amount of time over which motion signals could be integrated reduced the magnitude of the deficit, such that at the longest duration there was no difference between the ASC and control groups. Further, the deficit in motion integration at the shortest duration was significantly associated with the severity of autistic symptoms in our clinical population, and was independent from measures of intelligence. These results point to atypical integration of motion signals during the construction of a global percept in ASC. Based on the neural correlates of decision-making in global motion perception our findings suggest the global motion deficit observed in ASC could reflect a slower or more variable response from the primary motion area of the brain or longer accumulation of evidence towards a decision-bound in parietal areas.