Search

When a complexly structured natural image is presented twice in succession, first as adapter and then as test, neurons in area TE of macaque inferotemporal cortex exhibit repetition suppression, responding less strongly to the second presentation than to the first. This phenomenon, which has been studied primarily in TE, might plausibly be argued to arise in TE because TE neurons respond selectively to complex images and thus carry information adequate for determining whether an image is or is not a repeat. However, the idea has never been put to a direct test. To resolve this issue, we monitored neuronal responses to sequences of complex natural images under identical conditions in areas V2 and TE. We found that repetition suppression occurs in both areas. Moreover, in each area, suppression takes the form of a dynamic alteration whereby the initial peak of excitation is followed by a trough and then a rebound of firing rate. To assess whether repetition suppression in either area is transmitted from the other area, we analyzed the timing of the phenomenon and its degree of spatial generalization. Suppression occurs at shorter latency in V2 than in TE. Therefore it is not simply fed back from TE. Suppression occurs in TE but not in V2 under conditions in which the test and adapter are presented in different visual field quadrants. Therefore it is not simply fed forward from V2. We conclude that repetition suppression occurs independently in V2 and TE.

The intrinsically photosensitive retinal ganglion cells (ipRGCs) regulate pupil size and circadian rhythms. Stimulation of the ipRGCs using short-wavelength blue light causes a sustained pupil constriction known as the post-illumination pupil response (PIPR). Here we examined the effects of ipRGC stimulation on axial length changes to imposed optical defocus in young adults.Nearly emmetropic young participants were given either myopic (+3 D, n = 16) or hyperopic (-3 D, n = 17) defocus in their right eye for 2 h. Before and after defocus, a series of axial length measurements for up to 180 s were performed in the right eye using the IOL Master following exposure to 5 s red (625 nm, 3.74 × 10The PIPR with blue light was significantly stronger after 2 h of hyperopic defocus as indicated by a lower 6 and 30 s PIPR and a larger early and late AUC (all p0.05). Short-wavelength ipRGC stimulation also significantly exaggerated the ocular response to hyperopic defocus, causing a significantly greater increase in axial length than that resulting from the hyperopic defocus alone (p = 0.017). Neither wavelength had any effect on axial length with myopic defocus.These findings suggest an interaction between myopiagenic hyperopic defocus and ipRGC signaling.

Theories of cognitive fatigue disagree on whether performance decrement is caused by motivational or functional alterations. Here, drawing inspiration from the habituation and visual adaptation literature, we tested the assumption that keeping neural networks active for an extensive period of time entails consequences at the subjective and objective level-the defining characteristics of fatigue-when confounds such as motivation, boredom, and level of skill are controlled. In Experiment 1, we revealed that passive visual stimulation affected the performance of a subsequent task that was carried out in the same portion of visual space. While under conditions of low cognitive load and arousal, participants improved their performance in the stimulated quadrant; the reverse was observed under high arousal conditions. This latter performance decrement correlated also with the reported subjective level of fatigue and occurred while neural responses to the saturating stimulus remained constant, as assessed through steady-state EEG. In subsequent experiments, we replicated and further characterized this performance deterioration effect, revealing its specificity to the stimulated eye and stimulus orientation. Across the three experiments, the decrease in performance was correlated with pupil-linked arousal, suggesting its mediating effect in this phenomenon. In sum, we show that repeated stimulation of neural networks under high-arousal conditions leads to their altered functional performance, a mechanism which may play a role in the development of global cognitive fatigue. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

In the brain, spiking patterns live in a high-dimensional space of neurons and time. Thus, determining the intrinsic structure of this space presents a theoretical and experimental challenge. To address this challenge, we introduce a new framework for applying topological data analysis (TDA) to spike train data and use it to determine the geometry of spiking patterns in the visual cortex. Key to our approach is a parametrized family of distances based on the timing of spikes that quantifies the dissimilarity between neuronal responses. We applied TDA to visually driven single-unit and multiple single-unit spiking activity in macaque V1 and V2. TDA across timescales reveals a common geometry for spiking patterns in V1 and V2 which, among simple models, is most similar to that of a low-dimensional space endowed with Euclidean or hyperbolic geometry with modest curvature. Remarkably, the inferred geometry depends on timescale and is clearest for the timescales that are important for encoding contrast, orientation and spatial correlations., NIH grants number EY09314 and EY07977 from the National Eye Institute of the NIH. NIH grants no. EY034150 from the National Eye Institute and NS111019 from the National Institute of Neurological Disorders and Stroke. Ikerbasque. Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. Inria Associated Team "NeuroTransSF".

A growing body of literature suggests that cognitive impairment in people with schizophrenia (PSZ) results from disrupted cortical excitatory/inhibitory (E-I) balance, which may be linked to gamma entrainment and can be measured noninvasively using electroencephalography (EEG). However, it is not yet known the degree to which these entrainment abnormalities covary within subjects across sensory modalities. Furthermore, the degree to which cross-modal gamma entrainment reflects variation in biological processes associated with cognitive performance remains unclear. We used EEG to measure entrainment to repetitive auditory and visual stimulation at beta (20 Hz) and gamma (30 and 40 Hz) frequencies in PSZ (n = 78) and healthy control subjects (HCS; n = 80). Three indices were measured for each frequency and modality: event-related spectral perturbation (ERSP), intertrial coherence (ITC), and phase-lag angle (PLA). Cognition and symptom severity were also assessed. We found little evidence that gamma entrainment covaried across sensory modalities. PSZ exhibited a modest correlation between modalities at 40 Hz for ERSP and ITC measures (r = 0.23-0.24); however, no other significant correlations between modalities emerged for either HCS or PSZ. Both univariate and multivariate analyses revealed that (a) the pattern of entrainment abnormalities in PSZ differed across modalities, and (b) modality rather than frequency band was the main source of variance. Finally, we observed a significant association between cognition and gamma entrainment in the auditory domain only in HCS. Gamma-band EEG entrainment does not reflect a unitary transcortical mechanism but is instead modality specific. To the extent that entrainment reflects the integrity of cortical E-I balance, the deficits observed in PSZ appear to be modality specific and not consistently associated with cognitive impairment. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Today, the effect of extreme early-life exposure to digital devices is suggested as a risk factor for neurodevelopmental disorders. However, the multitude of factors that influence brain development with subsequent behavioural abnormalities have not been fully elucidated. Herein, we simulated extreme early-life exposure to digital devices in rats by audio and visual stimulation and investigated its effects on autism-related behaviours and brain structural alteration. Male rat pups were exposed to excessive audio–visual stimulation (EAVS) from PND (post-natal day) 12 to PND 35, with and without maternal separation (MS). Autism-related behaviours including abnormal sociability, stereotype behaviours, anxiety and locomotor dysfunction were tested at PND 42. Brain structural alternation was examined by considering the amygdala, mPFC (medial prefrontal cortex) and hippocampal regions while performing 3D quantitative stereological analysis. We found that EAVS led to social behaviour deficit and higher locomotion in rats, which were associated with increases in the number of neurons and volume of the amygdala. We also showed that MS did not exaggerate the effect of extreme sensory stimulation on behaviour and the structure of the brain. This study proposed EAVS in rats as an animal model of early exposure to digital devices for investigating possible neurobiological alternations underlying autistic-like behaviours with an emphasis on the amygdala area.

Motion opponency, first observed within the primate middle temporal cortex (MT), refers to the suppressing effect of opposite motion directions on neuronal activity. Namely, when opposing motion directional signals stimulate an MT neuron's receptive field, this neuron's response is comparable with that induced by flicker noise. Under such suppression, it is unknown whether any directional information is still represented at MT. In this study, we applied support vector machine (SVM) learning to human functional magnetic resonance imaging data to investigate if any motion defined orientation information was still available from suppressed MT. We found that, at least at the level of ±45° discrimination, such orientation information was still available. Interestingly, after behavioural perceptual learning that improved human discrimination of fine orientation discrimination (e.g. 42° vs. 48°) using the MT-suppressive motion stimuli, the SVM discrimination of ±45° worsened when functional magnetic resonance imaging (fMRI) signals at post-learning MT were used. This result is consistent with findings in Thompson et al. (2013) that, post-perceptual learning, MT suppression was not released, suggesting that motion opponency was perhaps functionally too important for perceptual learning to overcome.

Direction selective neurons in macaque primary visual cortex are narrowly tuned for orientation, and are thus afflicted by the aperture problem. At the next stage of motion processing, in the middle temporal (MT) area, some cells appear to solve this problem, responding to the pattern motion direction of plaids. Models have been proposed to account for this computation, but they do not replicate the diversity of responses observed in MT. We recorded from 386 cells in area MT of two male macaques, while presenting a wide range of random-line stimuli and their compositions into noise plaids. As we broadened the range of stimuli used to probe the cells, yielding ever more challenging conditions for extracting pattern motion, the diversity of the responses observed increased, and the fraction of cells that faithfully encoded pattern motion direction shrank. However, we show here that a pattern motion signal is present at the population level. We identified four mechanisms, one never proposed before, that together might account for the observed diversity in single-cell responses. Pattern motion is thus extracted in area MT, but it is encoded across the population, and not in a small subset of pattern neurons.SIGNIFICANCE STATEMENTSome neurons in the middle temporal area of macaques solve the aperture problem, signaling the direction of motion of complex patterns. As the number of pattern types used to probe this mechanism is increased, fewer and fewer cells retain this capability. We show here that different cells fail in different ways, and that simply summing their responses averages away their failures, yielding a clear pattern motion signal. Similar encodings, which unequivocally violate the “neuron as a feature detector” hypothesis that has dominated sensory processing theories for the past 50 years, might apply throughout the brain.

Pupil diameter, under constant illumination, is known to reflect individuals' internal states, such as surprise about observation and environmental uncertainty. Despite the growing use of pupillometry in cognitive learning studies as an additional measure for examining internal states, few studies have used pupillometry in human motor learning studies. Here, we provide the first detailed characterization of pupil diameter changes in a short-term reach adaptation paradigm. We measured pupil changes in 121 human participants while they adapted to abrupt, gradual, or switching force field conditions. Sudden increases in movement error caused by the introduction/reversal of the force field resulted in strong phasic pupil dilation during movement accompanied by a transient increase in tonic premovement baseline pupil diameter in subsequent trials. In contrast, pupil responses were reduced when the force field was gradually introduced, indicating that large, unexpected errors drove the changes in pupil responses. Interestingly, however, error-induced pupil responses gradually became insensitive after experiencing multiple force field reversals. We also found an association between baseline pupil diameter and incidental knowledge of the gradually introduced perturbation. Finally, in all experiments, we found a strong co-occurrence of larger baseline pupil diameter with slower reaction and movement times after each rest break. Collectively, these results suggest that tonic baseline pupil diameter reflects one's belief about environmental uncertainty, whereas phasic pupil dilation during movement reflects surprise about a sensory outcome (i.e., movement error), and both effects are modulated by novelty. Our results provide a new approach for nonverbally assessing participants' internal states during motor learning.

Self-motion through an environment induces various sensory signals, i.e., visual, vestibular, auditory, or tactile. Numerous studies have investigated the role of visual and vestibular stimulation for the perception of self-motion direction (heading). Here, we investigated the rarely considered interaction of visual and tactile stimuli in heading perception. Participants were presented optic flow simulating forward self-motion across a horizontal ground plane (visual), airflow toward the participants' forehead (tactile), or both. In separate blocks of trials, participants indicated perceived heading from unimodal visual or tactile or bimodal sensory signals. In bimodal trials, presented headings were either spatially congruent or incongruent with a maximum offset between visual and tactile heading of 30°. To investigate the reference frame in which visuo-tactile heading is encoded, we varied head and eye orientation during presentation of the stimuli. Visual and tactile stimuli were designed to achieve comparable precision of heading reports between modalities. Nevertheless, in bimodal trials heading perception was dominated by the visual stimulus. A change of head orientation had no significant effect on perceived heading, whereas, surprisingly, a change in eye orientation affected tactile heading perception. Overall, we conclude that tactile flow is more important to heading perception than previously thought.

A variety of reports suggest that rhythmic auditory stimulation can entrain visual perception, inducing perceptual oscillations as a function of time relative to the auditory rhythm. These effects have, to date, been reported only for stimulation frequencies at and below 3 Hz. Here we investigate the effects of rhythmic auditory stimulation on the detection of masked visual targets when this stimulation occurs at frequencies to which the visual system has been shown to entrain (8-12 Hz). Across four experiments, we found no consistent evidence of poststimulation modulations in performance induced by 8.5, 10.6, or 12.1 Hz auditory stimulation. This absence of a consistent auditory-to-visual effect was paralleled by an absence of unimodal effects (oscillations in auditory performance) following 12.1 Hz auditory stimulation. In a fifth experiment, we found that although auditory stimulation alone did not induce oscillations in visual performance, auditory stimulation did enhance the effects of concurrent visual stimulation. Notably, this enhancement did not require synchronous presentation of stimuli. These observations are consistent with recent reports that passive auditory stimulation beyond 5 Hz induces neither auditory perceptual oscillations nor visual cortical oscillations and suggest limits to the extent of cross-modal entrainment. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The electroretinogram (ERG) allows the investigation of retinal signaling pathways and has increasingly been applied in individuals with mental disorders in search for potential biomarkers of neurodevelopmental disorders. Preceding ERG examinations in individuals with autism spectrum disorders (ASD) showed inconsistent results, which might be due to the small number of participants, heterogeneity of the ASD population, differences in age ranges, and stimulation methods. The aim of this study was to investigate functional retinal responses in adults with ASD by means of the light-adapted (photopic) ERG. Light-adapted ERG measurements were obtained with the RETeval® system applying three different stimulation protocols. In the final analysis, the ERG parameters a-wave, b-wave, the photopic negative response (PhNR), the photopic hill parameters as well as additional amplitude ratios were compared between 32 adults with high-functioning ASD and 31 non-autistic controls. Both groups were matched with regard to sex and age. No significant functional retinal differences in amplitude or peak time of the a- or b-wave, PhNR, the photopic hill parameters or the ERG-amplitude ratios could be detected in individuals with ASD compared to non-autistic participants. The absence of electrophysiological functional retinal alterations in ASD, suggests that changes in visual perception, such as increased attention to detail or visual hypersensitivity in ASD, are not due to impairments at early levels of retinal signal processing.

Decisions about where to move the eyes depend on neurons in frontal eye field (FEF). Movement neurons in FEF accumulate salience evidence derived from FEF visual neurons to select the location of a saccade target among distractors. How visual neurons achieve this salience representation is unknown. We present a neuro-computational model of target selection called salience by competitive and recurrent interactions (SCRI), based on the competitive interaction model of attentional selection and decision-making (SmithSewell, 2013). SCRI selects targets by synthesizing localization and identification information to yield a dynamically evolving representation of salience across the visual field. SCRI accounts for neural spiking of individual FEF visual neurons, explaining idiosyncratic differences in neural dynamics with specific parameters. Many visual neurons resolve the competition between search items through feedforward inhibition between signals representing different search items, some also require lateral inhibition, and many act as recurrent gates to modulate the incoming flow of information about stimulus identity. SCRI was tested further by using simulated spiking representations of visual salience as input to the gated accumulator model of FEF movement neurons (Purcell et al., 2010, 2012). Predicted saccade response times fit those observed for search arrays of different set sizes and different target-distractor similarities, and accumulator trajectories replicated movement neuron discharge rates. These findings offer new insights into visual decision-making through converging neuro-computational constraints and provide a novel computational account of the diversity of FEF visual neurons. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Hearing a task-irrelevant sound during object encoding can improve visual recognition memory when the sound is object-congruent (e.g., a dog and a bark). However, previous studies have only used binary old/new memory tests, which do not distinguish between recognition based on the recollection of details about the studied event or stimulus familiarity. In the present research, we hypothesized that hearing a task-irrelevant but semantically congruent natural sound at encoding would facilitate the formation of richer memory representations, resulting in increased recollection of details of the encoded event. Experiment 1 replicates previous studies showing that participants were more confident about their memory for items that were initially encoded with a congruent sound compared to an incongruent sound. Experiment 2 suggests that congruent object-sound pairings specifically facilitate recollection and not familiarity-based recognition memory, and Experiment 3 demonstrates that this effect was coupled with more accurate memory for audiovisual congruency of the item and sound from encoding rather than another aspect of the episode. These results suggest that even when congruent sounds are task-irrelevant, they promote formation of multisensory memories and subsequent recollection-based retention. Given the ubiquity of encounters with multisensory objects in our everyday lives, considering their impact on episodic memory is integral to building models of memory that apply to naturalistic settings.

A cursor control system based on brain-computer interface (BCI) provides efficient computer access. These systems operate without any muscular activity from the user. Conventional BCI-based cursor control systems have several limitations. Therefore, hybrid SSVEP + P300 visual BCI (VBCI)-based cursor control is needed to overcome these limitations. This paper explores the feasibility of using noninvasive hybrid SSVEP + P300 VBCI for cursor control as a universal form of computer access. The proposed cursor control system has a graphical user interface (GUI) design that simultaneously evokes both SSVEP and P300 signals in the human cortex. The performance metrics of the proposed system are compared with conventional SSVEP VBCI and P300 VBCI-based cursor control systems. The proposed hybrid SSVEP + P300 BCI-based cursor control system achieves a maximum accuracy of 97.51% with a 27.15 bit/min information transfer rate (ITR). The results proved that the proposed system performed more efficiently than other systems. The proposed system was tested in a noisy environment and found to be suitable for real-world applications.

An increasing number of studies have shown that cross-modal interaction can occur in early sensory cortices. Yet, how neurons in sensory cortices integrate multisensory cues in perceptual tasks and to what extent this influences behavior is largely unclear. To investigate, we examined visual modulation of auditory responses in the primary auditory cortex (A1) in a two-alternative forced-choice task. During the task, male rats were required to make a behavioral choice based on the pure tone frequency (low vs high) of the self-triggered stimulus to get a water reward. The result showed that the presence of a noninformative visual cue did not uniformly influence auditory response, with frequently enhancing just one of them. Closely correlated with behavioral choice, the visual cue mainly enhanced responsiveness to the auditory cue indicating a movement direction contralateral to A1 being recorded. Operating in this fashion provided A1 neurons a superior capability to discriminate sound during multisensory trials. Concomitantly, behavioral data and decoding analysis revealed that visual cue presence could speed the process of sound discrimination. We also observed this differential multisensory integration effect in well-trained rats when tested with passive stimulation and under anesthesia, albeit to a much lesser extent. We did not see this differentially integrative effect while recording in A1 in another similar group of rats performing a free-choice task. These data suggest that auditory cortex can engage in meaningful audiovisual processing, and perceptual learning can modify its multisensory integration mechanism to meet task requirements.SIGNIFICANCE STATEMENTIn the natural environment, visual stimuli are frequently accompanied by auditory cues. Although multisensory integration has traditionally been seen as a feature of associational cortices, recent studies have shown that cross-modal inputs can also influence neuronal activity in primary sensory cortices. However, exactly how neurons in sensory cortices integrate multisensory cues to guide behavioral choice is still unclear. Here, we describe a novel model of multisensory integration used by A1 neurons to shape auditory representations when rats performed a cue-guided task. We found that a task-irrelevant visual cue could specifically enhance the response of neurons in sound guiding to the contralateral choice. This differentially integrative model facilitated sound discrimination and behavioral choice. This result indicates that task engagement can modulate multisensory integration.

A socially consequential test of the cognitive penetrability of visual perception is whether merely sharing a group membership with another person influences how you encode their face. Past research has examined this issue by manipulating group membership with techniques from social psychology and then measuring the face-sensitive N170 ERP. However, methodological differences across studies make it difficult to draw conclusions from this literature. In our research, we conducted two large-scale, preregistered ERP studies to address how critical methodological decisions could influence conclusions about top–down effects of group membership on face perception. Specifically, we examined how mere group membership, perceptual markers that signify group membership, number of trials included in the study design, the racial/ethnic identity of face stimuli, and the data analytic approach affect inferences about the N170 response to faces. In Study 1, we found no evidence that mere group membership significantly influenced the N170. However, we found that the background color used to signify group membership modulated the magnitude and latency of the N170. Exploratory analyses also showed effects of stimulus race/ethnicity. In Study 2, we dissociated background color from face encoding by presenting background color before the faces. In this second study, we found no main effect of group membership, background color, or stimulus race/ethnicity. However, we did see an unhypothesized mere group membership effect on trials toward the end of the study. Our results inform debates about social categorization effects on visual perception and show how bottom–up indicators of group membership can bias face encoding.

Audiovisual integrations and interactions happen everywhere, including in music concerts, where combined visual and auditory perception contributes to overall enjoyment. Thirty-three participants evaluated their overall subjective preference at various seats in four virtual auditoria, which comprised congruent and incongruent auditory and visual renders of two auditoria that differ only in size. Results show no significant difference between participants who completed the experiment in a fully calibrated and standardized laboratory environment and participants who completed remotely using various VR equipment in various environments. Both visual and auditory auditorium size have significant main effects, but no interaction. The larger hall is preferred for both conditions. Audiovisual congruency does not significantly affect preference.

The mouse visual system consists of several visual cortical areas thought to be specialized for different visual features and/or tasks. Previous studies have revealed differences between primary visual cortex (V1) and other higher visual areas, namely, anterolateral (AL) and posteromedial (PM), and their tuning preferences for spatial and temporal frequency. However, these differences have primarily been characterized using methods that are biased toward superficial layers of cortex, such as two-photon calcium imaging. Fewer studies have investigated cell types in deeper layers of these areas and their tuning preferences. Because superficial versus deep-layer neurons and different types of deep-layer neurons are known to have different feedforward and feedback inputs and outputs, comparing the tuning preferences of these groups is important for understanding cortical visual information processing. In this study, we used extracellular electrophysiology and two-photon calcium imaging targeted toward two different layer 5 cell classes to characterize their tuning properties in V1, AL, and PM. We find that deep-layer neurons, similar to superficial layer neurons, are also specialized for different spatial and temporal frequencies, with the strongest differences between AL and V1, and AL and PM, but not V1 and PM. However, we note that the deep-layer neuron populations preferred a larger range of SFs and TFs compared to previous studies. We also find that extratelencephalically projecting layer 5 neurons are more direction selective than intratelencephalically projecting layer 5 neurons.

Purpose: We compared the degree of cognitive processing needed by people with aphasia to identify themes depicted in grids and visual scene displays (VSDs). We also compared the accuracy of theme identification for both display types. Method: Eye-tracking technology was employed to measure the visual processing patterns of 21 adults with aphasia when interpreting themes presented through grids and VSDs. Additionally, we assessed theme identification accuracy by having participants select themes from four choices after viewing each display. Results: Participants more rapidly identified VSDs than grid displays, and VSDs required fewer visual fixations to process than grids. No significant differences were noted between grids and VSDs for theme identification accuracy; however, results indicate a ceiling effect for the variable, as participant accuracy levels were nearly 100% for both display conditions. Conclusions: Results from this study add to a growing body of evidence supporting the use of VSDs for adults with aphasia. Both display types were accurately identified; however, VSDs were processed more efficiently than grids indicating that both display types may prove effective for people with aphasia; however, VSDs may require less cognitive effort to effectively use than grid displays.

When interacting with the environment, humans exhibit robust biases towards information that pertains to themselves: Self-relevant information is processed faster and yields more accurate responses than information linked to others. Recent studies have shown that simple social associations can lead to the instant deployment of this benefit in the processing of abstract stimuli. However, how self-prioritization evolves across the processing hierarchy has been a subject of intense debate. Furthermore, there is little empirical evidence about the functional efficiency of social relevance in natural environments in which information is present across multiple senses. Across three experiments (each n = 40), the present study shows that self-prioritization effects (1) can arise in simple audio-visual numerosity judgements, (2) can be efficiently deployed across the senses by funnelling perception towards self-relevant information in the more reliable sensory modality, and (3) modulates the integration of auditory and visual information into a multisensory representation. Taken together, the present findings suggest that social salience can influence multisensory processing at both perceptual and post-perceptual stages via early attentional modulations of sensory integration and later, task-dependent attentional control.

The visual area V4 has been considered to play a crucial role in the intermediate representation of objects, where low-level image features are transformed into object-level representations. We estimated the intrinsic dimensionality in V4 for the representation of local patches generated from natural scenes. The dimensionality was approximately 40, which is approximately half of that reported in IT for the representation of whole natural objects. The analyses of the estimated dimensionality suggest both common and independent representations that code contour shapes and/or surfaces with textures, implying a relatively complex and mixed representation in the intermediate-level area.

Research findings on which brain areas demonstrate fMRI adaptation to the form, orientation, and size of visual stimuli has been mixed. Studies demonstrate effects in various subdivisions of the lateral occipital complex (LOC), including retinotopically tuned areas LO-1 and LO-2, and dorsal stream areas in the intraparietal sulcus (IPS). Therefore, we aimed to examine fMRI adaptation in four subdivisions of the LOC (LO-proper, posterior fusiform sulcus, LO-1, and LO-2) and three discrete regions in the IPS (caudal IPS, IPS-proper, and anterior IPS) to clarify the role that these structures play in form, orientation, and size processing. Participants performed three tasks which involved judging whether two serially presented novel objects shared the same form, orientation, or size. On each version of the task, one feature varied from trial-to-trial (e.g., form) while the other two features (e.g., orientation and size) were held constant. In this way, we were able to examine the unique fMRI signal changes in response to changes in form, orientation, and size in isolation. Form adaptation - a decrease in fMRI signal following repeated presentation of the same stimulus - was present in LO-proper and pFS, highlighting the role of LOC in form processing. Size repetition enhancement - an increase in fMRI signal following repeated presentations of the same stimulus - was observed in pFs. We propose that the latter result demonstrates the effects that top-down factors can have on visual areas, specifically when there is stimulus uncertainty. There was no evidence of orientation processing in any of the regions examined. Neither form adaptation nor size repetition enhancement was present in the three IPS regions-of-interest. Last, retinotopically defined LO1 and LO2 could not be reliably identified in participants and therefore we were unable to examine adaptation in these areas as we originally intended to do.

Spatial attention is generally believed to increase contrast appearance of visual stimuli. A recent study, however, revealed that attention attenuates appearance of high-contrast stimuli. This puzzling attentional attenuation was ascribed to the attentional modulation on supersaturating neurons, whose response initially increases to peak and then decreases as stimulus intensity increases. Here, in two experiments (

A person's focus of attention is conveyed by the direction of their eyes and face, providing a simple visual cue fundamental to social interaction. A growing body of research examines the visual mechanisms that encode the direction of another person's gaze as we observe them. Here we investigate the spatial receptive field properties of these mechanisms, by testing the spatial selectivity of sensory adaptation to gaze direction. Human observers were adapted to faces with averted gaze presented in one visual hemifield, then tested in their perception of gaze direction for faces presented in the same or opposite hemifield. Adaptation caused strong, repulsive perceptual aftereffects, but only for faces presented in the same hemifield as the adapter. This occurred even though adapting and test stimuli were in the same external location across saccades. Hence, there was clear evidence for retinotopic adaptation and a relative lack of either spatiotopic or spatially invariant adaptation. These results indicate that adaptable representations of gaze direction in the human visual system have retinotopic spatial receptive fields. This strategy of coding others' direction of gaze with positional specificity relative to one's own eye position may facilitate key functions of gaze perception, such as socially cued shifts in visual attention.

When an image is presented twice in succession, neurons in area TE of macaque inferotemporal cortex exhibit repetition suppression, responding less strongly to the second presentation than to the first. Suppression is known to occur if the adapter and the test image are subtly different from each other. However, it is not known whether cross suppression occurs between images that are radically different from each other but that share a subset of features. To explore this issue, we measured repetition suppression using colored shapes. On interleaved trials, the test image might be identical to the adapter, might share its shape or color alone, or might differ from it totally. At the level of the neuronal population as a whole, suppression was especially deep when adapter and test were identical, intermediate when they shared only one attribute, and minimal when they shared neither attribute. At the level of the individual neuron, the degree of suppression depended not only on the properties of the two images but also on the preferences of the neuron. Suppression was deeper when the repeated color or shape was preferred by the neuron than when it was not. This effect might arise from feature-specific adaptation or alternatively from adapter-induced fatigue. Both mechanisms conform to the principle that the degree of suppression is determined by the preferences of the neuron.

Mice detect decreases in illumination in dim light near the visual threshold with OFF retinal ganglion cells.

Serial dependence is an attractive pull that recent perceptual history exerts on current judgments. Theory suggests that this bias is due to a form of short-term plasticity prevalent specifically in the frontal lobe. We sought to test the importance of the frontal lobe to serial dependence by disrupting neural activity along its lateral surface during two tasks with distinct perceptual and motor demands. In our first experiment, stimulation of the lateral prefrontal cortex (LPFC) during an oculomotor delayed response task decreased serial dependence only in the first saccade to the target, whereas stimulation posterior to the LPFC decreased serial dependence only in adjustments to eye position after the first saccade. In our second experiment, which used an orientation discrimination task, stimulation anterior to, in, and posterior to the LPFC all caused equivalent decreases in serial dependence. In this experiment, serial dependence occurred only between stimuli at the same location; an alternation bias was observed across hemifields. Frontal stimulation had no effect on the alternation bias. Transcranial magnetic stimulation to parietal cortex had no effect on serial dependence in either experiment. In summary, our experiments provide evidence for both functional differentiation (Experiment 1) and redundancy (Experiment 2) in frontal cortex with respect to serial dependence.

Decades of research show that contextual information from the body, visual scene, and voices can facilitate judgments of facial expressions of emotion. To date, most research suggests that bodily expressions of emotion offer context for interpreting facial expressions, but not vice versa. The present research aimed to investigate the conditions under which mutual processing of facial and bodily displays of emotion facilitate and/or interfere with emotion recognition. In the current two studies, we examined whether body and face emotion recognition are enhanced through integration of shared emotion cues, and/or hindered through mixed signals (i.e., interference). We tested whether faces and bodies facilitate or interfere with emotion processing by pairing briefly presented (33 ms), backward-masked presentations of faces with supraliminally presented bodies (Experiment 1) and vice versa (Experiment 2). Both studies revealed strong support for integration effects, but not interference. Integration effects are most pronounced for low-emotional clarity facial and bodily expressions, suggesting that when more information is needed in one channel, the other channel is recruited to disentangle any ambiguity. That this occurs for briefly presented, backward-masked presentations reveals low-level visual integration of shared emotional signal value.

Our visual system is bombarded with numerous social interactions that form intangible social bonds among people, as exemplified by synchronized walking in crowds. Here, we investigated whether these perceived social bonds implicitly intrude on visual perception and induce a contextual effect. Using multiple point-light walkers and a classical contextual paradigm, we tested 72 college-age adults across six experiments and found that the perceived direction of the central walker was attracted toward the direction of the surrounding walkers. The observed contextual-attraction effect occurred even when the surrounding walkers differed from the central walker in gender and walking speed but disappeared when they were asynchronously presented or replaced by inanimate motion. Strikingly, this contextual-attraction effect partially persisted in the context of local motion rather than static figures. These findings, in contrast to the typical contextual-repulsion effect, lend support for the distinctiveness of perceived social bonds on contextual modulation and suggest a specialized contextual mechanism tuned to social factors.

The human middle-temporal region MT+ is highly specialized in processing visual motion. However, recent studies have shown that this region is modulated by extraretinal signals, suggesting a possible involvement in processing motion information also from non-visual modalities. Here, we used functional MRI data to investigate the influence of retinal and extraretinal signals on MT+ in a large sample of subjects. Moreover, we used resting-state functional MRI to assess how the subdivisions of MT+ (i.e., MST, FST, MT, and V4t) are functionally connected. We first compared responses in MST, FST, MT, and V4t to coherent vs. random visual motion. We found that only MST and FST were positively activated by coherent motion. Furthermore, regional analyses revealed that MST and FST were positively activated by leg, but not arm, movements, while MT and V4t were deactivated by arm, but not leg, movements. Taken together, regional analyses revealed a visuomotor role for the anterior areas MST and FST and a pure visual role for the anterior areas MT and V4t. These findings were mirrored by the pattern of functional connections between these areas and the rest of the brain. Visual and visuomotor regions showed distinct patterns of functional connectivity, with the latter preferentially connected with the somatosensory and motor areas representing leg and foot. Overall, these findings reveal a functional sensitivity for coherent visual motion and lower-limb movements in MST and FST, suggesting their possible involvement in integrating sensory and motor information to perform locomotion.

The spatial-frequency (SF) tuning of neurons in the early visual cortex is adjusted for stimulus contrast. As the contrast increases, SF tuning is modulated so that the transmission of fine features is facilitated. A variety of mechanisms are involved in shaping SF tunings, but those responsible for the contrast-dependent modulations are unclear. To address this, we measured the time course of SF tunings of area 17 neurons in male cats under different contrasts with a reverse correlation. After response onset, the optimal SF continuously shifted to a higher SF over time, with a larger shift for higher contrast. At high contrast, whereas neurons with a large shift of optimal SF exhibited a large bandwidth decrease, those with a negligible shift increased the bandwidth over time. Between these two extremes, the degree of SF shift and bandwidth change continuously varied. At low contrast, bandwidth generally decreased over time. These dynamic effects enhanced the processing of high-frequency range under a high-contrast condition and allowed time-average SF tuning curves to show contrast-dependent modulation, like that of steady-state SF tuning curves reported previously. Combinations of two mechanisms, one that decreases bandwidth and shifts optimal SF, and another that increases bandwidth without shifting optimal SF, would explain the full range of SF tuning dynamics. Our results indicate that one of the essential roles of tuning dynamics of area 17 neurons, which have been observed for various visual features, is to adjust tunings depending on contrast. SIGNIFICANCE STATEMENT The spatial scales of features transmitted by cortical neurons are adjusted depending on stimulus contrast. However, the underlying mechanism is not fully understood. We measured the time course of spatial frequency tunings of cat area 17 neurons under different contrast conditions and observed a variety of dynamic effects that contributed to spatial-scale adjustment, allowing neurons to adjust their spatial frequency tuning range depending on contrast. Our results suggest that one of the essential roles of tuning dynamics of area 17 neurons, which have been observed for various visual features, is to adjust tunings depending on contrast.

The debate about the scope and limits of unconscious visual processing under continuous flash suppression (CFS) has created a heterogeneous set of divergent findings that are yet to be reconciled. Attention has been suggested as an important factor in modulating the processing of suppressed visual information under CFS. Specifically, Eo et al. (2016) reported that semantic processing under CFS can be significantly facilitated when spatial attention is diverted away from the suppressed stimulus. Based on event-related potential (ERP) findings involving the N400, they proposed that inattention attenuates interocular suppression and thereby makes semantic processing available unconsciously, potentially reconciling conflicting evidence in the literature. In this study, we aimed to further investigate the "CFS-attenuation-by-inattention" hypothesis using functional magnetic resonance imaging (fMRI) and multivariate pattern analysis (MVPA). We tested whether the decodability of object category increases under CFS when attention is diverted away from the suppressed stimulus in a spatial cueing task. Our results provide no evidence for the "CFS-attenuation-by-inattention" hypothesis, but show higher decoding accuracies for visible stimuli than for invisible stimuli. We discuss the implications of our findings for the important endeavor of trying to reconcile the divergent reports of unconscious processing under CFS.

Observers can voluntarily avoid reversals of an ambiguous, reversible figure, extending the duration of an intended percept. This is usually attributed to high-level, top-down attentional processes. However, voluntary control is limited. Reversals occur despite attempts to avoid them. In two experiments, observers demonstrated significant, but limited, voluntary control over Necker cube perception. Cube size and cube completeness, variables associated with stimulus-driven processes involving neural adaptation, influenced the frequency of reversals regardless of observers’ intentions. Results are consistent with the hybrid hypothesis that both top-down and bottom-up processes contribute to Necker-cube perception and support the hypothesis that the contribution of bottom-up processes is responsible for the limitation on voluntary control.

Mastery criteria can be applied to individual targets or stimuli organized into sets. Wong et al. (2021) and Wong and Fienup (2022) found that participants who received special education services learned sight words more rapidly when an individual target mastery criterion was applied. The current study replicated and extended these findings across novel skills. Five participants with ASD received tact or intraverbal training in Experiment 1, and 2 participants with ASD received auditory-visual conditional discrimination training (AVCD) in Experiment 2. In both experiments, mastery criteria were applied to targets and stimulus sets to compare sessions to mastery. Results showed the target mastery criterion required fewer sessions of tact training for 3 of 5 participants and AVCD training for both participants. However, overselection of stimuli occurred for 20% of AVCD mastered targets, suggesting a false positive for acquisition of those targets. Maintenance was similar across conditions and experiments.

Detecting the direction of motion underlies many visually guided behaviors, from reflexive eye movements to identifying and catching moving objects. A subset of motion sensitive cells are direction selective - responding strongly to motion in one direction and weakly to motion in other directions. In mammals, direction-selective cells are found throughout the visual system, including the retina, superior colliculus, and primary visual cortex. Direction selectivity maps are well characterized in the mouse retina, where the preferred directions of retinal direction-selective cells follow the projections of optic flow, generated by the movements animals make as they navigate their environment. Here, we synthesize recent findings implicating activity-dependent mechanisms in the development of retinal direction selectivity maps, with primary focus on studies in mice, and discuss the implications for the development of direction-selective responses in downstream visual areas.

Background: Exposure to 40 Hz stroboscopic light, for one hour a day, has previously been published as a potential treatment option for Alzheimer’s disease in animal models. However, exposure for an hour a day to 40 Hz stroboscopic light can be strenuous and examining other types of 40 Hz inducing stimuli is paramount if chronic treatment is wanted. Objective: A core assumption behind ensuring a therapeutic outcome is that the visual stimuli can induce 40 Hz gamma entrainment. Here, we examine whether a specific visual stimulus, 40 Hz invisible spectral flicker (ISF), can induce gamma entrainment and how it differs from both continuous light (CON) and 40 Hz stroboscopic light (STROBE). Methods: The study included non-simultaneous EEG-fMRI neuroimaging of 13 young healthy volunteers during light exposure. Each light condition (i.e., CON, ISF, or STROBE) was active for 30 seconds followed immediately by the next. Results: Entrainment of 40 Hz neural activity were significantly higher signal-to-noise ratio during exposure to ISF (mean: 3.03, 95% CI 2.07 to 3.99) and STROBE (mean: 12.04, 95% CI 10.18 to 13.87) compared to CON. Additionally STROBE had a higher entrainment than ISF (mean: 9.01, 95% CI 7.16 to 12.14). Conclusion: This study presents a novel method of 40 Hz entrainment using ISF. This enables the possibility of future randomized placebo-controlled clinical trials with acceptable double blinding due to the essentially imperceivable flicker, which is expected to substantially reduce discomfort compared to interventions with stroboscopic flicker.

In our daily lives, the visual system receives a plethora of visual information that competes for the brain's limited processing capacity. Nevertheless, not all visual information is useful for our cognitive, emotional, social, and ultimately survival purposes. Therefore, the brain employs mechanisms to select critical information and thereby optimizes its limited resources. Attention is the selective process that serves such a function. In particular, covert spatial attention - attending to a particular location in the visual field without eye movements - improves spatial resolution and paradoxically deteriorates temporal resolution. The neural correlates underlying these attentional effects still remainelusive. In this work, we tested a neural model's predictions that explain these phenomena based on interactions between channels with different spatiotemporal sensitivities - namely, the magnocellular (transient) and parvocellular (sustained) channels. More specifically, our model postulates that spatial attention enhances activities in the parvocellular pathway, thereby producing improved performance in spatial resolution tasks. However, the enhancement of parvocellular activities leads to decreased magnocellular activities due to parvo-magno inhibitory interactions. As a result, spatial attention hampers temporal resolution. We compared the predictions of the model to psychophysical data, and show that our model can account qualitatively and quantitatively for the effects of spatial attention on spatial and temporal acuity.

To evaluate the ability of handheld chromatic pupillometry to reveal and localise retinal neural dysfunction in diabetic patients with and without diabetic retinopathy (DR).This cross-sectional study included 82 diabetics (DM) and 93 controls (60.4 ± 8.4 years, 44.1% males). DM patients included those without (n = 25, 64.7 ± 6.3 years, 44.0% males) and with DR (n = 57, 60.3 ± 8.5 years, 64.9% males). Changes in horizontal pupil radius in response to blue (469 nm) and red (640 nm) light stimuli were assessed monocularly, in clinics, using a custom-built handheld pupillometer. Pupillometric parameters (phasic constriction amplitudes [predominantly from the outer retina], maximal constriction amplitudes [from the inner and outer retina] and post-illumination pupillary responses [PIPRs; predominantly from the inner retina]) were extracted from baseline-adjusted pupillary light response traces and compared between controls, DM without DR, and DR. Net PIPR was defined as the difference between blue and red PIPRs.Phasic constriction amplitudes to blue and red lights were decreased in DR compared to controls (p 0.001; p 0.001). Maximal constriction amplitudes to blue and red lights were decreased in DR compared to DM without DR (p 0.001; p = 0.02), and in DM without DR compared to controls (p 0.001; p = 0.005). Net PIPR was decreased in both DR and DM without DR compared to controls (p = 0.02; p = 0.03), suggesting a wavelength-dependent (and hence retinal) pupillometric dysfunction in diabetic patients with or without DR.Handheld chromatic pupillometry can reveal retinal neural dysfunction in diabetes, even without DR. Patients with DM but no DR displayed primarily inner retinal dysfunction, while patients with DR showed both inner and outer retinal dysfunction.

The neural and perceptual effects of attention were traditionally assumed to be sustained over time, but recent work suggests that covert attention rhythmically switches between objects at 3–8 Hz. Here I use simulations to demonstrate that the analysis approaches commonly used to test for rhythmic oscillations generate false positives in the presence of aperiodic temporal structure. I then propose two alternative analyses that are better able to discriminate between periodic and aperiodic structure in time series. Finally, I apply these alternative analyses to published datasets and find no evidence for behavioural rhythms in attentional switching after accounting for aperiodic temporal structure. The techniques presented here will help clarify the periodic and aperiodic dynamics of perception and of cognition more broadly.

The center-surround receptive field of retinal ganglion cells represents a fundamental concept for how the retina processes and encodes visual information. Yet, traditional approaches of using the receptive field as a linear filter to integrate light intensity over space often do not capture the responses of a ganglion cell to complex visual stimuli. Thus, models with local nonlinearities in subunits of the receptive field or with local temporal dynamics are emerging to better reflect relevant aspects of retinal circuitry and capture stimulus encoding. Here, we review recent efforts to identify such receptive-field substructure and evaluate its role in visual stimulus encoding. The concomitant development of new computational tools may pave the way toward a model-based, functional approach to retinal circuit analysis.

Pupil size changes constantly and is mainly determined by global luminance signals. In addition, the pupil responds to various cognitive and arousal processes, with larger pupil dilation observed in higher levels of cognitive or arousal processing. Although these task-evoked pupillary responses are extensively used in the pupil research, pupil analysis focusing on the frequency domain, particularly in the context of arousal and cognitive modulations, is less established. Fourier Transform method (FFT) has been used to understand the modulation of task difficulty on pupil oscillations. However, physiological signals are often characterized as non-linear and non-stationary waves, and the conventional spectral analytical method with linearity presumption is less appropriate to reveal modulation dynamics between time and frequency. Here, we used Hilbert-Huang Transform (HHT) to examine the time-frequency modulations on pupil size regulated by arousal, cognitive, and global luminance signals. Consistent with previous research, using FFT, higher spectral densities were obtained with lower luminance background. Moreover, higher spectral densities were found in the high emotional arousal condition. With HHT, we further demonstrated temporal changes on amplitude spectrum and inter-trial phase coherence (ITPC) in each intrinsic mode function (IMF), with stronger amplitudes in higher IMFs (i.e., low frequencies). Moreover, although global luminance, arousal and saccade preparation modulated pupil oscillatory responses, the modulation pattern in different IMFs was different. Together, our results demonstrated dynamics between the time and frequency domain on pupil oscillatory responses, highlighting the importance of examining the time-frequency interactions in the context of various pupil modulations.

Reduced cerebrovascular response to neuronal activation is observed in patients with neurodegenerative disease. In the present study, we examined the correlation between reduced cerebrovascular response to visual activation (ΔCBFVis.Act) and subclinical cognitive deficits in a human population of mid-sixties individuals without neurodegenerative disease. Such a correlation would suggest that impaired cerebrovascular function occurs before overt neurodegenerative disease. A total of 187 subjects (age 64–67 years) of the Metropolit Danish Male Birth Cohort participated in the study. ΔCBFVis.Act was measured using arterial spin labelling (ASL) MRI. ΔCBFVis.Act correlated positively with cognitive performance in: Global cognition (p = 0.046), paired associative memory (p = 0.025), spatial recognition (p = 0.026), planning (p = 0.016), simple processing speed (p p −5), and more complex processing speed (p −3), the latter two explaining approximately 11–13% of the variance. Reduced ΔCBFVis.Act was independent of brain atrophy. Our findings suggest that inhibited cerebrovascular response to neuronal activation is an early deficit in the ageing brain and associated with subclinical cognitive deficits. Cerebrovascular dysfunction could be an early sign of a trajectory pointing towards the development of neurodegenerative disease. Future efforts should elucidate if maintenance of a healthy cerebrovascular function can protect against the development of dementia.

Holographic structured illumination combined with optogenetics enables patterned stimulation of neurons and glial cells in an intact living brain. Moreover, in vivo functional imaging of cellular activity with recent advanced microscope technologies allows for visualization of the cellular responses during learning, emotion and cognition. Integrating these techniques can be used to verify the link between cell function and behavior output. However, there are technical limitations to stimulate multiple cells with high spatial and temporal resolution with available techniques of optogenetic stimulation. Here, we summarized a two-photon microscope combined with holographic system to stimulate multiple cells with high spatial and temporal resolution for living mice and their biological application.

Pre-stimulus electroencephalogram (EEG) oscillations, especially in the alpha range (8-13 Hz), can affect the sensitivity to temporal lags between modalities in multisensory perception. The effects of alpha power are often explained in terms of alpha's inhibitory functions, whereas effects of alpha frequency have bolstered theories of discrete perceptual cycles, where the length of a cycle, or window of integration, is determined by alpha frequency. Such studies typically employ visual detection paradigms with near-threshold or even illusory stimuli. It is unclear whether such results generalize to above-threshold stimuli. Here, we recorded EEG, while measuring temporal discrimination sensitivity in a temporal-order judgement task using above-threshold auditory and visual stimuli. We tested whether the power and instantaneous frequency of pre-stimulus oscillations predict audiovisual temporal discrimination sensitivity on a trial-by-trial basis. By applying a jackknife procedure to link single-trial pre-stimulus oscillatory power and instantaneous frequency to psychometric measures, we identified a posterior cluster where lower alpha power was associated with higher temporal sensitivity of audiovisual discrimination. No statistically significant relationship between instantaneous alpha frequency and temporal sensitivity was found. These results suggest that temporal sensitivity for above-threshold multisensory stimuli fluctuates from moment to moment and is indexed by modulations in alpha power.

A user-friendly steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) prefers no calibration for its target recognition algorithm, however, the existing calibration-free schemes perform still far behind their calibration-based counterparts. To tackle this issue, learning online from the subject's unlabeled data is investigated as a potential approach to boost the performance of the calibration-free SSVEP-based BCIs.An online adaptation scheme is developed to tune the spatial filters using the online unlabeled data from previous trials, and then developing the online adaptive canonical correlation analysis (OACCA) method.A simulation study on two public SSVEP datasets (Dataset I and II) with a total of 105 subjects demonstrated that the proposed online adaptation scheme can boost the CCA's averaged information transfer rate (ITR) from 94.60 to 158.87 bits/min in Dataset I and from 85.80 to 123.91 bits/min in Dataset II. Furthermore, in our online experiment it boosted the CCA's ITR from 55.81 bits/min to 95.73 bits/min. More importantly, this online adaptation scheme can be easily combined with any spatial filtering-based algorithms to achieve online learning.By online adaptation, the proposed OACCA performed much better than the calibration-free CCA, and comparable to the calibration-based algorithms.This work provides a general way for the SSVEP-based BCIs to learn online from unlabeled data and thus avoid calibration.

Evidence has shown that multisensory integration benefits to unisensory perception performance are asymmetric and that auditory perception performance can receive more multisensory benefits, especially when the attention focus is directed toward a task-irrelevant visual stimulus. At present, whether the benefits of semantically (in)congruent multisensory integration with modal-based attention for subsequent unisensory short-term memory (STM) retrieval are also asymmetric remains unclear. Using a delayed matching-to-sample paradigm, the present study investigated this issue by manipulating the attention focus during multisensory memory encoding. The results revealed that both visual and auditory STM retrieval reaction times were faster under semantically congruent multisensory conditions than under unisensory memory encoding conditions. We suggest that coherent multisensory representation formation might be optimized by restricted multisensory encoding and can be rapidly triggered by subsequent unisensory memory retrieval demands. Crucially, auditory STM retrieval is exclusively accelerated by semantically congruent multisensory memory encoding, indicating that the less effective sensory modality of memory retrieval relies more on the coherent prior formation of a multisensory representation optimized by modal-based attention.

Maintaining object correspondence among multiple moving objects is an essential task of the perceptual system in many everyday life activities. A substantial body of research has confirmed that observers are able to track multiple target objects amongst identical distractors based only on their spatiotemporal information. However, naturalistic tasks typically involve the integration of information from more than one modality, and there is limited research investigating whether auditory and audio-visual cues improve tracking. In two experiments, we asked participants to track either five target objects or three versus five target objects amongst similarly indistinguishable distractor objects for 14 s. During the tracking interval, the target objects bounced occasionally against the boundary of a centralised orange circle. A visual cue, an auditory cue, neither or both coincided with these collisions. Following the motion interval, the participants were asked to indicate all target objects. Across both experiments and both set sizes, our results indicated that visual and auditory cues increased tracking accuracy although visual cues were more effective than auditory cues. Audio-visual cues, however, did not increase tracking performance beyond the level of purely visual cues for both high and low load conditions. We discuss the theoretical implications of our findings for multiple object tracking as well as for the principles of multisensory integration.

Images of visual scenes comprise essential features important for visual cognition of the brain. The complexity of visual features lies at different levels, from simple artificial patterns to natural images with different scenes. It has been a focus of using stimulus images to predict neural responses. However, it remains unclear how to extract features from neuronal responses. Here we address this question by leveraging two-photon calcium neural data recorded from the visual cortex of awake macaque monkeys. With stimuli including various categories of artificial patterns and diverse scenes of natural images, we employed a deep neural network decoder inspired by image segmentation technique. Consistent with the notation of sparse coding for natural images, a few neurons with stronger responses dominated the decoding performance, whereas decoding of ar tificial patterns needs a large number of neurons. When natural images using the model pretrained on artificial patterns are decoded, salient features of natural scenes can be extracted, as well as the conventional category information. Altogether, our results give a new perspective on studying neural encoding principles using reverse-engineering decoding strategies.

OBJECTIVE: in this work, we aim to develop a more efficient visual motion-onset based Brain-computer interface (BCI). Brain-computer interfaces provide communication facilities that do not rely on the brain's usual pathways. Visual BCIs are based on changes in EEG activity in response to attended flashing or flickering targets. A less taxing way to encode such targets is with briefly moving stimuli, the onset of which elicits a lateralized EEG potential over the parieto-occipital scalp area called the motion-onset visual evoked potential (mVEP). METHODS: We recruited 21 healthy subjects for an experiment in which motion-onset stimulations translating leftwards (LT) or rightwards (RT) were encoding 9 displayed targets. We propose a novel algorithm that exploits the phase-shift between EEG electrodes to improve target decoding performance. We hereto extend the spatiotemporal beamformer (stBF) with a phase extracting procedure, leading to the phase-spatial beamformer (psBF). RESULTS: we show that psBF performs significantly better than the stBF (p < 0.001 for 1 and 2 stimulus repetitions and p < 0.01 for 3 to 5 stimulus repetitions), as well as the previously validated linear support-vector machines (p < 0.001 for 5 stimulus repetitions and p < 0.01 for 1,2 and 6 stimulus repetitions) and stepwise linear discriminant analysis decoders (p < 0.001 for all repetitions) when simultaneously addressing timing and translation direction. CONCLUSION: We provide evidence of decodability of joint direction and target in mVEP responses. SIGNIFICANCE: the described methods can aid in the development of a faster and more comfortable BCI based on mVEPs. ispartof: IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING vol:69 issue:5 pages:1802-1812 ispartof: location:United States status: published