Search

Parietal and frontal cortex are involved in saccade generation, and their output signals modify visual signals throughout cortex. Local signals associated with these interactions are well described, but their large-scale progression and network dynamics are unknown. Here, we combined source localized electroencephalography (EEG) and graph theory analysis (GTA) to understand how saccades and presaccadic visual stimuli interactively alter cortical network dynamics in humans. Twenty-one participants viewed 1–3 vertical/horizontal grids, followed by grid with the opposite orientation just before a horizontal saccade or continued fixation. EEG signals from the presaccadic interval (or equivalent fixation period) were used for analysis. Source localization-through-time revealed a rapid frontoparietal progression of presaccadic motor signals and stimulus-motor interactions, with additional band-specific modulations in several frontoparietal regions. GTA analysis revealed a saccade-specific functional network with major hubs in inferior parietal cortex (alpha) and the frontal eye fields (beta), and major saccade-repetition interactions in left prefrontal (theta) and supramarginal gyrus (gamma). This network showed enhanced segregation, integration, synchronization, and complexity (compared with fixation), whereas stimulus repetition interactions reduced synchronization and complexity. These cortical results demonstrate a widespread influence of saccades on both regional and network dynamics, likely responsible for both the motor and perceptual aspects of saccades.

Nonhuman primates have been used extensively to study eye-head coordination and eye-hand coordination, but the combination—eye-head-hand coordination—has not been studied. Our goal was to determine whether reaching influences eye-head coordination (and vice versa) in rhesus macaques. Eye, head, and hand motion were recorded in two animals with search coil and touch screen technology, respectively. Animals were seated in a customized “chair” that allowed unencumbered head motion and reaching in depth. In the reach condition, animals were trained to touch a central LED at waist level while maintaining central gaze and were then rewarded if they touched a target appearing at 1 of 15 locations in a 40° × 20° (visual angle) array. In other variants, initial hand or gaze position was varied in the horizontal plane. In similar control tasks, animals were rewarded for gaze accuracy in the absence of reach. In the Reach task, animals made eye-head gaze shifts toward the target followed by reaches that were accompanied by prolonged head motion toward the target. This resulted in significantly higher head velocities and amplitudes (and lower eye-in-head ranges) compared with the gaze control condition. Gaze shifts had shorter latencies and higher velocities and were more precise, despite the lack of gaze reward. Initial hand position did not influence gaze, but initial gaze position influenced reach latency. These results suggest that eye-head coordination is optimized for visually guided reach, first by quickly and accurately placing gaze at the target to guide reach transport and then by centering the eyes in the head, likely to improve depth vision as the hand approaches the target. NEW & NOTEWORTHY Eye-head and eye-hand coordination have been studied in nonhuman primates but not the combination of all three effectors. Here we examined the timing and kinematics of eye-head-hand coordination in rhesus macaques during a simple reach-to-touch task. Our most novel finding was that (compared with hand-restrained gaze shifts) reaching produced prolonged, increased head rotation toward the target, tending to center the binocular field of view on the target/hand.

A remarkable feature of primate behavior is the ability to predict future events based on past experience and current sensory cues. To understand how the brain plans movements in the presence of unstable cues, we recorded gaze-related activity in the frontal cortex of two monkeys engaged in a quasi-predictable cue-conflict task. Animals were trained to look toward remembered visual targets in the presence of a landmark that shifted with fixed amplitude but randomized direction. As simulated by a probabilistic model based on known physiology/behavior, gaze end points assumed a circular distribution around the target, mirroring the possible directions of the landmark shift. This predictive strategy was reflected in frontal cortex activity (especially supplementary eye fields), which anticipated future gaze distributions before the actual landmark shift. In general, these results implicate prefrontal cortex in the predictive integration of environmental cues and their learned statistical properties to mitigate spatial uncertainty.

Parietal and frontal cortex are involved in saccade generation, but their output signals also modify visual signals throughout cortex. These signals produce well-documented behavioral phenomena (saccades, saccadic suppression, various perisaccadic perceptual distortions) but their underlying influence on cortical network dynamics is not known. Here, we combined electroencephalography (EEG) with frequency-dependent source localization and graph theory analysis (GTA) to understand how saccades and pre-saccadic visual stimuli interactively alter cortical network dynamics in humans. 21 participants viewed series of 1-3 vertical or horizontal grids, followed by grid with the opposite orientation just before a horizontal saccade or continued fixation. EEG signals from the presaccadic interval (cue + 200ms, or equivalent fixation period) were used for source localization. Source localization (saccade – fixation) identified bilateral dorsomedial frontoparietal activity across frequency bands, whereas stimulus repetition produced band-specific modulations in left prefrontal, posterior parietal, and central-superior frontal and/or parietal cortex, with significant saccade-repetition interactions in frontal and parietal regions. GTA analysis revealed a saccade-specific functional network with major hubs in inferior parietal cortex (alpha) and the frontal eye fields (beta), and major saccade-repetition interactions in left prefrontal (theta) and supramarginal gyrus (gamma). Overall, quantitative measures of whole-brain network topology and dynamics (segregation, integration, synchronization, complexity) were enhanced during the presaccadic interval, but repetition interactions reduced synchronization and complexity. These results show that presaccadic signals have widespread, coherent influence on cortical network dynamics, likely responsible for both saccade production and the perceptual phenomena associated with saccades.HighlightsSource localization & graph theory were used to analyze presaccadic EEG signalsPresaccadic signals produced band-specific modulations/hubs in parietofrontal cortexFrontal/parietal eye fields showed extensive functional connectivity across all lobesPresaccadic stimulus repetition further modulated functional network connectivitySaccades and repetition both influenced network clustering, integration, & complexity

Sensorimotor transformations require spatiotemporal coordination of signals, that is, through both time and space. For example, the gaze control system employs signals that are time‐locked to various sensorimotor events, but the spatial content of these signals is difficult to assess during ordinary gaze shifts. In this review, we describe the various models and methods that have been devised to test this question, and their limitations. We then describe a new method that can (a) simultaneously test between all of these models during natural, head‐unrestrained conditions, and (b) track the evolving spatial continuum from target (T) to future gaze coding (G, including errors) through time. We then summarize some applications of this technique, comparing spatiotemporal coding in the primate frontal eye field (FEF) and superior colliculus (SC). The results confirm that these areas preferentially encode eye‐centered, effector‐independent parameters, and show—for the first time in ordinary gaze shifts—a spatial transformation between visual and motor responses from T to G coding. We introduce a new set of spatial models (T‐G continuum) that revealed task‐dependent timing of this transformation: progressive during a memory delay between vision and action, and almost immediate without such a delay. We synthesize the results from our studies and supplement it with previous knowledge of anatomy and physiology to propose a conceptual model where cumulative transformation noise is realized as inaccuracies in gaze behavior. We conclude that the spatiotemporal transformation for gaze is both local (observed within and across neurons in a given area) and distributed (with common signals shared across remote but interconnected structures)., In this Review we describe challenges the literature has faced to study spatial transformations in the visuomotor pathway required for orienting to visual stimuli during natural head‐unrestrained gaze behavior. We summarize our relatively new method developed to address this question and review our recent studies that apply it to neural activity in two key structures in gaze control, the frontal eye field and superior colliculus. We synthesize these results and supplement it with the previous literature to propose a new conceptual model where cumulative transformation noise, within local and distributed circuits, is realized as inaccuracies in gaze behavior.

Posterior parietal cortex (PPC), specifically right supramarginal gyrus, is involved in transsaccadic memory of object orientation for both perception and action. Here, we investigated whether PPC is involved in transsaccadic memory of other features, namely spatial frequency. We employed a functional magnetic resonance imaging paradigm where participants briefly viewed a grating stimulus with a specific spatial frequency that later reappeared with the same or different frequency, after a saccade or continuous fixation. Post-saccadic frequency modulation activated a region in the right hemisphere spanning medial PPC (ventral precuneus) and posterior cingulate cortex. Importantly, the site of peak precuneus activation showed saccade-specific feature modulation (compared to fixation) and task-specific saccade modulation (compared to a saccade localizer task). Psychophysiological interaction analysis revealed functional connectivity between this precuneus site and the precentral gyrus (M1), lingual gyrus (V1/V2), and medial occipitotemporal sulcus. This differed from the transsaccadic orientation network, perhaps because spatial frequency signaled changes in object identity. Overall, this experiment supports a general role for PPC in transsaccadic vision, but suggests that different networks are employed for specific features.

Various models (e.g., scalar, state-dependent network, and vector models) have been proposed to explain the global aspects of time perception, but they have not been tested against specific visual phenomena like perisaccadic time compression and novel stimulus time dilation. Here, in two separate experiments (N=31), we tested how the perceived duration of a novel stimulus is influenced by 1) a simultaneous saccade, in combination with 2) a prior series of repeated stimuli in human participants. This yielded a novel behavioral interaction: pre-saccadic stimulus repetition neutralizes perisaccadic time compression. We then tested these results against simulations of the above models. Our data yielded low correlations against scalar model simulations, high but non-specific correlations for our feedforward neural network, and correlations that were both high and specific for a vector model based on identity of objective and subjective time. These results demonstrate the power of global time perception models in explaining disparate empirical phenomena and suggest that subjective time has a similar essence to time’s physical vector.

Eye-centered (egocentric) and landmark-centered (allocentric) visual signals influence spatial cognition, navigation and goal-directed action, but the neural mechanisms that integrate these signals for motor control are poorly understood. A likely candidate for ego/allocentric integration in the gaze control system is the supplementary eye fields (SEF), a mediofrontal structure with high-level ‘executive’ functions, spatially tuned visual/motor response fields, and reciprocal projections with the frontal eye fields (FEF). To test this hypothesis, we trained two head-unrestrained monkeys (Macaca mulatta) to saccade toward a remembered visual target in the presence of a visual landmark that shifted during the delay, causing gaze end points to shift partially in the same direction. 256 SEF neurons were recorded, including 68 with spatially tuned response fields. Model fits to the latter established that, like the FEF and superior colliculus, spatially tuned SEF responses primarily showed an egocentric (eye-centered) target-to-gaze position transformation. However, the landmark shift influenced this default egocentric transformation: during the delay, motor neurons (with no visual response) showed a transient but unintegrated shift (i.e., not correlated with the target-to-gaze transformation), whereas during the saccade-related burst visuomotor neurons showed an integrated shift (i.e., correlated with the target-to-gaze transformation). This differed from our simultaneous FEF recordings (Bharmauria et al., 2020), which showed a transient shift in visuomotor neurons, followed by an integrated response in all motor responses. Based on these findings and past literature, we propose that prefrontal cortex incorporates landmark-centered information into a distributed, eye-centered target-to-gaze transformation through a reciprocal prefrontal circuit. Significance Statement It is thought that the brain integrates egocentric (self-centered) and allocentric (landmark-centered) visual signals to generate accurate goal-directed movements, but the neural mechanism is not known. Here, by shifting a visual landmark while recording frontal cortex activity in awake behaving monkeys, we show that the supplementary eye fields (SEF) incorporates landmark-centered information (in memory and motor activity) when it transforms target location into future gaze position commands. We propose a circuit model in which the SEF provides control signals to implement an integrated gaze command in the frontal eye fields (Bharmauria et al., 2020). Taken together, these experiments explain normal ego/allocentric integration and might suggest rehabilitation strategies for neurological patients who have lost one of these visual mechanisms.

Coordinated reach-to-grasp movements are often accompanied by rapid eye movements (saccades) that displace the desired object image relative to the retina. Parietal cortex compensates for this by updating reach goals relative to current gaze direction, but its role in the integration of oculomotor and visual orientation signals for updating grasp plans is unknown. Based on a recent perceptual experiment, we hypothesized that inferior parietal cortex (specifically supramarginal gyrus [SMG]) integrates saccade and visual signals to update grasp plans in additional intraparietal/superior parietal regions. To test this hypothesis in humans (7 females, 6 males), we used a functional magnetic resonance paradigm, where saccades sometimes interrupted grasp preparation toward a briefly presented object that later reappeared (with the same/different orientation) just before movement. Right SMG and several parietal grasp regions, namely, left anterior intraparietal sulcus and bilateral superior parietal lobule, met our criteria for transsaccadic orientation integration: they showed task-dependent saccade modulations and, during grasp execution, they were specifically sensitive to changes in object orientation that followed saccades. Finally, SMG showed enhanced functional connectivity with both prefrontal saccade regions (consistent with oculomotor input) and anterior intraparietal sulcus/superior parietal lobule (consistent with sensorimotor output). These results support the general role of parietal cortex for the integration of visuospatial perturbations, and provide specific cortical modules for the integration of oculomotor and visual signals for grasp updating. SIGNIFICANCE STATEMENT How does the brain simultaneously compensate for both external and internally driven changes in visual input? For example, how do we grasp an unstable object while eye movements are simultaneously changing its retinal location? Here, we used fMRI to identify a group of inferior parietal (supramarginal gyrus) and superior parietal (intraparietal and superior parietal) regions that show saccade-specific modulations during unexpected changes in object/grasp orientation, and functional connectivity with frontal cortex saccade centers. This provides a network, complementary to the reach goal updater, that integrates visuospatial updating into grasp plans, and may help to explain some of the more complex symptoms associated with parietal damage, such as constructional ataxia.

Visual landmarks influence spatial cognition, navigation and goal-directed behavior, but their influence on visual coding in sensorimotor systems is poorly understood. We hypothesized that visual responses in frontal cortex gaze control areas encode potential targets in an intermediate gaze-centered / landmark-centered reference frame that might depend on specific target-landmark configurations rather than a global mechanism. We tested this hypothesis by recording neural activity in the frontal eye fields (FEF) and supplementary eye fields (SEF) while head-unrestrained macaques engaged in a memory-delay gaze task. Visual response fields (the area of visual space where targets modulate activity) were tested for each neuron in the presence of a background landmark placed at one of four oblique configurations relative to the target stimulus. 102 of 312 FEF and 43 of 256 SEF neurons showed spatially tuned response fields in this task. We then fit these data against a mathematical continuum between a gaze-centered model and a landmark-centered model. When we pooled data across the entire dataset for each neuron, our response field fits did not deviate significantly from the gaze-centered model. However, when we fit response fields separately for each target-landmark configuration, the best fits shifted (mean 37% / 40%) toward landmark-centered coding in FEF / SEF respectively. This confirmed an intermediate gaze / landmark-centered mechanism dependent on local (configuration-dependent) interactions. Overall, these data show that external landmarks influence prefrontal visual responses, likely helping to stabilize gaze goals in the presence of variable eye and head orientations.

Cervical dystonia (CD) is characterized by abnormal twisting and turning of the head with associated head oscillations. It is the most common form of dystonia, which is a third most common movement disorder. Despite frequent occurrence there is paucity in adequate therapy, much of which is attributed to its uncertain pathophysiology. Recently we proposed a unifying network model highlighting the role of head neural integrator (hNI) for the pathophysiology of CD. According to our hypothesis the CD is due to abnormal output of hNI; the latter itself is not affected but its dysfunction is secondary to abnormal feedback. We hypothesized that asymmetry in the feedback to hNI is associated with severity in CD; the feedback asymmetry is greater in CD with lateralized head postures, such as turning of head in yaw plane (torticollis) or roll plane (laterocollis). The hypothesis also specifies that feedback to hNI-cerebellum, proprioception, and basal ganglia outflow (pallidus) are connected in a network; thus asymmetry is distributed through the feedback network. In 15 CD patients undergoing deep brain stimulation (DBS) surgery, with their informed consent, we used the opportunity to collect single unit neural responses and local field potential from the globus pallidus to measure whether feedback to hNI is asymmetric. Using machine learning algorithms developed to analyze single unit data, we found: (1) globus pallidus interna (GPi) firing rate, discharge pattern and gamma oscillation were asymmetric in patients with robust torticollis; (2) there was no asymmetry in these parameters in retrocollis; and (3) in those patients with oppositely directed laterocollis and torticollis. Firing rate was higher in GPi cells ipsilateral to the direction of head rotation; the asymmetry was more pronounced in tonic cells compared to burst neurons. In addition to confirming that CD is associated with an asymmetric pallidal activity, our data showed that neuronal asymmetry correlated with the degree of involuntary head turning. We propose that asymmetric pallidal activity results in asymmetric feedback to hNI causing its dysfunction.

Cervical dystonia (CD) is characterized by abnormal twisting and turning of the head with associated head oscillations. It is the most common form of dystonia, which is a third most common movement disorder. Despite frequent occurrence there is paucity in adequate therapy, much of which is attributed to its uncertain pathophysiology. Recently we proposed a unifying network model highlighting the role of head neural integrator (hNI) for the pathophysiology of CD. According to our hypothesis the CD is due to abnormal output of hNI; the latter itself is not affected but its dysfunction is secondary to abnormal feedback. We hypothesized that asymmetry in the feedback to hNI is associated with severity in CD; the feedback asymmetry is greater in CD with lateralized head postures, such as turning of head in yaw plane (torticollis) or roll plane (laterocollis). The hypothesis also specifies that feedback to hNI—cerebellum, proprioception, and basal ganglia outflow (pallidus) are connected in a network; thus asymmetry is distributed through the feedback network. In 15 CD patients undergoing deep brain stimulation (DBS) surgery, with their informed consent, we used the opportunity to collect single unit neural responses and local field potential from the globus pallidus to measure whether feedback to hNI is asymmetric. Using machine learning algorithms developed to analyze single unit data, we found: (1) globus pallidus interna (GPi) firing rate, discharge pattern and gamma oscillation were asymmetric in patients with robust torticollis; (2) there was no asymmetry in these parameters in retrocollis; and (3) in those patients with oppositely directed laterocollis and torticollis. Firing rate was higher in GPi cells ipsilateral to the direction of head rotation; the asymmetry was more pronounced in tonic cells compared to burst neurons. In addition to confirming that CD is associated with an asymmetric pallidal activity, our data showed that neuronal asymmetry correlated with the degree of involuntary head turning. We propose that asymmetric pallidal activity results in asymmetric feedback to hNI causing its dysfunction.

We previously reported that visuomotor activity in the superior colliculus (SC)--a key midbrain structure for the generation of rapid eye movements--preferentially encodes target position relative to the eye (Te) during low-latency head-unrestrained gaze shifts (DeSouza et al., 2011). Here, we trained two monkeys to perform head-unrestrained gaze shifts after a variable post-stimulus delay (400-700 ms), to test whether temporally separated SC visual and motor responses show different spatial codes. Target positions, final gaze positions and various frames of reference (eye, head, and space) were dissociated through natural (untrained) trial-to-trial variations in behaviour. 3D eye and head orientations were recorded, and 2D response field data were fitted against multiple models by use of a statistical method reported previously (Keith et al., 2009). Of 60 neurons, 17 showed a visual response, 12 showed a motor response, and 31 showed both visual and motor responses. The combined visual response field population (n = 48) showed a significant preference for Te, which was also preferred in each visual subpopulation. In contrast, the motor response field population (n = 43) showed a preference for final (relative to initial) gaze position models, and the Te model was statistically eliminated in the motor-only population. There was also a significant shift of coding from the visual to motor response within visuomotor neurons. These data confirm that SC response fields are gaze-centred, and show a target-to-gaze transformation between visual and motor responses. Thus, visuomotor transformations can occur between, and even within, neurons within a single frame of reference and brain structure.

Sensorimotor integration involves feedforward and reentrant processing of sensory input. Grasp-related motor activity precedes and is thought to influence visual object processing. Yet, while the importance of reentrant feedback is well established in perception, the top-down modulations for action and the neural circuits involved in this process have received less attention. Do action-specific intentions influence the processing of visual information in the human cortex? Using a cue-separation fMRI paradigm, we found that action-specific instruction (manual alignment vs. grasp) influences the cortical processing of object orientation several seconds after the object had been viewed. This influence occurred as early as in the primary visual cortex and extended to ventral and dorsal visual stream areas. Importantly, this modulation was unrelated to non-specific action planning. Further, the primary visual cortex showed stronger functional connectivity with frontal-parietal areas and the inferior temporal cortex during the delay following orientation processing for align than grasping movements, strengthening the idea of reentrant feedback from dorsal visual stream areas involved in action. To our knowledge, this is the first demonstration that intended manual actions have such an early, pervasive, and differential influence on the cortical processing of vision.

Effector-specific cortical mechanisms can be difficult to establish using fMRI, in part because low time resolution might temporally conflate different signals related to target representation, motor planning, and motor execution. Here, we used an event-related fMRI protocol and a cue-separation paradigm to temporally separate these three major sensorimotor stages for saccades vs. reaches. In each trial, subjects (N=12) 1) briefly viewed a target 4-7° left or right of midline fixation on a touchscreen, followed by an 8 second delay (effector-independent target memory phase), 2) were instructed by an auditory cue to perform a reach or a saccade, followed by a second delay of 8 seconds (effector-specific planning phase), and finally 3) were prompted to move by reaching-to-touch or performing a saccade towards the remembered target (effector-specific execution phase). Our analysis of saccade and reach activation (vs. a non-spatial control task) revealed modest effector-agnostic target memory activity (left AG, bilateral mIPS) followed by independent effector parietofrontal sites and time courses during the motor components of the task, specifically: more medial (pIPS, mIPS, M1, and PMd) activity during both reach planning and execution, and more lateral (mIPS, AG, and FEF) activity only during saccade execution. These motor activations were bilateral, with a left (contralateral) preference for reach. A conjunction analysis revealed that left mIPS and right AG, PCu, SPOC, FEF/PMv and LOTC showed activation for both saccades and reaches. Overall, effector-preference contrasts (reach vs. saccade) revealed significantly more parietofrontal activation for reaches than saccades during both planning and execution, with the exception of FEF. Cross-correlation of reach, saccade, and reach-saccade activation through time revealed correlated activation both within and across effectors in each hemisphere, but with a tendency toward higher correlations in the right hemisphere, especially between the eye and hand. These results demonstrate substantially independent but temporally correlated cortical networks for human eye, hand, and eye-hand control, that follow explicit spatiotemporal rules for effector-specific timing, medial-lateral distribution, and hemispheric lateralization.

Transsaccadic integration of visual features can operate in various frames of reference, but the corresponding neural mechanisms have not been differentiated. A recent fMRIa (adaptation) study identified two cortical regions in supramarginal gyrus (SMG) and extrastriate cortex that were sensitive to transsaccadic changes in stimulus orientation (Dunkley et al., 2016). Here, we modified this paradigm to identify the neural correlates for transsaccadic comparison of object orientations in: 1) Spatially Congruent (SC), 2) Retinally Congruent (RC) or 3) Spatially Incongruent (SI)) coordinates. Functional data were recorded from 12 human participants while they observed a grating (oriented 45° or 135°) before a saccade, and then judged whether a post-saccadic grating (in SC, RC, or SI configuration) had the same or different orientation. Our analysis focused on areas that showed a significant repetition suppression (Different > Same) or repetition enhancement (Same > Different) BOLD responses. Several cortical areas were significantly modulated in all three conditions: premotor/motor cortex (likely related to the manual response), and posterior-middle intraparietal sulcus. In the SC condition, uniquely activated areas included left SMG and left lateral occipitotemporal gyrus (LOtG). In the RC condition, unique areas included inferior frontal gyrus and the left lateral BA 7. In the SI condition, uniquely activated areas included the frontal eye field, medial BA 7, and right LOtG. Overall, the SC results were significantly different from both RC and SI. These data suggest that different cortical networks are used to compare pre- and post-saccadic orientation information, depending on the spatial nature of the task.Significance StatementEvery time one makes a saccade, the brain must compare and integrate stored visual information with new information. It has recently been shown that ‘transsaccadic integration’ of visual object orientation involves specific areas within parietal and occipital cortex (Dunkley et al., 2016). Here, we show that this pattern of cortical activation also depends on the spatial nature of the task: when the visual object is fixed relative to space, the eye, or relative to neither space nor the eye, different frontal, parietal, and occipital regions are engaged. More generally, these findings suggest that different aspects of trans-saccadic integration flexibly employ different cortical networks.

Movement planning involves transforming the sensory signals into a command in motor coordinates. Surprisingly, the real-time dynamics of sensorimotor transformations at the whole brain level remain unknown, in part due to the spatiotemporal limitations of fMRI and neurophysiological recordings. Here, we used magnetoencephalography (MEG) during pro-/anti-wrist pointing to determine (1) the cortical areas involved in transforming visual signals into appropriate hand motor commands, and (2) how this transformation occurs in real time, both within and across the regions involved. We computed sensory, motor, and sensorimotor indices in 16 bilateral brain regions for direction coding based on hemispherically lateralized de/synchronization in the α (7–15 Hz) and β (15–35 Hz) bands. We found a visuomotor progression, from pure sensory codes in ‘early’ occipital-parietal areas, to a temporal transition from sensory to motor coding in the majority of parietal-frontal sensorimotor areas, to a pure motor code, in both the α and β bands. Further, the timing of these transformations revealed a top-down pro/anti cue influence that propagated ‘backwards’ from frontal through posterior cortical areas. These data directly demonstrate a progressive, real-time transformation both within and across the entire occipital-parietal-frontal network that follows specific rules of spatial distribution and temporal order.

Gaze saccades –rapid shifts of the eyes and head toward a goal— have provided fundamental insights into the neural control of movement. For example, it has been shown that the superior colliculus (SC) transforms a visual target (T) code to future gaze (G) location commands after a memory delay. However, this transformation has not been observed in ‘reactive’ saccades made directly to a stimulus, so its contribution to normal gaze behavior is unclear. Here, we tested this using a quantitative measure of the spatial continuum between T and G coding based on variable gaze errors. We demonstrate that a rapid T-G transformation occurs between SC visual and motor responses during reactive saccades, evenwithinvisuomotor cells, with a continuous spatiotemporal shift in coding occurring in cell types (visual, visuomotor, motor). We further show that the primary determinant of this spatial code was not the intrinsic visual-motor index of different cells or populations, but rather thetimingof the response inallcells. These results suggest that the SC provides a rapid spatiotemporal transformation for normal gaze saccades, that its motor responses contribute to variable gaze errors, and that those errors arise from a noisy spatiotemporal transformation involving all SC neurons.Significance StatementOculomotor studies have demonstrated visuomotor transformations in structures like the superior colliculus with the use of trained behavioral manipulations, like the memory delay and antisaccades tasks, but it is not known how this happens during normal saccades. Here, using a spatiotemporal model fitting method based on endogenous gaze errors in ‘reactive’ gaze saccades, we show that the superior colliculus provides a rapid spatiotemporal transformation from target to gaze coding that involves visual, visuomotor, and motor neurons. This technique demonstrates that SC spatial codes are not fixed, and may provide a quantitative biomarker for assessing the health of sensorimotor transformations.

A fundamental question in sensorimotor control concerns the transformation of spatial signals from the retina into eye and head motor commands required for accurate gaze shifts. Here, we investigated these transformations by identifying the spatial codes embedded in visually evoked and movement-related responses in the frontal eye fields (FEFs) during head-unrestrained gaze shifts. Monkeys made delayed gaze shifts to the remembered location of briefly presented visual stimuli, with delay serving to dissociate visual and movement responses. A statistical analysis of nonparametric model fits to response field data from 57 neurons (38 with visual and 49 with movement activities) eliminated most effector-specific, head-fixed, and space-fixed models, but confirmed the dominance of eye-centered codes observed in head-restrained studies. More importantly, the visual response encoded target location, whereas the movement response mainly encoded the final position of the imminent gaze shift (including gaze errors). This spatiotemporal distinction between target and gaze coding was present not only at the population level, but even at the single-cell level. We propose that an imperfect visual-motor transformation occurs during the brief memory interval between perception and action, and further transformations from the FEF's eye-centered gaze motor code to effector-specific codes in motor frames occur downstream in the subcortical areas.

Humans typically make several rapid eye movements (saccades) per second. It is thought that visual working memory can retain and spatially integrate three to four objects or features across each saccade but little is known about this neural mechanism. Previously we showed that transcranial magnetic stimulation (TMS) to the posterior parietal cortex and frontal eye fields degrade trans-saccadic memory of multiple object features (Prime, Vesia, & Crawford, 2008, Journal of Neuroscience, 28(27), 6938-6949; Prime, Vesia, & Crawford, 2010, Cerebral Cortex, 20(4), 759-772.). Here, we used a similar protocol to investigate whether dorsolateral prefrontal cortex (DLPFC), an area involved in spatial working memory, is also involved in trans-saccadic memory. Subjects were required to report changes in stimulus orientation with (saccade task) or without (fixation task) an eye movement in the intervening memory interval. We applied single-pulse TMS to left and right DLPFC during the memory delay, timed at three intervals to arrive approximately 100ms before, 100ms after, or at saccade onset. In the fixation task, left DLPFC TMS produced inconsistent results, whereas right DLPFC TMS disrupted performance at all three intervals (significantly for presaccadic TMS). In contrast, in the saccade task, TMS consistently facilitated performance (significantly for left DLPFC/perisaccadic TMS and right DLPFC/postsaccadic TMS) suggesting a dis-inhibition of trans-saccadic processing. These results are consistent with a neural circuit of trans-saccadic memory that overlaps and interacts with, but is partially separate from the circuit for visual working memory during sustained fixation.

The location of a remembered reach target can be encoded in egocentric and/or allocentric reference frames. Cortical mechanisms for egocentric reach are relatively well described, but the corresponding allocentric representations are essentially unknown. Here, we used an event-related fMRI design to distinguish human brain areas involved in these two types of representation. Our paradigm consisted of three tasks with identical stimulus display but different instructions: egocentric reach (remember absolute target location), allocentric reach (remember target location relative to a visual landmark), and a nonspatial control, color report (report color of target). During the delay phase (when only target location was specified), the egocentric and allocentric tasks elicited widely overlapping regions of cortical activity (relative to the control), but with higher activation in parietofrontal cortex for egocentric task and higher activation in early visual cortex for allocentric tasks. In addition, egocentric directional selectivity (target relative to gaze) was observed in the superior occipital gyrus and the inferior occipital gyrus, whereas allocentric directional selectivity (target relative to a visual landmark) was observed in the inferior temporal gyrus and inferior occipital gyrus. During the response phase (after movement direction had been specified either by reappearance of the visual landmark or a pro-/anti-reach instruction), the parietofrontal network resumed egocentric directional selectivity, showing higher activation for contralateral than ipsilateral reaches. These results show that allocentric and egocentric reach mechanisms use partially overlapping but different cortical substrates and that directional specification is different for target memory versus reach response.

Relatively little is known about the role of proprioception in eye-hand coordination. In a previous article (Ren et al. J Neurophysiol 96:1464-1477, 2006), we observed anisotropic (direction-dependent) saccade overshoots to hand-held targets during active, but not passive hand movements. We hypothesized that these errors arose from a limb-centered anisotropic efference copy which was transformed (uncompensated) into head coordinates for saccade control. Here, we tested this hypothesis and the role of head orientation signals in this transformation, by dissociating limb coordinates from head coordinates. Twelve human subjects made saccades to hand-held targets actively placed at eight radial locations on a frontally placed table in a dark room with four conditions: (1) right hand (body and head centered), (2) left hand (body and head centered), (3) right hand (head tilted counter-clockwise), (4) right hand (head tilted clockwise). In condition 1, we observed the same anisotropic pattern of overshooting errors-approximately along the axis of the forearm-that we reported previously. Overall, these amplitude errors were much smaller for the left hand. However, the anisotropic pattern was observed for both hands, but reversed symmetrically between the right versus left hand. Head tilt did not cause any systematic errors in saccade direction. Moreover, during head tilt, the anisotropic amplitude errors-while showing some distortions-did not rotate with the head. These findings suggest that transformations of somatosensory information into oculomotor coordinates account for head orientation, and that the anisotropic amplitude errors in hand-guided saccades arise in limb coordinates.

Posterior parietal cortex (PPC) has been implicated in the integration of visual and proprioceptive information for the planning of action. We previously reported that single-pulse transcranial magnetic stimulation (TMS) over dorsal–lateral PPC perturbs the early stages of spatial processing for memory-guided reaching. However, our data did not distinguish whether TMS disrupted the reach goal or the internal estimate of initial hand position needed to calculate the reach vector. To test between these hypotheses, we investigated reaching in six healthy humans during left and right parietal TMS while varying visual feedback of the movement. We reasoned that if TMS were disrupting the internal representation of hand position, visual feedback from the hand might still recalibrate this signal. We tested four viewing conditions: 1) final vision of hand position; 2) full vision of hand position; 3) initial and final vision of hand position; and 4) middle and final vision of hand position. During the final vision condition, left parietal stimulation significantly increased endpoint variability, whereas right parietal stimulation produced a significant leftward shift in both visual fields. However, these errors significantly decreased with visual feedback of the hand during both planning and control stages of the reach movement. These new findings demonstrate that 1) visual feedback of hand position during the planning and early execution of the reach can recalibrate the perturbed signal and, importantly, and 2) TMS over dorsal–lateral PPC does not disrupt the internal representation of the visual goal, but rather the reach vector, or more likely the sense of initial hand position that is used to calculate this vector.

Dorsal posterior parietal cortex (PPC) has been implicated through single-unit recordings, neuroimaging data, and studies of brain-damaged humans in the spatial guidance of reaching and pointing movements. The present study examines the causal effect of single-pulse transcranial magnetic stimulation (TMS) over the left and right dorsal posterior parietal cortex during a memory-guided “reach-to-touch” movement task in six human subjects. Stimulation of the left parietal hemisphere significantly increased endpoint variability, independent of visual field, with no horizontal bias. In contrast, right parietal stimulation did not increase variability, but instead produced a significantly systematic leftward directional shift in pointing (contralateral to stimulation site) in both visual fields. Furthermore, the same lateralized pattern persisted with left-hand movement, suggesting that these aspects of parietal control of pointing movements are spatially fixed. To test whether the right parietal TMS shift occurs in visual or motor coordinates, we trained subjects to point correctly to optically reversed peripheral targets, viewed through a left–right Dove reversing prism. After prism adaptation, the horizontal pointing direction for a given visual target reversed, but the direction of shift during right parietal TMS did not reverse. Taken together, these data suggest that induction of a focal current reveals a hemispheric asymmetry in the early stages of the putative spatial processing in PPC. These results also suggest that a brief TMS pulse modifies the output of the right PPC in motor coordinates downstream from the adapted visuomotor reversal, rather than modifying the upstream visual coordinates of the memory representation.

Transsaccadic integration (TSI) refers to the perceptual integration of visual information collected across separate gaze fixations. Current theories of TSI disagree on whether it relies solely on visual algorithms or also uses extra-retinal signals. We designed a task in which subjects had to rely on internal oculomotor signals to synthesize remembered stimulus features presented within separate fixations. Using a mouse-controlled pointer, subjects estimated the intersection point of two successively presented bars, in the dark, under two conditions: Saccade task (bars viewed in separate fixations) and Fixation task (bars viewed in one fixation). Small, but systematic biases were observed in both intersection tasks, including position-dependent vertical undershoots and order-dependent horizontal biases. However, the magnitude of these errors was statistically indistinguishable in the Saccade and Fixation tasks. Moreover, part of the errors in the Saccade task were dependent on saccade metrics, showing that egocentric oculomotor signals were used to fuse remembered location and orientation features across saccades. We hypothesize that these extra-retinal signals are normally used to reduce the computational load of calculating visual correspondence between fixations. We further hypothesize that TSI may be implemented within dynamically updated recurrent feedback loops that interconnect a common eye-centered map in occipital cortex with both the "dorsal" and "ventral" streams of visual analysis.

Item does not contain fulltext Eye-hand coordination is geometrically complex. To compute the location of a visual target relative to the hand, the brain must consider every anatomical link in the chain from retinas to fingertips. Here we focus on the first three links, studying how the brain handles information about the angles of the two eyes and the head. It is known that people, even in darkness, reach more accurately when the eye looks toward the target, rather than right or left of it. We show that reaching is also impaired when the binocular fixation point is displaced from the target in depth: reaching becomes not just sloppy, but systematically inaccurate. Surprisingly, though, in normal Gaze-On-Target reaching we found no strong correlations between errors in aiming the eyes and hand onto the target site. We also asked people to reach when the head was not facing the target. When the eyes were on-target, people reached accurately, but when gaze was off-target, performance degraded. Taking all these findings together, we suggest that the brain's computational networks have learned the complex geometry of reaching for well-practiced tasks, but that the networks are poorly calibrated for less common tasks such as Gaze-Off-Target reaching.

To point or reach to a visual target, you need to know its direction relative to your shoulder. That direction can be computed from the retinal image, if the brain also knows the orientation of the eyeball, head, and clavicle. To be geometrically exact, the neural computation would have to involve rotary operations.1 When the eye was turned 30° up, for instance, the brain would take the locations of all objects relative to the eye and rotate them 30° down to find the locations relative to the head. Some theories2,3,5–7 suggest that the brain might approximate that rotation by a simpler vector addition, merely shifting all retinal locations by the same vector. But that strategy would lead to marked errors in some situations. FIGURE 1A shows five earthfixed, horizontal arcs wrapped around a cylinder centered on an eyeball. In front of the eye are two spheres and 90° to the right are two cubes, one sphere-cube pair at eye level and the other 30° up. When the eye fixates the eye-level sphere, the retinal image of the eye-level cube falls on the eye’s horizontal meridian (FIG. 1B). But when the eye fixates the sphere at 30° up, the retinal image of the corresponding cube falls well below the horizontal meridian (FIG. 1C). That is, both pairs of spheres and cubes are horizontally separated, but in one case their retinal images are not, owing to the rotation of the eye. If the brain tried to compute these objects’ locations relative to the head or the shoulder simply by shifting all their retinal locations by a fixed vector, it would misestimate the elevations of at least some objects, and would misaim its eye and arm movements. Here we study whether the visuomotor transformation for arm movements uses a vector-additive strategy or correctly accounts for the rotary geometry of retinal projection.

Eye-hand coordination requires the brain to integrate visual information with the continuous changes in eye, head, and arm positions. This is a geometrically complex process because the eyes, head, and shoulder have different centers of rotation. As a result, head rotation causes the eye to translate with respect to the shoulder. The present study examines the consequences of this geometry for planning accurate arm movements in a pointing task with the head at different orientations. When asked to point at an object, subjects oriented their arm to position the fingertip on the line running from the target to the viewing eye. But this eye-target line shifts when the eyes translate with each new head orientation, thereby requiring a new arm pointing direction. We confirmed that subjects do realign their fingertip with the eye-target line during closed-loop pointing across various horizontal head orientations when gaze is on target. More importantly, subjects also showed this head-position–dependent pattern of pointing responses for the same paradigm performed in complete darkness. However, when gaze was not on target, compensation for these translations in the rotational centers partially broke down. As a result, subjects tended to overshoot the target direction relative to current gaze; perhaps explaining previously reported errors in aiming the arm to retinally peripheral targets. These results suggest that knowledge of head position signals and the resulting relative displacements in the centers of rotation of the eye and shoulder are incorporated using open-loop mechanisms for eye-hand coordination, but these translations are best calibrated for foveated, gaze-on-target movements.

Item does not contain fulltext Grasping behaviors require the selection of grasp-relevant object dimensions, independent of overall object size. Previous neuroimaging studies found that the intraparietal cortex processes object size, but it is unknown whether the graspable dimension (i.e., grasp axis between selected points on the object) or the overall size of objects triggers activation in that region. We used functional magnetic resonance imaging adaptation to investigate human brain areas involved in processing the grasp-relevant dimension of real 3-dimensional objects in grasping and viewing tasks. Trials consisted of 2 sequential stimuli in which the object's grasp-relevant dimension, its global size, or both were novel or repeated. We found that calcarine and extrastriate visual areas adapted to object size regardless of the grasp-relevant dimension during viewing tasks. In contrast, the superior parietal occipital cortex (SPOC) and lateral occipital complex of the left hemisphere adapted to the grasp-relevant dimension regardless of object size and task. Finally, the dorsal premotor cortex adapted to the grasp-relevant dimension in grasping, but not in viewing, tasks, suggesting that motor processing was complete at this stage. Taken together, our results provide a complete cortical circuit for progressive transformation of general object properties into grasp-related responses. 15 p.

In the 19th century, Donders observed that only one three-dimensional eye orientation is used for each gaze direction. Listing's law further specifies that the full set of eye orientation vectors forms a plane, whereas the equivalent Donders' law for the head, the Fick strategy, specifies a twisted two-dimensional range. Surprisingly, despite considerable research and speculation, the biological reasons for choosing one such range over another remain obscure. In the current study, human subjects performed head-free gaze shifts between visual targets while wearing pinhole goggles. During fixations, the head orientation range still obeyed Donders' law, but in most subjects, it immediately changed from the twisted Fick-like range to a flattened Listing-like range. Further controls showed that this was not attributable to loss of binocular vision or increased range of head motion, nor was it attributable to blocked peripheral vision; when subjects pointed a helmet-mounted laser toward targets (a task with goggle-like motor demands but normal vision), the head followed Listing's law even more closely. Donders' law of the head only broke down (in favor of a “minimum-rotation strategy”) when head motion was dissociated from gaze. These behaviors could not be modeled using current “Donders' operators” but were readily simulated nonholonomically, i.e., by modulating head velocity commands as a function of position and task. We conclude that the gaze control system uses such velocity rules to shape Donders' law on a moment-to-moment basis, not primarily to satisfy perceptual or anatomic demands, but rather for motor optimization; the Fick strategy optimizes the role of the head as a platform for eye movement, whereas Listing's law optimizes rapid control of the eye (or head) as a gaze pointer.

Establishing a coherent internal reference frame for visuospatial representation and maintaining the integrity of this frame during eye movements are thought to be crucial for both perception and motor control. A stable headcentric representation could be constructed by internally comparing retinal signals with eye position. Alternatively, visual memory traces could be actively remapped within an oculocentric frame to compensate for each eye movement. We tested these models by measuring errors in manual pointing (in complete darkness) toward briefly flashed central targets during three oculomotor paradigms; subjects pointed accurately when gaze was maintained on the target location (control paradigm). However, when steadily fixating peripheral locations (static paradigm), subjects exaggerated the retinal eccentricity of the central target by 13.4 ± 5.1%. In the key “dynamic” paradigm, subjects briefly foveated the central target and then saccaded peripherally before pointing toward the remembered location of the target. Our headcentric model predicted accurate pointing (as seen in the control paradigm) independent of the saccade, whereas our oculocentric model predicted misestimation (as seen in the static paradigm) of an internally shifted retinotopic trace. In fact, pointing errors were significantly larger than were control errors (p≤ 0.003) and were indistinguishable (p≥ 0.25) from the static paradigm errors. Scatter plots of pointing errors (dynamic vs static paradigm) for various final fixation directions showed an overall slope of 0.97, contradicting the headcentric prediction (0.0) and supporting the oculocentric prediction (1.0). Varying both fixation and pointing-target direction confirmed that these errors were a function of retinotopically shifted memory traces rather than eye position per se. To reconcile these results with previous pointing experiments, we propose a “conversion-on-demand” model of visuomotor control in which multiple visual targets are stored and rotated (noncommutatively) within the oculocentric frame, whereas only select targets are transformed further into head- or bodycentric frames for motor execution.

The frontal eye fields (FEFs) participate in both working memory and sensorimotor transformations for saccades, but their role in integrating these functions through time remains unclear. Here, we tracked FEF spatial codes through time using a novel analytic method applied to the classic memory-delay saccade task. Three-dimensional recordings of head-unrestrained gaze shifts were made in two monkeys trained to make gaze shifts toward briefly flashed targets after a variable delay (450-1500 ms). A preliminary analysis of visual and motor response fields in 74 FEF neurons eliminated most potential models for spatial coding at the neuron population level, as in our previous study (Sajad et al., 2015). We then focused on the spatiotemporal transition from an eye-centered target code (T; preferred in the visual response) to an eye-centered intended gaze position code (G; preferred in the movement response) during the memory delay interval. We treated neural population codes as a continuous spatiotemporal variable by dividing the space spanningTandGinto intermediateT–Gmodels and dividing the task into discrete steps through time. We found that FEF delay activity, especially in visuomovement cells, progressively transitions fromTthrough intermediateT–Gcodes that approach, but do not reach,G. This was followed by a final discrete transition from these intermediateT–Gdelay codes to a “pure”Gcode in movement cells without delay activity. These results demonstrate that FEF activity undergoes a series of sensory–memory–motor transformations, including a dynamically evolving spatial memory signal and an imperfect memory-to-motor transformation.