Search

Virtual reality (VR) is a powerful tool for researchers due to its potential to study dynamic human behavior in highly naturalistic environments while retaining full control over the presented stimuli. Due to advancements in consumer hardware, VR devices are now very affordable and have also started to include technologies such as eye tracking, further extending potential research applications. Rendering engines such as Unity, Unreal, or Vizard now enable researchers to easily create complex VR environments. However, implementing the experimental design can still pose a challenge, and these packages do not provide out-of-the-box support for trial-based behavioral experiments. Here, we present a Python toolbox, designed to facilitate common tasks when developing experiments using the Vizard VR platform. It includes functionality for common tasks like creating, randomizing, and presenting trial-based experimental designs or saving results to standardized file formats. Moreover, the toolbox greatly simplifies continuous recording of eye and body movements using any hardware supported in Vizard. We further implement and describe a simple goal-directed reaching task in VR and show sample data recorded from five volunteers. The toolbox, example code, and data are all available on GitHub under an open-source license. We hope that our toolbox can simplify VR experiment development, reduce code duplication, and aid reproducibility and open-science efforts.

Allocentric and egocentric reference frames are used to determine the spatial position of action targets in reference to objects in the environment, i.e., landmarks (allocentric), or the observer (egocentric). Previous research investigated reference frames in isolation, for example, by shifting landmarks relative to the target and asking participants to reach to the remembered target location. Systematic reaching errors were found in the direction of the landmark shift and used as a proxy for allocentric spatial coding. Here we examined the interaction of both allocentric and egocentric reference frames by shifting the landmarksas well asthe observer. We asked participants to encode a 3D configuration of balls, and to reproduce this configuration from memory after a short delay followed by a landmark or observer shift. We also manipulated the number of landmarks to test its effect on the use of allocentric and egocentric reference frames. Shifting the observer resulted in larger configurational errors. In addition, an increase in the number of landmarks led to a stronger reliance on allocentric cues and a weaker contribution of egocentric cues. In sum, our results highlight the important role of egocentric cues for allocentric spatial coding in the context of memory-guided actions.New & NoteworthyObjects in our environment are coded relative to each other (allocentrically) and are thought to serve as independent and reliable cues (landmarks) in the context of unreliable egocentric signals. Contrary to this assumption, we demonstrate that egocentric cues alter the spatial memory of landmark configurations, which could reflect recently discovered interactions between allocentric and egocentric neural processing pathways. Further, additional landmarks lead to a higher contribution of allocentric and a lower contribution of egocentric cues.

Bowers et al. focus their criticisms on research that compares behavioral and brain data from the ventral stream with a class of deep neural networks for object recognition. While they are right to identify issues with current benchmarking research programs, they overlook a much more fundamental limitation of this literature: disregarding the importance of action and interaction for perception.

International audience; Everyday movements are guided by objects’ positions relative to other items in the scene (allocentric information) as well as by objects’ positions relative to oneself (egocentric information). Allocentric information can guide movements to the remembered positions of hidden objects, but is it also used when the object remains visible? To stimulate the use of allocentric information, the position of the participant’s finger controlled the velocity of a cursor that they used to intercept moving targets, so there was no one-to-one mapping between egocentric positions of the hand and cursor. We evaluated whether participants relied on allocentric information by shifting all task-relevant items simultaneously leaving their allocentric relationships unchanged. If participants rely on allocentric information they should not respond to this perturbation. However, they did. They responded in accordance with their responses to each item shifting independently, supporting the idea that fast guidance of ongoing movements primarily relies on egocentric information.

In everyday life, our brain constantly builds spatial representations of the objects surrounding us. Many studies have investigated the nature of these spatial representations. It is well established that we use allocentric information in real-time and memory-guided movements. Most studies relied on small-scale and static experiments, leaving it unclear whether similar paradigms yield the same results on a larger scale using dynamic objects. We created a virtual reality task that required participants to encode the landing position of a virtual ball thrown by an avatar. Encoding differed in the nature of the task in that it was either purely perceptual (“view where the ball landed while standing still”—Experiment 1) or involved an action (“intercept the ball with the foot just before it lands”—Experiment 2). After encoding, participants were asked to place a real ball at the remembered landing position in the virtual scene. In some trials, we subtly shifted either the thrower or the midfield line on a soccer field to manipulate allocentric coding of the ball’s landing position. In both experiments, we were able to replicate classic findings from small-scale experiments and to generalize these results to different encoding tasks (perception vs. action) and response modes (reaching vs. walking-and-placing). Moreover, we found that participants preferably encoded the ball relative to the thrower when they had to intercept the ball, suggesting that the use of allocentric information is determined by the encoding task by enhancing task-relevant allocentric information. Our findings indicate that results previously obtained from memory-guided reaching are not restricted to small-scale movements, but generalize to whole-body movements in large-scale dynamic scenes.

The 2-streams model of vision suggests that egocentric and allocentric reference frames are utilized by the dorsal and the ventral stream for real-time and memory-guided movements, respectively. Recent studies argue against such a strict functional distinction and suggest that real-time and memory-guided movements recruit the same spatial maps. In this study we focus on allocentric spatial coding and updating of targets by using landmark information in real-time and memory-guided reaching. We presented participants with a naturalistic scene which consisted of six objects on a table that served as potential reach targets. Participants were informed about the target object after scene encoding, and were prompted by a go cue to reach to its position. After target identification a brief air-puff was applied to the participant's right eye inducing an eye blink. During the blink the target object disappeared from the scene, and in half of the trials the remaining objects, that functioned as landmarks, were shifted horizontally in the same direction. We found that landmark shifts systematically influenced participants' reaching endpoints irrespective of whether the movements were controlled online based on available target information (real-time movement) or memory-guided based on remembered target information (memory-guided movement). Overall, the effect of landmark shift was stronger for memory-guided than real-time reaching. Our findings suggest that humans can encode and update reach targets in an allocentric reference frame for both real-time and memory-guided movements and show stronger allocentric coding when the movement is based on memory.

Adaptation to delays between actions and sensory feedback is important for efficiently interacting with our environment. Adaptation may rely on predictions of action-feedback pairing (motor-sensory component), or predictions of tactile-proprioceptive sensation from the action and sensory feedback of the action (inter-sensory component). Reliability of temporal information might differ across sensory feedback modalities (e.g. auditory or visual), influencing adaptation. Here, we investigated the role of motor-sensory and inter-sensory components on sensorimotor temporal recalibration for motor-auditory events (button press-tone) and motor-visual events (button press-Gabor patch). In the adaptation phase of the experiment, the motor action-feedback event pairs were presented with systematic temporal delays (0ms or 150ms). In the subsequent test phase, sensory feedback of the action were presented with variable delays. The participants were then asked whether this delay could be detected. To disentangle motor-sensory from inter-sensory component, we varied movements (active button press or passive depression of button) at adaptation and test. Our results suggest that motor-auditory recalibration is mainly driven by motor-sensory component, whereas motor-visual recalibration is mainly driven by inter-sensory component. Recalibration transferred from vision to audition, but not from audition to vision. These results indicate that motor-sensory and inter-sensory components of recalibration are weighted in a modality-dependent manner.

Tactile perception is impaired in a limb that is moving compared to when it is static. A possible mechanism that explains this phenomenon is an internal forward model that estimates future sensory states of the moving limb and suppresses associated feedback signals arising from that limb. Because sensorimotor estimations are based on an interplay of efferent and afferent feedback signals, the strength of tactile suppression may also depend on the relative utilization of sensory feedback from the moving limb. To test how the need to process somatosensory feedback influences movement-induced tactile suppression, we asked participants to perform reach-to-grasp movement of different demands: the target object was covered with materials of different frictional properties and the task was performed both with and without vision. As expected, participants performed the grasping movement more carefully when interacting with objects of low than high friction surfaces and when performing the task without vision. This denotes a greater need for somatosensory guidance of the digits to appropriately position the digits on the object. Accordingly, tactile suppression was weaker when grasping low than high friction objects, but only when grasping without vision. This suggests that movement-induced tactile suppression is modulated by grasping demands. Tactile suppression is downregulated when the need to process somatosensory feedback signals from that moving limb increases, like in situations when somatosensory input about the digit's state is the sole source of sensory information and when this information is particularly important for the task at hand.

Somatosensory signals on a moving limb are typically suppressed. This results mainly from a predictive mechanism that generates an efference copy, and attenuates the predicted sensory consequences of that movement. Sensory feedback is, however, important for movement control. Behavioral studies show that the strength of suppression on a moving limb increases during somatosensory reaching, when reach-relevant somatosensory signals from the target limb can be additionally used to plan and guide the movement, leading to increased reliability of sensorimotor predictions. It is still unknown how this suppression is neurally implemented. In this fMRI study, participants reached to a somatosensory (static finger) or an external target (touch-screen) without vision. To probe suppression, participants detected brief vibrotactile stimuli on their moving finger shortly before reach onset. As expected, sensitivity to probes was reduced during reaching compared to baseline (resting), and this suppression was stronger during somatosensory than external reaching. BOLD activation associated with suppression was also modulated by the reach target: relative to baseline, processing of probes during somatosensory reaching led to distinct BOLD deactivations in somatosensory regions (postcentral gyrus, supramarginal gyrus-SMG) whereas probes during external reaching led to deactivations in the cerebellum. In line with the behavioral results, we also found additional deactivations during somatosensory relative to external reaching in the supplementary motor area, a region linked with sensorimotor prediction. Somatosensory reaching was also linked with increased functional connectivity between the left SMG and the right parietal operculum along with the right anterior insula. We show that somatosensory processing on a moving limb is reduced when additional reach-relevant feedback signals from the target limb contribute to the movement, by down-regulating activation in regions associated with predictive and feedback processing.

When presented with a set of possible reach targets, the movement trajectory can reveal aspects of the underlying competition for action selection. Current goals and physical salience can affect the trajectory of reaching movements to be attracted towards a distractor. Some studies demonstrated that stimuli associated with reward can also cause an attraction when reaching towards the reward stimulus was previously rewarded and the reward stimulus was physically salient. Here we demonstrate that a non-salient stimulus that signals the availability of reward attracts reaching movements even when moving towards it was never necessary nor rewarded. Moreover, the attraction by reward is particularly evident with short-latency movements. We conclude that neither physical salience nor reinforcing the movement towards a stimulus is necessary for reward to gain priority in the selection for action.

For any type of goal-directed hand and eye movement, it is important to determine the position of the target. Though many of these movements are directed toward visual targets, humans also perform movements to targets derived by somatosensory information only, such as proprioceptive (sensory signals about static limb position), kinesthetic (sensory signals about limb movement), and tactile signals (sensory signals about touch on skin). In this study we investigated how each of these types of somatosensory information influences goal-directed hand and eye movements. Furthermore, we examined whether somatosensory target information has a differential influence on isolated and combined eye-hand movements. Participants performed right-hand reaching, eye, or coordinated eye-hand movements to their left index or middle fingers in the absence of any visual information. We varied somatosensory target information by allowing proprioceptive, proprioceptive-kinesthetic, proprioceptive-tactile, or proprioceptive-kinesthetic-tactile information. Reach endpoint precision was poorest when the target was derived by proprioceptive information only but improved when two different types of input were available. In addition, reach endpoints in conditions with kinesthetic target information were systematically shifted toward the direction of movement, while static somatosensory information decayed over time and led to systematic undershoots of the reach target location. In contrast to the effect on reaches, somatosensory information did not influence gaze endpoint accuracy or precision. When performing coordinated eye-hand movements reach accuracy and gaze endpoint precision improved, suggesting a bidirectional use of efferent information. We conclude that somatosensory target information influence endpoint control differently for goal-directed hand and eye movements to unseen targets.NEW & NOTEWORTHY A systematic investigation of contributions of different somatosensory modalities (proprioception, kinesthesia, tactile) for goal-directed movements is missing. Here we demonstrate that while eye movements are not affected by different types of somatosensory information, reach precision improves when two different types of information are available. Moreover, reach accuracy and gaze precision to unseen somatosensory targets improve when performing coordinated eye-hand movements, suggesting bidirectional contributions of efferent information in reach and eye movement control.

When reaching to a visual target, humans need to transform the spatial target representation into the coordinate system of their moving arm. It has been shown that increased uncertainty in such coordinate transformations, for instance, when the head is rolled toward one shoulder, leads to higher movement variability and influence movement decisions. However, it is unknown whether the brain incorporates such added variability in planning and executing movements. We designed an obstacle avoidance task in which participants had to reach with or without visual feedback of the hand to a visual target while avoiding collisions with an obstacle. We varied coordinate transformation uncertainty by varying head roll (straight, 30° clockwise, and 30° counterclockwise). In agreement with previous studies, we observed that the reaching variability increased when the head was tilted. Indeed, head roll did not influence the number of collisions during reaching compared with the head-straight condition, but it did systematically change the obstacle avoidance behavior. Participants changed the preferred direction of passing the obstacle and increased the safety margins indicated by stronger movement curvature. These results suggest that the brain takes the added movement variability during head roll into account and compensates for it by adjusting the reaching trajectories. NEW & NOTEWORTHY We show that changing body geometry such as head roll results in compensatory reaching behaviors around obstacles. Specifically, we observed head roll causes changed preferred movement direction and increased trajectory curvature. As has been shown before, head roll increases movement variability due to stochastic coordinate transformations. Thus these results provide evidence that the brain must consider the added movement variability caused by coordinate transformations for accurate reach movements.

Previous research on the spatiotemporal dynamics of exogenous and endogenous attentional allocation during saccade preparation yielded conflicting results. We hypothesize that this can be explained by the cueing type used to orient attention in a perceptual task. We investigated the time-course of attentional allocation as a function of cueing type (central vs peripheral), spatial congruency of the cued perceptual and saccade task locations, and cue validity in a dual-task paradigm. Participants performed a visual discrimination task during saccade preparation. We found that central and peripheral cues differentially affected the time-course of attentional allocation depending on spatial congruency and cue validity. Peripheral cues quickly and transiently oriented attention to the cued location. In the congruent condition, attention was maintained by the pre-saccadic attention shift, but declined in the spatially incongruent condition. Central cues slowly oriented attention to the cued location. In the congruent condition, attention was boosted by the pre-saccadic attention shift compared to a slower increase in the spatially incongruent condition. The pre-saccadic attention shift - the automatic and obligatory shift of attention to the saccade target - observed in the invalid spatially incongruent condition was not differentially affected by the cueing type orienting attention away from it. Our results suggest that exogenous and endogenous attention is dynamically and flexibly allocated to cued locations during saccade preparation while pre-saccadic attentional resources are progressively shifted to the saccade target irrespective of the cueing type. We argue that attentional selection for perception represents a partially independent process in contrast to the pre-saccadic attention shift.

Simultaneous eye and hand movements are highly coordinated and tightly coupled. This raises the question whether the selection of eye and hand targets relies on a shared attentional mechanism or separate attentional systems. Previous studies have revealed conflicting results by reporting evidence for both a shared as well as separate systems. Movement properties such as movement curvature can provide novel insights into this question as they provide a sensitive measure for attentional allocation during target selection. In the current study, participants performed simultaneous eye and hand movements to the same or different visual target locations. We show that both saccade and reaching movements curve away from the other effector’s target location when they are simultaneously performed to spatially distinct locations. We argue that there is a shared attentional mechanism involved in selecting eye and hand targets that may be found on the level of effector-independent priority maps. NEW & NOTEWORTHY Movement properties such as movement curvature have been widely neglected as important sources of information in investigating whether the attentional systems underlying target selection for eye and hand movements are separate or shared. We convincingly show that movement curvature is influenced by the other effector’s target location in simultaneous eye and hand movements to spatially distinct locations. Our results provide evidence for shared attentional systems involved in the selection of saccade and reach targets.

The current study investigated the role of automatic encoding and maintenance of remembered, past, and present visual distractors for reach movement planning. The previous research on eye movements showed that saccades curve away from locations actively kept in working memory and also from task-irrelevant perceptually present visual distractors, but not from task-irrelevant past distractors. Curvature away has been associated with an inhibitory mechanism resolving the competition between multiple active movement plans. Here, we examined whether reach movements underlie a similar inhibitory mechanism and thus show systematic modulation of reach trajectories when the location of a previously presented distractor has to be (a) maintained in working memory or (b) ignored, or (c) when the distractor is perceptually present. Participants performed vertical reach movements on a computer monitor from a home to a target location. Distractors appeared laterally and near or far from the target (equidistant from central fixation). We found that reaches curved away from the distractors located close to the target when the distractor location had to be memorized and when it was perceptually present, but not when the past distractor had to be ignored. Our findings suggest that automatically encoding present distractors and actively maintaining the location of past distractors in working memory evoke a similar response competition resolved by inhibition, as has been previously shown for saccadic eye movements.

Tactile stimuli on moving limbs are typically attenuated during reach planning and execution. This phenomenon has been related to internal forward models that predict the sensory consequences of a movement. Tactile suppression is considered to occur due to a match between the actual and predicted sensory consequences of a movement, which might free capacities to process novel or task-relevant sensory signals. Here, we examined whether and how tactile suppression depends on the relevance of somatosensory information for reaching. Participants reached with their left or right index finger to the unseen index finger of their other hand (body target) or an unseen pad on a screen (external target). In the body target condition, somatosensory signals from the static hand were available for localizing the reach target. Vibrotactile stimuli were presented on the moving index finger before or during reaching or in a separate no-movement baseline block, and participants indicated whether they detected a stimulus. As expected, detection thresholds before or during reaching were higher compared with baseline. Tactile suppression was also stronger for reaches to body targets than external targets, as reflected by higher detection thresholds and lower precision of detectability. Moreover, detection thresholds were higher when reaching with the left than with the right hand. Our results suggest that tactile suppression is modulated by position signals from the target limb that are required to reach successfully to the own body. Moreover, limb dominance seems to affect tactile suppression, presumably due to disparate uncertainty of feedback signals from the moving limb. NEW & NOTEWORTHY Tactile suppression on a moving limb has been suggested to release computational resources for processing other relevant sensory events. In the current study, we show that tactile sensitivity on the moving limb decreases more when reaching to body targets than external targets. This indicates that tactile perception can be modulated by allocating processing capacities to movement-relevant somatosensory information at the target location. Our results contribute to understanding tactile processing and predictive mechanisms in the brain.

The wide diversity of articles in this issue reveals an explosion of evidence for the mechanisms of prediction in the visual system. When thought of as visual priors, predictive mechanisms can be seen as tightly interwoven with incoming sensory data. Prediction is thus a fundamental and essential aspect not only of visual perception but of the actions that are guided by perception.

Exploring an object's shape by touch also renders information about its surface roughness. It has been suggested that shape and roughness are processed distinctly in the brain, a result based on comparing brain activation when exploring objects that differed in one of these features. To investigate the neural mechanisms of top‐down control on haptic perception of shape and roughness, we presented the same multidimensional objects but varied the relevance of each feature. Specifically, participants explored two objects that varied in shape (oblongness of cuboids) and surface roughness. They either had to compare the shape or the roughness in an alternative‐forced‐choice‐task. Moreover, we examined whether the activation strength of the identified brain regions as measured by functional magnetic resonance imaging (fMRI) can predict the behavioral performance in the haptic discrimination task. We observed a widespread network of activation for shape and roughness perception comprising bilateral precentral and postcentral gyrus, cerebellum, and insula. Task‐relevance of the object's shape increased activation in the right supramarginal gyrus (SMG/BA 40) and the right precentral gyrus (PreCG/BA 44) suggesting that activation in these areas does not merely reflect stimulus‐driven processes, such as exploring shape, but also entails top‐down controlled processes driven by task‐relevance. Moreover, the strength of the SMG/PreCG activation predicted individual performance in the shape but not in the roughness discrimination task. No activation was found for the reversed contrast (roughness > shape). We conclude that macrogeometric properties, such as shape, can be modulated by top‐down mechanisms whereas roughness, a microgeometric feature, seems to be processed automatically.

Prediction allows humans and other animals to prepare for future interactions with their environment. This is important in our dynamically changing world that requires fast and accurate reactions to external events. Knowing when and where an event is likely to occur allows us to plan eye, hand, and body movements that are suitable for the circumstances. Predicting the sensory consequences of such movements helps to differentiate between self-produced and externally generated movements. In this review, we provide a selective overview of experimental studies on predictive mechanisms in human vision for action. We present classic paradigms and novel approaches investigating mechanisms that underlie the prediction of events guiding eye and hand movements.

Tactile suppression refers to the phenomenon that tactile signals are attenuated during movement planning and execution when presented on a moving limb compared to rest. It is usually explained in the context of the forward model of movement control that predicts the sensory consequences of an action. Recent research suggests that aging increases reliance on sensorimotor predictions resulting in stronger somatosensory suppression. However, the mechanisms contributing to this age effect remain to be clarified. We measured age-related differences in tactile suppression during reaching and investigated the modulation by cognitive processes. A total of 23 younger (18-27 years) and 26 older (59-78 years) adults participated in our study. We found robust suppression of tactile signals when executing reaching movements. Age group differences corroborated stronger suppression in old age. Cognitive task demands during reaching, although overall boosting suppression effects, did not modulate the age effect. Across age groups, stronger suppression was associated with lower individual executive capacities. There was no evidence that baseline sensitivity had a prominent impact on the magnitude of suppression. We conclude that aging alters the weighting of sensory signals and sensorimotor predictions during movement control. Our findings suggest that individual differences in tactile suppression are critically driven by executive functions.

Brain lesion and stimulation studies have suggested posterior parietal cortex and the medial intraparietal sulcus in particular as a crucial hub for online movement error corrections. However, causal evidence for this is still sparse. Indeed, lesion studies are potentially confounded by compensatory reorganization mechanisms while brain stimulation studies have produced heterogeneous results when employing transcranial magnetic stimulation. Here we designed a new complementary paradigm using fMRI-guided high-definition transcranial direct current stimulation (HD-tDCS) of the left medial intraparietal sulcus (mIPS) together with regression-based mediation analysis to re-examine the causal role of mIPS in online reach corrections to jumping targets. We obtained two independent measures of stimulation-induced changes in brain activity by modeling current flow in the brain and through EEG recordings before and after HD-tDCS stimulation. Third, to quantify behavioral effects of HD-tDCS we computed movement curvature as a measure of online correction. We demonstrate that both of our measurements of brain activity were consistent with a polarity-specific modulation of the online correction for targets jumping to the contralateral side of the stimulation. Importantly, using a mediation analysis of the relationship between stimulation current and movement curvature suggests that the induced current modifies brain activity, which then leads to the observed behavioral changes. This unique combination of methods and analysis thus provides complementary evidence for the crucial role of the posterior parietal cortex in online error correction, while at the same time setting a new methodological standard with respect to the causal influence of transcranial direct current stimulation.New & NoteworthyTranscranial direct current stimulation (tDCS) is an interesting and potentially useful tool for asking causal scientific questions and design clinical treatments. With our unique combination of highly accurate fMRI guided stimulation, current forward modeling, EEG recordings before and after the stimulation and behavioral changes we could unravel the causal structure of tDCS. Our approach naturally deals with the variability of tDCS results, increasing its potential usefulness as a tool for research and clinical applications.

The perception of tactile stimuli presented on a moving hand is systematically suppressed. Such suppression has been attributed to the limited capacity of the brain to process task-irrelevant sensory information. Here, we examined whether humans do not only suppress movement-irrelevant but also enhance in parallel movement-relevant tactile signals when performing a goal-directed reaching movement. Participants reached to either a visual (LED) or somatosensory target (thumb or index finger of their unseen static hand) and discriminated 2 simultaneously presented tactile stimuli: a reference stimulus on the little finger of their static hand and a comparison stimulus on the index finger of their moving hand. Thus, during somatosensory reaching the location of the reference stimulus was task-relevant. Tactile suppression, as reflected by the increased points-of-subjective-equality (PSE) and just-noticeable-differences (JND), was stronger during reaching to somatosensory than visual targets. In Experiment 2, we presented the reference stimulus at a task-irrelevant location (sternum) and found similar suppression for somatosensory and visual reaching. This suggests that participants enhanced the sensation of the reference stimulus at the target hand during somatosensory reaching in Experiment 1. This suggestion was confirmed in Experiment 3 using a detection task in which we found lower detection thresholds on the target hand during somatosensory but not during visual reaching. We postulate that humans can flexibly modulate their tactile sensitivity by suppressing movement-irrelevant and enhancing movement-relevant signals in parallel when executing a reaching movement. (PsycINFO Database Record