Search

During their first months of life, infants learn to coordinate their perceptions and actions across different modalities. For example, eye-hand coordination relies on combining visual and proprioceptive sensory inputs for controlling eye and hand movements. What drives the development and calibration of such coordination? Here, we put forward a multimodal hierarchical extension of the Active Efficient Coding framework to learn a simple form of eye-hand coordination. By learning to actively compress visual and proprioceptive inputs into a combined multimodal representation, our embodied infant model learns to make eye movements to track an object held in its hand. We find that the abstract multimodal representation improves the tracking accuracy, but only if it emerges after the establishment of the single-modality systems. This suggests the existence of a 'less-is-more' effect for the development of coordinated multimodal sensorimotor behaviors. © 2023 IEEE.

This dissertation explores how vision is used to guide interactions with 2-D computer-generated targets, with the goal of furthering our understanding of how perception and action interact during visually guided action. Study 1: Participants used their right hands to grasp square targets that either remained stationary or travelled horizontally across the screen. Fixations and digit placement favoured the near side of stationary targets presented at non-central positions, and the trailing side of moving targets, suggesting participants minimized the energy required to grasp the stationary targets and selected ‘safer’ contact points when the target was moving. Gaze and grasp positions were shifted rightward when grasping target shapes that discouraged digit placement near the horizontal midline, suggesting participants tried to avoid obstructing the view of the target when grasping. Study 2: Fixations and grasp positions were compared when grasping 2-D targets versus 3-D versions of these targets. Fixations and digit placement were comparable when grasping 2-D and 3-D stimuli positioned in the center of the display. When grasping non-central stimuli however, participants fixated and placed their digits at more ‘stable’ locations near the 3-D object’s midline compared to locations biased toward the near side of the 2-D targets. Intended manipulation of each stimulus type produced similar adjustments in fixation and digit placement, suggesting participants may have attributed physical properties to the 2-D targets in response to the manipulative task demands. Study 3: The Ebbinghaus illusion was used to explore how the perceived size of an on-screen target influences cursor movements toward that target. Participants’ perceptual size judgments, but not accuracy or movement time, were influenced by the illusion. However, the illusion affected cursor trajectories toward the target, suggesting the illusion may have influenced the planning and early stages of the cursor mov

Petanque sport is a sport that has 11 competition numbers, such as: single man, single woman, double man, double woman, double mix, triple man, triple woman, triple mix 1 woman 2 man, triple mix 2 woman 1 man, shooting men, shooting woman, team. In the game of petanque there are two throwing techniques, is called pointing and shooting. Shooting is the most important part of the petanque game. The success of shooting must also be considered, namely: ball grip (technique in holding bosi), body position towards the target (body alignment with the target), static balance of the legs, low body position and leaning forward, release the ball (release the ball) follow through. The purpose of this research was to determine the relationship between eye-hand coordination and concentration on the shooting results of UKM Petanque UIR athletes. The type of this research is multiple correlation. The population and sample in this research were athletes from UKM Petanque UIR, amounting to 16 people. The research instrument used was a test of hand eye coordination, a concentration test and a shoting petanque test. The data analysis technique used is the multiple correlation value test. Based on the results of the research that has been done, a conclusion can be drawn as follows (1). There is a relationship between eye-hand coordination and the shooting results of UKM Petanque UIR athletes with a value of tcount = 3.61 > ttable = 3.61, (2). There is a relationship between concentration and the shooting results of UKM Petanque UIR athletes with a value of tcount = 2.68 > ttable = 1.753, (3) There is a relationship between eye-hand coordination and concentration with the shooting results of UKM Petanque UIR athletes with a value of tcount = 3.76 > ttable = 1.753.

Petanque sport is a sport that has 11 competition numbers, such as: single man, single woman, double man, double woman, double mix, triple man, triple woman, triple mix 1 woman 2 man, triple mix 2 woman 1 man, shooting men, shooting woman, team. In the game of petanque there are two throwing techniques, is called pointing and shooting. Shooting is the most important part of the petanque game. The success of shooting must also be considered, namely: ball grip (technique in holding bosi), body position towards the target (body alignment with the target), static balance of the legs, low body position and leaning forward, release the ball (release the ball) follow through. The purpose of this research was to determine the relationship between eye-hand coordination and concentration on the shooting results of UKM Petanque UIR athletes. The type of this research is multiple correlation. The population and sample in this research were athletes from UKM Petanque UIR, amounting to 16 people. The research instrument used was a test of hand eye coordination, a concentration test and a shoting petanque test. The data analysis technique used is the multiple correlation value test. Based on the results of the research that has been done, a conclusion can be drawn as follows (1). There is a relationship between eye-hand coordination and the shooting results of UKM Petanque UIR athletes with a value of tcount = 3.61 > ttable = 3.61, (2). There is a relationship between concentration and the shooting results of UKM Petanque UIR athletes with a value of tcount = 2.68 > ttable = 1.753, (3) There is a relationship between eye-hand coordination and concentration with the shooting results of UKM Petanque UIR athletes with a value of tcount = 3.76 > ttable = 1.753.

This dissertation explores how vision is used to guide interactions with 2-D computer-generated targets, with the goal of furthering our understanding of how perception and action interact during visually guided action. Study 1: Participants used their right hands to grasp square targets that either remained stationary or travelled horizontally across the screen. Fixations and digit placement favoured the near side of stationary targets presented at non-central positions, and the trailing side of moving targets, suggesting participants minimized the energy required to grasp the stationary targets and selected ‘safer’ contact points when the target was moving. Gaze and grasp positions were shifted rightward when grasping target shapes that discouraged digit placement near the horizontal midline, suggesting participants tried to avoid obstructing the view of the target when grasping. Study 2: Fixations and grasp positions were compared when grasping 2-D targets versus 3-D versions of these targets. Fixations and digit placement were comparable when grasping 2-D and 3-D stimuli positioned in the center of the display. When grasping non-central stimuli however, participants fixated and placed their digits at more ‘stable’ locations near the 3-D object’s midline compared to locations biased toward the near side of the 2-D targets. Intended manipulation of each stimulus type produced similar adjustments in fixation and digit placement, suggesting participants may have attributed physical properties to the 2-D targets in response to the manipulative task demands. Study 3: The Ebbinghaus illusion was used to explore how the perceived size of an on-screen target influences cursor movements toward that target. Participants’ perceptual size judgments, but not accuracy or movement time, were influenced by the illusion. However, the illusion affected cursor trajectories toward the target, suggesting the illusion may have influenced the planning and early stages of the cursor mov

Fitts’ law is very robust and used extensively in human-computer interaction (HCI). It predicts the movement time (MT) of pointing tasks under various conditions. However, the impact of visual inversion on hand movement performance has not been quantitatively investigated. This is really of essence as it provides a means to “unlearn” what we know and thus investigate how the eye and hand coordinate works. This thesis comprises three experiments with three different factors to investigate their impact as well as the underlying mechanism of aimed movement. The first and the third experiments were related to visual inversion. In order to determine the dominant factor of distal-pointing, the second experiment was designed. The first experiment focused on the impact of eye-hand incompatibility and investigated the substructure of the complete movemenet to investigate the presence of a ballistic region when task difficulty is low. For this purpose, MT was separated into initiation time (IT), distance covering time (DCT), and acquisition time (AT). The AT gives a better separation of the ballistic and visually controlled regions on the index of difficulty (ID) than movement time indicating the existence of a ballistic region. The second experiment focused on pointing at different depths. The visual angles of movement amplitude and target width are better predictors of movement time than the linear dimensions and depth does not have a significant impact. With increasing depth, hand and arm tremor effects increase. The third experiment noted that the individual kinematic perception is correlated to one’s MT and people do not need to see the hand during the whole movement when aiming. What we need is a window nearer to the target. The duration and endpoint variability of the first ballistic submovement of the first and the third experiment were compared and that helped explain the performance difference among the different conditions. This validates that different visual cond

Sensory-motor learning in humans and other animals is largely an unsupervised, adaptive, and active process. Exteroceptive and proprioceptive modalities are crucial for motor action generation. Generated actions then alter the perception of the outside world. This forms an action-perception loop that can be modeled by the active efficient coding framework. In this thesis, we propose self-calibrating models based on the active efficient coding framework to autonomously learn the sensorimotor control for tracking and localization. First, humans learn multisensory eye-hand coordination starting from infancy without supervision. In the past work, visuomotor mapping was often learned using either fiducial markers or pre-defined feature descriptors. We propose an architecture to combine visual and proprioceptive cues to learn an eye-hand coordination task autonomously. Using the iCub simulator we demonstrate that the robots can self-calibrate themselves to track the end effector. The visual sensory representation and visuomotor coupling emerge during learning while the robot babbles its arm like an infant does in early development. The evolved eye control policy has characteristics that are qualitatively similar to human oculomotor plant responses. Second, the brain frequently computes the sensory consequence of motor actions. For instance, the primary visual cortex was found to encode the mismatch between the actual and predicted visual flow in the retina. Inspired partly by these experiments, we propose a predictive multisensory integration architecture to learn the robot end-effector tracking task. We demonstrate that the use of prediction in multisensory integration enables the agent to incorporate the information from proprioceptive and visual cues better. The robot develops the ability to perform smooth pursuit-like eye movements to track its hand, both in the presence and absence of visual input, and to track exteroceptive visual motions. Third, we get inspiration

When tracking targets moving in various directions with one's eyes, horizontal components of pursuit are more precise than vertical ones. Is this because horizontal target motion is predicted better or because horizontal movements of the eyes are controlled more precisely? When tracking a visual target with the hand, the eyes also track the target. We investigated whether the directional asymmetries that have been found during isolated eye movements are also present during such manual tracking, and if so, whether individual participants' asymmetry in eye movements is accompanied by a similar asymmetry in hand movements. We examined the data of 62 participants who used a joystick to track a visual target with a cursor. The target followed a smooth but unpredictable trajectory in two dimensions. Both the mean gaze-target distance and the mean cursor-target distance were about 20% larger in the vertical direction than in the horizontal direction. Gaze and cursor both followed the target with a slightly longer delay in the vertical than in the horizontal direction, irrespective of the target's trajectory. The delays of gaze and cursor were correlated, as were their errors in tracking the target. Gaze clearly followed the target rather than the cursor, so the asymmetry in both eye and hand movements presumably results from better predictions of the target's horizontal than of its vertical motion. Altogether this study speaks for the presence of anisotropic predictive processes that are shared across effectors.

When tracking targets moving in various directions with one's eyes, horizontal components of pursuit are more precise than vertical ones. Is this because horizontal target motion is predicted better or because horizontal movements of the eyes are controlled more precisely? When tracking a visual target with the hand, the eyes also track the target. We investigated whether the directional asymmetries that have been found during isolated eye movements are also present during such manual tracking, and if so, whether individual participants' asymmetry in eye movements is accompanied by a similar asymmetry in hand movements. We examined the data of 62 participants who used a joystick to track a visual target with a cursor. The target followed a smooth but unpredictable trajectory in two dimensions. Both the mean gaze-target distance and the mean cursor-target distance were about 20% larger in the vertical direction than in the horizontal direction. Gaze and cursor both followed the target with a slightly longer delay in the vertical than in the horizontal direction, irrespective of the target's trajectory. The delays of gaze and cursor were correlated, as were their errors in tracking the target. Gaze clearly followed the target rather than the cursor, so the asymmetry in both eye and hand movements presumably results from better predictions of the target's horizontal than of its vertical motion. Altogether this study speaks for the presence of anisotropic predictive processes that are shared across effectors.

Sensory-motor learning in humans and other animals is largely an unsupervised, adaptive, and active process. Exteroceptive and proprioceptive modalities are crucial for motor action generation. Generated actions then alter the perception of the outside world. This forms an action-perception loop that can be modeled by the active efficient coding framework. In this thesis, we propose self-calibrating models based on the active efficient coding framework to autonomously learn the sensorimotor control for tracking and localization. First, humans learn multisensory eye-hand coordination starting from infancy without supervision. In the past work, visuomotor mapping was often learned using either fiducial markers or pre-defined feature descriptors. We propose an architecture to combine visual and proprioceptive cues to learn an eye-hand coordination task autonomously. Using the iCub simulator we demonstrate that the robots can self-calibrate themselves to track the end effector. The visual sensory representation and visuomotor coupling emerge during learning while the robot babbles its arm like an infant does in early development. The evolved eye control policy has characteristics that are qualitatively similar to human oculomotor plant responses. Second, the brain frequently computes the sensory consequence of motor actions. For instance, the primary visual cortex was found to encode the mismatch between the actual and predicted visual flow in the retina. Inspired partly by these experiments, we propose a predictive multisensory integration architecture to learn the robot end-effector tracking task. We demonstrate that the use of prediction in multisensory integration enables the agent to incorporate the information from proprioceptive and visual cues better. The robot develops the ability to perform smooth pursuit-like eye movements to track its hand, both in the presence and absence of visual input, and to track exteroceptive visual motions. Third, we get inspiration

Fitts’ law is very robust and used extensively in human-computer interaction (HCI). It predicts the movement time (MT) of pointing tasks under various conditions. However, the impact of visual inversion on hand movement performance has not been quantitatively investigated. This is really of essence as it provides a means to “unlearn” what we know and thus investigate how the eye and hand coordinate works. This thesis comprises three experiments with three different factors to investigate their impact as well as the underlying mechanism of aimed movement. The first and the third experiments were related to visual inversion. In order to determine the dominant factor of distal-pointing, the second experiment was designed. The first experiment focused on the impact of eye-hand incompatibility and investigated the substructure of the complete movemenet to investigate the presence of a ballistic region when task difficulty is low. For this purpose, MT was separated into initiation time (IT), distance covering time (DCT), and acquisition time (AT). The AT gives a better separation of the ballistic and visually controlled regions on the index of difficulty (ID) than movement time indicating the existence of a ballistic region. The second experiment focused on pointing at different depths. The visual angles of movement amplitude and target width are better predictors of movement time than the linear dimensions and depth does not have a significant impact. With increasing depth, hand and arm tremor effects increase. The third experiment noted that the individual kinematic perception is correlated to one’s MT and people do not need to see the hand during the whole movement when aiming. What we need is a window nearer to the target. The duration and endpoint variability of the first ballistic submovement of the first and the third experiment were compared and that helped explain the performance difference among the different conditions. This validates that different visual cond

When tracking targets moving in various directions with one's eyes, horizontal components of pursuit are more precise than vertical ones. Is this because horizontal target motion is predicted better or because horizontal movements of the eyes are controlled more precisely? When tracking a visual target with the hand, the eyes also track the target. We investigated whether the directional asymmetries that have been found during isolated eye movements are also present during such manual tracking, and if so, whether individual participants' asymmetry in eye movements is accompanied by a similar asymmetry in hand movements. We examined the data of 62 participants who used a joystick to track a visual target with a cursor. The target followed a smooth but unpredictable trajectory in two dimensions. Both the mean gaze-target distance and the mean cursor-target distance were about 20% larger in the vertical direction than in the horizontal direction. Gaze and cursor both followed the target with a slightly longer delay in the vertical than in the horizontal direction, irrespective of the target's trajectory. The delays of gaze and cursor were correlated, as were their errors in tracking the target. Gaze clearly followed the target rather than the cursor, so the asymmetry in both eye and hand movements presumably results from better predictions of the target's horizontal than of its vertical motion. Altogether this study speaks for the presence of anisotropic predictive processes that are shared across effectors.

During concept design of new vehicles, work places, and other complex artifacts, it is critical to assess positioning of instruments and regulators from the perspective of the end user. One common way to do these kinds of assessments during early product development is by the use of Digital Human Modelling (DHM). DHM tools are able to produce detailed simulations, including vision. Many of these tools comprise evaluations of direct vision and some tools are also able to assess other perceptual features. However, to our knowledge, all DHM tools available today require manual selection of manikin viewpoint. This can be both cumbersome and difficult, and requires that the DHM user possesses detailed knowledge about visual behavior of the workers in the task being modelled. In the present study, we take the first steps towards an automatic selection of viewpoint through a computational model of eye-hand coordination. We here report descriptive statistics on visual behavior in a pick-and-place task executed in virtual reality. During reaching actions, results reveal a very high degree of eye-gaze towards the target object. Participants look at the target object at least once during basically every trial, even during a repetitive action. The object remains focused during large proportions of the reaching action, even when participants are forced to move in order to reach the object. These results are in line with previous research on eye-hand coordination and suggest that DHM tools should, by default, set the viewpoint to match the manikin’s grasping location., CC BY-NC 4.0Funder: Knowledge Foundation and the INFINIT research environment (KKS Dnr. 20180167). This work was financially supported by the synergy Virtual Ergonomics funded by the Swedish Knowledge Foundation, dnr 20180167. https://www.his.se/sve

People often look at objects that they are holding in their hands. It is therefore reasonable to expect them to be able to direct their gaze precisely with respect to their fingers. However, we know that people make reproducible idiosyncratic errors of up to a few centimetres when they try to align a visible cursor to their own finger hidden below a surface. To find out whether they also make idiosyncratic errors when they try to look at their finger, we asked participants to hold their finger in front of their head in the dark, and look at it. Participants made idiosyncratic errors of a similar magnitude to those previously found when matching a visual cursor to their hidden finger. This shows that proprioceptive position sense of finger and gaze are not aligned, suggesting that people rely on vision to guide their gaze to their own finger.

People often look at objects that they are holding in their hands. It is therefore reasonable to expect them to be able to direct their gaze precisely with respect to their fingers. However, we know that people make reproducible idiosyncratic errors of up to a few centimetres when they try to align a visible cursor to their own finger hidden below a surface. To find out whether they also make idiosyncratic errors when they try to look at their finger, we asked participants to hold their finger in front of their head in the dark, and look at it. Participants made idiosyncratic errors of a similar magnitude to those previously found when matching a visual cursor to their hidden finger. This shows that proprioceptive position sense of finger and gaze are not aligned, suggesting that people rely on vision to guide their gaze to their own finger.

During concept design of new vehicles, work places, and other complex artifacts, it is critical to assess positioning of instruments and regulators from the perspective of the end user. One common way to do these kinds of assessments during early product development is by the use of Digital Human Modelling (DHM). DHM tools are able to produce detailed simulations, including vision. Many of these tools comprise evaluations of direct vision and some tools are also able to assess other perceptual features. However, to our knowledge, all DHM tools available today require manual selection of manikin viewpoint. This can be both cumbersome and difficult, and requires that the DHM user possesses detailed knowledge about visual behavior of the workers in the task being modelled. In the present study, we take the first steps towards an automatic selection of viewpoint through a computational model of eye-hand coordination. We here report descriptive statistics on visual behavior in a pick-and-place task executed in virtual reality. During reaching actions, results reveal a very high degree of eye-gaze towards the target object. Participants look at the target object at least once during basically every trial, even during a repetitive action. The object remains focused during large proportions of the reaching action, even when participants are forced to move in order to reach the object. These results are in line with previous research on eye-hand coordination and suggest that DHM tools should, by default, set the viewpoint to match the manikin’s grasping location., CC BY-NC 4.0Funder: Knowledge Foundation and the INFINIT research environment (KKS Dnr. 20180167). This work was financially supported by the synergy Virtual Ergonomics funded by the Swedish Knowledge Foundation, dnr 20180167. https://www.his.se/sve

During concept design of new vehicles, work places, and other complex artifacts, it is critical to assess positioning of instruments and regulators from the perspective of the end user. One common way to do these kinds of assessments during early product development is by the use of Digital Human Modelling (DHM). DHM tools are able to produce detailed simulations, including vision. Many of these tools comprise evaluations of direct vision and some tools are also able to assess other perceptual features. However, to our knowledge, all DHM tools available today require manual selection of manikin viewpoint. This can be both cumbersome and difficult, and requires that the DHM user possesses detailed knowledge about visual behavior of the workers in the task being modelled. In the present study, we take the first steps towards an automatic selection of viewpoint through a computational model of eye-hand coordination. We here report descriptive statistics on visual behavior in a pick-and-place task executed in virtual reality. During reaching actions, results reveal a very high degree of eye-gaze towards the target object. Participants look at the target object at least once during basically every trial, even during a repetitive action. The object remains focused during large proportions of the reaching action, even when participants are forced to move in order to reach the object. These results are in line with previous research on eye-hand coordination and suggest that DHM tools should, by default, set the viewpoint to match the manikin’s grasping location., CC BY-NC 4.0Funder: Knowledge Foundation and the INFINIT research environment (KKS Dnr. 20180167). This work was financially supported by the synergy Virtual Ergonomics funded by the Swedish Knowledge Foundation, dnr 20180167. https://www.his.se/sve

During concept design of new vehicles, work places, and other complex artifacts, it is critical to assess positioning of instruments and regulators from the perspective of the end user. One common way to do these kinds of assessments during early product development is by the use of Digital Human Modelling (DHM). DHM tools are able to produce detailed simulations, including vision. Many of these tools comprise evaluations of direct vision and some tools are also able to assess other perceptual features. However, to our knowledge, all DHM tools available today require manual selection of manikin viewpoint. This can be both cumbersome and difficult, and requires that the DHM user possesses detailed knowledge about visual behavior of the workers in the task being modelled. In the present study, we take the first steps towards an automatic selection of viewpoint through a computational model of eye-hand coordination. We here report descriptive statistics on visual behavior in a pick-and-place task executed in virtual reality. During reaching actions, results reveal a very high degree of eye-gaze towards the target object. Participants look at the target object at least once during basically every trial, even during a repetitive action. The object remains focused during large proportions of the reaching action, even when participants are forced to move in order to reach the object. These results are in line with previous research on eye-hand coordination and suggest that DHM tools should, by default, set the viewpoint to match the manikin’s grasping location., CC BY-NC 4.0Funder: Knowledge Foundation and the INFINIT research environment (KKS Dnr. 20180167). This work was financially supported by the synergy Virtual Ergonomics funded by the Swedish Knowledge Foundation, dnr 20180167. https://www.his.se/sve

During concept design of new vehicles, work places, and other complex artifacts, it is critical to assess positioning of instruments and regulators from the perspective of the end user. One common way to do these kinds of assessments during early product development is by the use of Digital Human Modelling (DHM). DHM tools are able to produce detailed simulations, including vision. Many of these tools comprise evaluations of direct vision and some tools are also able to assess other perceptual features. However, to our knowledge, all DHM tools available today require manual selection of manikin viewpoint. This can be both cumbersome and difficult, and requires that the DHM user possesses detailed knowledge about visual behavior of the workers in the task being modelled. In the present study, we take the first steps towards an automatic selection of viewpoint through a computational model of eye-hand coordination. We here report descriptive statistics on visual behavior in a pick-and-place task executed in virtual reality. During reaching actions, results reveal a very high degree of eye-gaze towards the target object. Participants look at the target object at least once during basically every trial, even during a repetitive action. The object remains focused during large proportions of the reaching action, even when participants are forced to move in order to reach the object. These results are in line with previous research on eye-hand coordination and suggest that DHM tools should, by default, set the viewpoint to match the manikin’s grasping location., CC BY-NC 4.0Funder: Knowledge Foundation and the INFINIT research environment (KKS Dnr. 20180167). This work was financially supported by the synergy Virtual Ergonomics funded by the Swedish Knowledge Foundation, dnr 20180167. https://www.his.se/sve

Adapting hand movements to changes in our body or the environment is essential for skilled motor behavior. Although eye movements are known to assist hand movement control, how eye movements might contribute to the adaptation of hand movements remains largely unexplored. To determine to what extent eye movements contribute to visuomotor adaptation of hand tracking, participants were asked to track a visual target that followed an unpredictable trajectory with a cursor using a joystick. During blocks of trials, participants were either allowed to look wherever they liked or required to fixate a cross at the center of the screen. Eye movements were tracked to ensure gaze fixation as well as to examine free gaze behavior. The cursor initially responded normally to the joystick, but after several trials, the direction in which it responded was rotated by 90°. Although fixating the eyes had a detrimental influence on hand tracking performance, participants exhibited a rather similar time course of adaptation to rotated visual feedback in the gaze-fixed and gaze-free conditions. More importantly, there was extensive transfer of adaptation between the gaze-fixed and gaze-free conditions. We conclude that although eye movements are relevant for the online control of hand tracking, they do not play an important role in the visuomotor adaptation of such tracking. These results suggest that participants do not adapt by changing the mapping between eye and hand movements, but rather by changing the mapping between hand movements and the cursor's motion independently of eye movements. NEW & NOTEWORTHY Eye movements assist hand movements in everyday activities, but their contribution to visuomotor adaptation remains largely unknown. We compared adaptation of hand tracking under free gaze and fixed gaze. Although our results confirm that following the target with the eyes increases the accuracy of hand movements, they unexpectedly demonstrate that gaze fixation does not hin

The often reported reduction of Reaction Time (RT) by Vision Training) is successfully replicated by 81 athletes across sports. This enabled us to achieve a mean reduction of RTs for athletes eye-hand coordination of more than 10%, with high statistical significance. We explain how such an observed effect of Sensorimotor systems’ plasticity causing reduced RT can last in practice for multiple days and even weeks in subjects, via a proof of principle. Its mathematical neural model can be forced outside a previous stable (but long) RT into a state leading to reduced eye-hand coordination RT, which is, again, in a stable neural state., Interactive Intelligence

Adapting hand movements to changes in our body or the environment is essential for skilled motor behavior. Although eye movements are known to assist hand movement control, how eye movements might contribute to the adaptation of hand movements remains largely unexplored. To determine to what extent eye movements contribute to visuomotor adaptation of hand tracking, participants were asked to track a visual target that followed an unpredictable trajectory with a cursor using a joystick. During blocks of trials, participants were either allowed to look wherever they liked or required to fixate a cross at the center of the screen. Eye movements were tracked to ensure gaze fixation as well as to examine free gaze behavior. The cursor initially responded normally to the joystick, but after several trials, the direction in which it responded was rotated by 90°. Although fixating the eyes had a detrimental influence on hand tracking performance, participants exhibited a rather similar time course of adaptation to rotated visual feedback in the gaze-fixed and gaze-free conditions. More importantly, there was extensive transfer of adaptation between the gaze-fixed and gaze-free conditions. We conclude that although eye movements are relevant for the online control of hand tracking, they do not play an important role in the visuomotor adaptation of such tracking. These results suggest that participants do not adapt by changing the mapping between eye and hand movements, but rather by changing the mapping between hand movements and the cursor’s motion independently of eye movements.

Skilled motor behavior relies on the ability to control the body and to predict the sensory consequences of this control. Although there is ample evidence that manual dexterity depends on handedness, it remains unclear whether control and prediction are similarly impacted. To address this issue, right-handed human participants performed two tasks with either the right or the left hand. In the first task, participants had to move a cursor with their hand so as to track a target that followed a quasi-random trajectory. This hand-tracking task allowed testing the ability to control the hand along an imposed trajectory. In the second task, participants had to track with their eyes a target that was self-moved through voluntary hand motion. This eye-tracking task allowed testing the ability to predict the visual consequences of hand movements. As expected, results showed that hand tracking was more accurate with the right hand than with the left hand. In contrast, eye tracking was similar in terms of spatial and temporal gaze attributes whether the target was moved by the right or the left hand. Although these results extend previous evidence for different levels of control by the two hands, they show that the ability to predict the visual consequences of self-generated actions does not depend on handedness. We propose that the greater dexterity exhibited by the dominant hand in many motor tasks stems from advantages in control, not in prediction. Finally, these findings support the notion that prediction and control are distinct processes.

We often reach for remembered objects, such as when picking up a coffee cup from behind our laptop. In cases like this, we rely on visuospatial memory, encoded by the perceptual mechanisms of the ventral visual stream, to guide our actions, rather than on the real-time control of action by the dorsal visual stream (Milner & Goodale, 1995). Further, our motor plans must often accommodate for the messy spaces within which we act, avoiding irrelevant objects in our way. Little research has examined obstacle avoidance during memory-guided grasping, though it is likely obstacles perceived by the ventral stream as more salient will produce exacerbated avoidance maneuvers. This study examined how the availability of visual feedback altered eye-hand coordination in an obstacle avoidance paradigm. Eye and hand movements were monitored as subjects had to reach through a pair of obstacles in order to grasp a 3-D target object, under full visual feedback (visually-guided), immediately in the absence of visual feedback (memory-guided no-delay), or after a 2-s delay in the absence of visual feedback (memory-guided delay). Positions and widths of obstacles were manipulated, though their inner edges remained a constant distance apart. We expected the memory-guided delay group to exhibit exaggerated avoidance strategies due to a reliance on the perceptual mechanisms of the ventral steam. Results revealed successful obstacle avoidance and grasps of the target object in all groups, however different avoidance strategies emerged depending on the availability of visual feedback. The visually-guided and memory-guided no-delay groups used real-time visual information to alter the paths of the index finger and wrist and adjust final index finger positions on the target object, to account for positioned obstacles. Still, the no-delay group showed wider index finger paths and a failure to adjust final fixations, resulting from the inability to use visual information for the online control of

Adapting hand movements to changes in our body or the environment is essential for skilled motor behavior. Although eye movements are known to assist hand movement control, how eye movements might contribute to the adaptation of hand movements remains largely unexplored. To determine to what extent eye movements contribute to visuomotor adaptation of hand tracking, participants were asked to track a visual target that followed an unpredictable trajectory with a cursor using a joystick. During blocks of trials, participants were either allowed to look wherever they liked or required to fixate a cross at the center of the screen. Eye movements were tracked to ensure gaze fixation as well as to examine free gaze behavior. The cursor initially responded normally to the joystick, but after several trials, the direction in which it responded was rotated by 90°. Although fixating the eyes had a detrimental influence on hand tracking performance, participants exhibited a rather similar time course of adaptation to rotated visual feedback in the gaze-fixed and gaze-free conditions. More importantly, there was extensive transfer of adaptation between the gaze-fixed and gaze-free conditions. We conclude that although eye movements are relevant for the online control of hand tracking, they do not play an important role in the visuomotor adaptation of such tracking. These results suggest that participants do not adapt by changing the mapping between eye and hand movements, but rather by changing the mapping between hand movements and the cursor's motion independently of eye movements. NEW & NOTEWORTHY Eye movements assist hand movements in everyday activities, but their contribution to visuomotor adaptation remains largely unknown. We compared adaptation of hand tracking under free gaze and fixed gaze. Although our results confirm that following the target with the eyes increases the accuracy of hand movements, they unexpectedly demonstrate that gaze fixation does not hin

The often reported reduction of Reaction Time (RT) by Vision Training) is successfully replicated by 81 athletes across sports. This enabled us to achieve a mean reduction of RTs for athletes eye-hand coordination of more than 10%, with high statistical significance. We explain how such an observed effect of Sensorimotor systems’ plasticity causing reduced RT can last in practice for multiple days and even weeks in subjects, via a proof of principle. Its mathematical neural model can be forced outside a previous stable (but long) RT into a state leading to reduced eye-hand coordination RT, which is, again, in a stable neural state., Interactive Intelligence

We often reach for remembered objects, such as when picking up a coffee cup from behind our laptop. In cases like this, we rely on visuospatial memory, encoded by the perceptual mechanisms of the ventral visual stream, to guide our actions, rather than on the real-time control of action by the dorsal visual stream (Milner & Goodale, 1995). Further, our motor plans must often accommodate for the messy spaces within which we act, avoiding irrelevant objects in our way. Little research has examined obstacle avoidance during memory-guided grasping, though it is likely obstacles perceived by the ventral stream as more salient will produce exacerbated avoidance maneuvers. This study examined how the availability of visual feedback altered eye-hand coordination in an obstacle avoidance paradigm. Eye and hand movements were monitored as subjects had to reach through a pair of obstacles in order to grasp a 3-D target object, under full visual feedback (visually-guided), immediately in the absence of visual feedback (memory-guided no-delay), or after a 2-s delay in the absence of visual feedback (memory-guided delay). Positions and widths of obstacles were manipulated, though their inner edges remained a constant distance apart. We expected the memory-guided delay group to exhibit exaggerated avoidance strategies due to a reliance on the perceptual mechanisms of the ventral steam. Results revealed successful obstacle avoidance and grasps of the target object in all groups, however different avoidance strategies emerged depending on the availability of visual feedback. The visually-guided and memory-guided no-delay groups used real-time visual information to alter the paths of the index finger and wrist and adjust final index finger positions on the target object, to account for positioned obstacles. Still, the no-delay group showed wider index finger paths and a failure to adjust final fixations, resulting from the inability to use visual information for the online control of

A number of companies are marketing general eye–hand coordination training devices, which are purported to enhance performance on the device and in a sporting domain. An act comprising eye–hand coordination involves the complex combination of a number of distinct functions, and an investigation of what tasks share this common factor has not been completed. There is also a lack of evidence investigating the interrelationship between different tests to assess eye–hand coordination using these devices. A number of different eye–hand coordination abilities, rather than one common factor, could potentially underpin any range of tasks involving eye–hand coordination and visual stimuli. Therefore, the present study investigated the theoretical assumption upon which such eye–hand coordination training devices are based, that is, whether eye–hand coordination is a general ability. Eighty-seven currently active sportspeople (age: 18.6 ± 0.9 years; 58 males and 29 females) completed four tests of eye–hand coordination: three laboratory tasks (the Sports Vision Trainer™; Batak Pro™; and Graded Pegboard) and a field task (wall catch test). Intercorrelations between the tasks ranged from weak to strong, but the percentage of shared variance was typically low. Overall, the results do not support the existence of a common eye–hand coordination ability underpinning the performance on general eye–hand coordination training devices. Consequently, coaches and sport scientists should be aware that training on general eye–hand coordination training devices is unlikely to transfer to sporting performances. Instead, practitioners are encouraged to explore sport-specific assessment and training of eye–hand coordination.

Current theories suggest that the ability to control the body and to predict its associated sensory consequences is key for skilled motor behavior. It is also suggested that these abilities need to be updated when the mapping between motor commands and sensory consequences is altered. Here we challenge this view by investigating the transfer of adaptation to rotated visual feedback between one task in which human participants had to control a cursor with their hand in order to track a moving target, and another in which they had to predict with their eyes the visual consequences of their hand movement on the cursor. Hand and eye tracking performances were evaluated respectively through cursor–target and eye–cursor distance. Results reveal a striking dissociation: although prior adaptation of hand tracking greatly facilitates eye tracking, the adaptation of eye tracking does not transfer to hand tracking. We conclude that although the update of control is associated with the update of prediction, prediction can be updated independently of control. To account for this pattern of results, we propose that task demands mediate the update of prediction and control. Although a joint update of prediction and control seemed mandatory for success in our hand tracking task, the update of control was only facultative for success in our eye tracking task. More generally, those results promote the view that prediction and control are mediated by separate neural processes and suggest that people can learn to predict movement consequences without necessarily promoting their ability to control these movements.

A number of companies are marketing general eye–hand coordination training devices, which are purported to enhance performance on the device and in a sporting domain. An act comprising eye–hand coordination involves the complex combination of a number of distinct functions, and an investigation of what tasks share this common factor has not been completed. There is also a lack of evidence investigating the interrelationship between different tests to assess eye–hand coordination using these devices. A number of different eye–hand coordination abilities, rather than one common factor, could potentially underpin any range of tasks involving eye–hand coordination and visual stimuli. Therefore, the present study investigated the theoretical assumption upon which such eye–hand coordination training devices are based, that is, whether eye–hand coordination is a general ability. Eighty-seven currently active sportspeople (age: 18.6 ± 0.9 years; 58 males and 29 females) completed four tests of eye–hand coordination: three laboratory tasks (the Sports Vision Trainer™; Batak Pro™; and Graded Pegboard) and a field task (wall catch test). Intercorrelations between the tasks ranged from weak to strong, but the percentage of shared variance was typically low. Overall, the results do not support the existence of a common eye–hand coordination ability underpinning the performance on general eye–hand coordination training devices. Consequently, coaches and sport scientists should be aware that training on general eye–hand coordination training devices is unlikely to transfer to sporting performances. Instead, practitioners are encouraged to explore sport-specific assessment and training of eye–hand coordination.

This thesis aims to investigate an eye tracking calibration method in virtual reality where users’ visual attention from eye-hand coordination is used when interacting with in-game items. This could potentially allow eye tracking calibration without interrupting the virtual experience, in comparison with traditional eye tracking calibration which is cumbersome, disruptive, and requires the user’s full attention. A user study was conducted with 15 participants in which they were tasked to complete three different interactions. The interactions tested were a knob, a slider and a liftable cube. Where in the virtual environment the participants were looking during the interactions was recorded and processed to enable comparison. The processed data was analysed to find factors that influenced the calibration method. Additionally, the results was analysed to find at what point during the interactions that has the most consistent eye tracking fixations on the interacted item, and therefore most potential for eye tracking calibration. The results showed that when the participant is interacting with the item and the interacted item is stationary, we received a fixation for around 60% of all trials at any time. When the interacted item was moving the results indicated a lower percentage. To increase this number, the gaze data should be filtered instead of using raw gaze data in order to avoid flickering from the eye tracker. Regarding factors that influence the calibration method, the choice of interaction has a big impact on the method’s success where interactions in which the interacted item is stationary has more potential. Additionally, interactions that take longer time and requires precision in order to complete the interaction positively influences the potential of the calibration method. The surrounding virtual environment also has an influence, as a more distracting environment can negatively impact the calibration method., Denna avhandling ämnar att att undersöka en kalibreringsmetod för ögonspårning i virtuell verklighet där användarnas visuella uppmärksamhet från ögon och hand koordination används när man interagerar med objekt i den virtuella verkligheten. Detta kan möjliggöra kalibrering av ögonspårning utan att avbryta den virtuella upplevelsen, i jämförelse med traditionell kalibrering som är besvärlig, störande och kräver användarens fulla uppmärksamhet. En användarstudie genomfördes med 15 deltagare där de hade till uppgift att slutföra tre olika interaktioner. De testade interaktionerna var en knopp, en slider och en lyftbar kub. Deltagarnas blick spelades in under interaktionerna och analyserades för att möjliggöra jämförelse. Den bearbetade data analyserades för att hitta faktorer som påverkar kalibreringsmetoden. Dessutom analyserades resultaten för att hitta vid vilken tidpunkt under interaktionerna som hade de mest konsekventa ögonfixeringarna på objektet de interagerade med och därmed störst potential för kalibrering av ögonspårning. Resultaten visade att när deltagaren interagerar med objektet och det interaktiva objektet är stillastående, fick vi en fixering för omkring 60% av alla försök under godtycklig tidpunkt. När det interaktiva objektet rörde sig, visade resultaten en lägre procentandel. För att öka antalet, ska blickdatan filtreras i stället för att använda rå blickdata för att undvika att flicker från ögonspåraren. När det gäller faktorer som påverkar kalibreringsmetoden har valet av interaktion stor inverkan på metodens framgång, där interaktioner där det interaktiva objektet är stationärt har större potential. Dessutom påverkar interaktioner som tar längre tid och kräver precision för att slutföra interaktionen kalibreringsmetodens potential positivt. Den omgivande virtuella miljön har också inflytande, eftersom en mer distraherande miljö kan negativt påverka kalibreringsmetoden.

Previous studies on smooth pursuit eye movements have shown that humans can continue to track the position of their hand, or a target controlled by the hand, after it is occluded, thereby demonstrating that arm motor commands contribute to the prediction of target motion driving pursuit eye movements. Here, we investigated this predictive mechanism by manipulating both the complexity of the hand-target mapping and the provision of haptic feedback. Two hand-target mappings were used, either a rigid (simple) one in which hand and target motion matched perfectly or a nonrigid (complex) one in which the target behaved as a mass attached to the hand by means of a spring. Target animation was obtained by asking participants to oscillate a lightweight robotic device that provided (or not) haptic feedback consistent with the target dynamics. Results showed that as long as 7 s after target occlusion, smooth pursuit continued to be the main contributor to total eye displacement (∼60%). However, the accuracy of eye-tracking varied substantially across experimental conditions. In general, eye-tracking was less accurate under the nonrigid mapping, as reflected by higher positional and velocity errors. Interestingly, haptic feedback helped to reduce the detrimental effects of target occlusion when participants used the nonrigid mapping, but not when they used the rigid one. Overall, we conclude that the ability to maintain smooth pursuit in the absence of visual information can extend to complex hand-target mappings, but the provision of haptic feedback is critical for the maintenance of accurate eye-tracking performance.

The ability to track a moving target with the eye is substantially improved when the target is self-moved compared with when it is moved by an external agent. To account for this observation, it has been postulated that the oculomotor system has access to hand efference copy, thereby allowing to predict the motion of the visual target. Along this scheme, we tested the effect of transcranial magnetic stimulation (TMS) over the hand area of the primary motor cortex (M1) when human participants (50% females) are asked to track with their eyes a visual target whose horizontal motion is driven by their grip force.Wereasoned that, if the output ofM1is used by the oculomotor system to keep track of the target, on top of inducing short latency disturbance of grip force, single-pulse TMS should also quickly disrupt ongoing eye motion. For comparison purposes, the effect of TMS overM1was monitored when subjects tracked an externally moved target (while keeping their hand at rest or not). Inbothcases, resultsshowednoalterations insmoothpursuit,meaningthat its velocitywasunaffected within the25–125msepochthat followed TMS. Overall, our results imply that the output ofM1has limited contribution in driving the eye motion during our eye-hand coordination task. This study suggests that, if hand motor signals are accessed by the oculomotor system, this is upstream of M1.

Throughout their development humans learn to communicate using movements, or body language. These skills develop more slowly in individuals with autism spectrum disorder. The present study examined how social communication skills correlate with movement performance in typically developing individuals. Participants performed eyes-only, button-press and pointing movements to a picture identified by a visual or auditory cue. Reaction time (RT) and movement time (MT) were analyzed and correlated with the scores on the broad autism phenotype questionnaire. Saccade RT was longest and most variable in the eyes-only condition, suggesting increased task difficulty. Hand RT was shorter for pointing movements, which is consistent with a peripheral vision advantage. Longer saccade RTs were correlated with higher rigidity in the eyes-only task. Lower pragmatic language was negatively correlated with hand RT and saccade MT for the visual eye-hand coordinated tasks. Overall, an individual’s self-report of their social communication skills were related to motor performance.

This thesis aims to investigate an eye tracking calibration method in virtual reality where users’ visual attention from eye-hand coordination is used when interacting with in-game items. This could potentially allow eye tracking calibration without interrupting the virtual experience, in comparison with traditional eye tracking calibration which is cumbersome, disruptive, and requires the user’s full attention. A user study was conducted with 15 participants in which they were tasked to complete three different interactions. The interactions tested were a knob, a slider and a liftable cube. Where in the virtual environment the participants were looking during the interactions was recorded and processed to enable comparison. The processed data was analysed to find factors that influenced the calibration method. Additionally, the results was analysed to find at what point during the interactions that has the most consistent eye tracking fixations on the interacted item, and therefore most potential for eye tracking calibration. The results showed that when the participant is interacting with the item and the interacted item is stationary, we received a fixation for around 60% of all trials at any time. When the interacted item was moving the results indicated a lower percentage. To increase this number, the gaze data should be filtered instead of using raw gaze data in order to avoid flickering from the eye tracker. Regarding factors that influence the calibration method, the choice of interaction has a big impact on the method’s success where interactions in which the interacted item is stationary has more potential. Additionally, interactions that take longer time and requires precision in order to complete the interaction positively influences the potential of the calibration method. The surrounding virtual environment also has an influence, as a more distracting environment can negatively impact the calibration method., Denna avhandling ämnar att att undersöka en kalibreringsmetod för ögonspårning i virtuell verklighet där användarnas visuella uppmärksamhet från ögon och hand koordination används när man interagerar med objekt i den virtuella verkligheten. Detta kan möjliggöra kalibrering av ögonspårning utan att avbryta den virtuella upplevelsen, i jämförelse med traditionell kalibrering som är besvärlig, störande och kräver användarens fulla uppmärksamhet. En användarstudie genomfördes med 15 deltagare där de hade till uppgift att slutföra tre olika interaktioner. De testade interaktionerna var en knopp, en slider och en lyftbar kub. Deltagarnas blick spelades in under interaktionerna och analyserades för att möjliggöra jämförelse. Den bearbetade data analyserades för att hitta faktorer som påverkar kalibreringsmetoden. Dessutom analyserades resultaten för att hitta vid vilken tidpunkt under interaktionerna som hade de mest konsekventa ögonfixeringarna på objektet de interagerade med och därmed störst potential för kalibrering av ögonspårning. Resultaten visade att när deltagaren interagerar med objektet och det interaktiva objektet är stillastående, fick vi en fixering för omkring 60% av alla försök under godtycklig tidpunkt. När det interaktiva objektet rörde sig, visade resultaten en lägre procentandel. För att öka antalet, ska blickdatan filtreras i stället för att använda rå blickdata för att undvika att flicker från ögonspåraren. När det gäller faktorer som påverkar kalibreringsmetoden har valet av interaktion stor inverkan på metodens framgång, där interaktioner där det interaktiva objektet är stationärt har större potential. Dessutom påverkar interaktioner som tar längre tid och kräver precision för att slutföra interaktionen kalibreringsmetodens potential positivt. Den omgivande virtuella miljön har också inflytande, eftersom en mer distraherande miljö kan negativt påverka kalibreringsmetoden.

Throughout their development humans learn to communicate using movements, or body language. These skills develop more slowly in individuals with autism spectrum disorder. The present study examined how social communication skills correlate with movement performance in typically developing individuals. Participants performed eyes-only, button-press and pointing movements to a picture identified by a visual or auditory cue. Reaction time (RT) and movement time (MT) were analyzed and correlated with the scores on the broad autism phenotype questionnaire. Saccade RT was longest and most variable in the eyes-only condition, suggesting increased task difficulty. Hand RT was shorter for pointing movements, which is consistent with a peripheral vision advantage. Longer saccade RTs were correlated with higher rigidity in the eyes-only task. Lower pragmatic language was negatively correlated with hand RT and saccade MT for the visual eye-hand coordinated tasks. Overall, an individual’s self-report of their social communication skills were related to motor performance.

In recent years more and more complex humanoid robots have been developed. On the other hand programming these systems has become more difficult. There is a clear need for such robots to be able to adapt and perform certain tasks autonomously, or even learn by themselves how to act. An important issue to tackle is the closing of the sensorimotor loop. Especially when talking about humanoids the tight integration of perception with actions will allow for improved behaviours, embedding adaptation on the lower-level of the system.

We describe our software system enabling a tight integration between vision and control modules on complex, high-DOF humanoid robots. This is demonstrated with the iCub humanoid robot performing visual object detection and reaching and grasping actions. A key capability of this system is reactive avoidance of obstacle objects detected from the video stream while carrying out reach-and-grasp tasks. The subsystems of our architecture can independently be improved and updated, for example, we show that by using machine learning techniques we can improve visual perception by collecting images during the robot's interaction with the environment. We describe the task and software design constraints that led to the layered modular system architecture.

Participants executed right-handed reach-to-grasp movements toward horizontally translating targets. Visual feedback of the target when reaching, as well as the presence of additional cues placed close (Experiment 1) or far (Experiment 2) above and below the target’s path was manipulated. Additional cue presence appeared to impair participants’ ability to extrapolate the disappeared target’s motion, and caused grasps for occluded targets to be less accurate. Final gaze and grasp positions were more accurate when reaching for leftward moving targets, suggesting individuals use different grasp strategies when reaching for targets travelling away from the reaching hand. Comparison of average fixations at reach onset and at the time of the grasp suggested that participants accurately extrapolated the occluded target’s motion prior to reach onset, but not after, resulting in inaccurate grasps. New information is provided about the eye-hand strategies used when reaching for moving targets in unpredictable visual conditions.

Purpose: We intended to assess consequences of reduced visual acuity for performance in a natural simple motor task (tracing) using objective kinematic performance measures. Specifically, we intended to elucidate the kind of relationship between the task performance and best corrected binocular visual acuity and to determine the threshold of visual acuity when task performance starts to deteriorate. Methods: Ninety-five individuals with different best corrected visual acuity participated in the study (age 49±12 years, mean±SD, 27 men and 68 women). The participants manually traced maze-like visual patterns of different spatial complexity presented on the screen of a portable notebook computer using Clinical Kinematic Assessment Tool software. Tracing error was computed as performance measure in each trial with a spatial pattern matching technique - rigid point set registration method. Results: The segmented linear regression analysis showed that the relation between visual acuity and tracing errors was best described with a regression function having a break point between two data segments. Tracing performance was unaffected by values of visual acuity below 0.2 on logMAR scale, but when logMAR values increased above this critical limit (i.e. when visual acuity is further reduced), tracing errors linearly increased. The rate of the increase of the tracing error correlated with the complexity of visual stimulus shape. Conclusion: Testing of fine motor functions with objective kinematic measures during visuomotor tasks may help differentiating between actual effects of reduced visual acuity on eye-hand coordination in individuals with similar levels of impairment of visual acuity.

Purpose We intended to assess consequences of reduced visual acuity for performance in a natural simple motor task (tracing) using objective kinematic performance measures. Specifically, we intended to elucidate the kind of relationship between the task performance and best corrected binocular visual acuity and to determine the threshold of visual acuity when task performance starts to deteriorate. Methods Ninety-five individuals with different best corrected visual acuity participated in the study (age 49 ± 12 years, mean ± SD, 27 men and 68 women). The participants manually traced maze-like visual patterns of different spatial complexity presented on the screen of a portable notebook computer using Clinical Kinematic Assessment Tool software. Tracing error was computed as performance measure in each trial with a spatial pattern matching technique – rigid point set registration method. Results The segmented linear regression analysis showed that the relation between visual acuity and tracing errors was best described with a regression function having a break point between two data segments. Tracing performance was unaffected by values of visual acuity below 0.2 on logMAR scale, but when logMAR values increased above this critical limit (i.e. when visual acuity is further reduced), tracing errors linearly increased. The rate of the increase of the tracing error correlated with the complexity of visual stimulus shape. Conclusion Testing of fine motor functions with objective kinematic measures during visuomotor tasks may help differentiating between actual effects of reduced visual acuity on eye–hand coordination in individuals with similar levels of impairment of visual acuity.

Participants executed right-handed reach-to-grasp movements toward horizontally translating targets. Visual feedback of the target when reaching, as well as the presence of additional cues placed close (Experiment 1) or far (Experiment 2) above and below the target’s path was manipulated. Additional cue presence appeared to impair participants’ ability to extrapolate the disappeared target’s motion, and caused grasps for occluded targets to be less accurate. Final gaze and grasp positions were more accurate when reaching for leftward moving targets, suggesting individuals use different grasp strategies when reaching for targets travelling away from the reaching hand. Comparison of average fixations at reach onset and at the time of the grasp suggested that participants accurately extrapolated the occluded target’s motion prior to reach onset, but not after, resulting in inaccurate grasps. New information is provided about the eye-hand strategies used when reaching for moving targets in unpredictable visual conditions.

Introduction: There is a hypothesis that Eye-Hand Coordination (EHC) is a general ability presenting an opportunity to explore it's mechanisms via a series of innovative studies. The thesis outlines two major aims: 1) to establish reliable measurement techniques and protocols for EHC using the Sport Vision Trainer (SVTâ„¢); 2) to explore different training methods to understand if performance can be improved. Methods: Four hundred and seventy-six participants volunteered for the studies, predominately recruited from the undergraduate population of the sport and exercise science degree at Edge Hill University, apart from the final training study of a local table tennis team. A total of 23,112 trials were recorded in the technical evaluation using the SVTâ„¢. Three measurement studies were conducted to establish test-retest reliability, performance predictors, and effect of sporting experience. In addition, three training studies were completed investigating performance under different illumination levels, stroboscopic training, and a general vision training (GVT) programmes. Results: Reliable measurement protocols are reported for the SVTâ„¢ along with original insight into the effects on EHC performance. Discussion: The concept of EHC as a general ability in the sense of an overall element supporting performance on a range of associated tasks is explored. As the sport vision literature identifies a requirement to isolate individual components of visual software the final study gives unique insight into the effectiveness of a GVT programme focusing on EHC with a team of club table tennis players. Specific training implications, limitations and recommendations for further research are also presented. Conclusions: The existence of an inherent EHC ability is doubtful and whilst the usefulness of GVT programmes has been criticised, the focus on EHC as an isolated visuo-motor skill yielded both EHC improvement and performance gains in a sporting context.

We propose a system incorporating a tight integration between computer vision and robot control modules on a complex, high-DOF humanoid robot. Its functionality is showcased by having our iCub humanoid robot pick-up objects from a table in front of it. An important feature is that the system can avoid obstacles - other objects detected in the visual stream - while reaching for the intended target object. Our integration also allows for non-static environments, i.e. the reaching is adapted on-the-fly from the visual feedback received, e.g. when an obstacle is moved into the trajectory. Furthermore we show that this system can be used both in autonomous and tele-operation scenarios.

Amblyopia occurs in up to 5% of the normal population, and is a unilateral or (rarely) bilateral visual disorder that occurs during the critical period of brain development and produces deficits in visual function such as acuity and stereopsis, plus abnormalities in eye movement (EM) and eye-hand coordination (EHC). Previous approaches to the assessment of EM and EHC are limited and poorly developed for clinical use and research. In particular, they are subjective, require a high degree of skill, are time consuming or are costly. The aim of this study was to develop novel tests of EM and EHC using simple computer-based objective tasks. These two tests (the Lee EM Test and Lee-Ryan EHC Test) were validated and used to examine differences in performance in normal, untreated amblyopic and treated-amblyopic vision under monocular and binocular viewing conditions in children and teenager/young adults. The Lee EM Test was based on the ‘Northeastern State University College of Optometry (NSUCO) Test’. In the Lee EM test, video recordings were made of individuals making saccadic and pursuit movements. Recordings were analysed regarding ability and accuracy of EM plus head and body movements on 5-point scales.The Lee-Ryan EHC Test comprised an i-Pad® app and the involved the use of a stylus pen to trace 13 shapes of different degrees of difficulty. For each shape, time taken to complete the trace, number of errors, mean speed, x/y axis deviation and mean deviation are measured and automatically saved by a program on the iPad platform.The EM testing revealed significant differences between normal, amblyopic and treated-amblyopic vision in both saccadic (p<0.05) and pursuit (p<0.05) movements. The biggest difference was significantly worse performance for amblyopes than normals for accuracy in both saccades and pursuits in monocular or binocular viewing conditions (p<0.0001).For EHC, both children and teenager/adults with untreated amblyopia made significantly more errors

Although robotics research has seen advances over the last decades robots are still not in widespread use outside industrial applications. Yet a range of proposed scenarios have robots working together, helping and coexisting with humans in daily life. In all these a clear need to deal with a more unstructured, changing environment arises. I herein present a system that aims to overcome the limitations of highly complex robotic systems, in terms of autonomy and adaptation. The main focus of research is to investigate the use of visual feedback for improving reaching and grasping capabilities of complex robots. To facilitate this a combined integration of computer vision and machine learning techniques is employed. From a robot vision point of view the combination of domain knowledge from both imaging processing and machine learning techniques, can expand the capabilities of robots. I present a novel framework called Cartesian Genetic Programming for Image Processing (CGP-IP). CGP-IP can be trained to detect objects in the incoming camera streams and successfully demonstrated on many different problem domains. The approach requires only a few training images (it was tested with 5 to 10 images per experiment) is fast, scalable and robust yet requires very small training sets. Additionally, it can generate human readable programs that can be further customized and tuned. While CGP-IP is a supervised-learning technique, I show an integration on the iCub, that allows for the autonomous learning of object detection and identification. Finally this dissertation includes two proof-of-concepts that integrate the motion and action sides. First, reactive reaching and grasping is shown. It allows the robot to avoid obstacles detected in the visual stream, while reaching for the intended target object. Furthermore the integration enables us to use the robot in non-static environments, i.e. the reaching is adapted on-the- fly from the visual feedback received, e.g. when an obstac

The authors present a study of the relationship between individual variation in working memory capacity (WMC) and visually guided hand control in the face of visual distraction. WMC was assessed with the automated operation span task. Hand control was measured by requesting participants to track a visual target with a hand-held touch screen pen. Tracking error increased when nontarget visual objects (distractors) appeared, especially in individuals with low WMC. High-WMC individuals are less impaired by distractors than their low-WMC counterpart, because they resume target tracking more quickly after distractor onset. The results suggest that visual distractors cause a momentary interruption to tracking movements and that high WMC attenuates this interruption by facilitating visual search., Funding Agencies|AFA|090231|Swedish Council for Working Life, Social Research Grant|2009-1761

The correlation between gaze strategy and kinematics of bimanual movements isassessed using repetitive bimanual object transfers as an experimental paradigm. The hypothesis isthat visual demand in such tasks may be a critical bottleneck determining bimanual coordination.Kinematics and eye-movements were compared before and after practice of this repetitive task.New eye-hand coordination strategies emerged with practice. Also, with practice, a systematicprioritization of the left hand movement to be „primary‟ and the right hand movement to be„secondary‟ emerged. This choice implied that the left hand movement kinematics was similar tounimanual left hand movements, whereas the performance of the right hand task was contingent onsuccessful completion of the primary task. This was revealed by „anticipatory adjustments‟ of theright hand kinematics (Right-hand peak velocity ranged from 100%-70% of the left-hand, and thescaling was dependent on task conditions and the corresponding eye-hand coordination strategiesused). We use this evidence to argue that the CNS, aware of an inherent asymmetry between thetwo hand systems, learns to anticipate the need and availability of visual feedback for successfultask completion, and uses this knowledge to optimize movement coordination - specifically suchthat the right-hand control was modulated to take visual constraints into account.

We asked whether people's actions are understood by projecting them onto one's own action programs, according to the direct matching hypothesis, and whether this mode of control functions in infants. Adults' and infants' gaze and hand movements were measured in two live situations. The task was either to move an object between two places in the visual field, or to observe the corresponding action performed by another person. When performing the action, infants and adults behaved strikingly similar. Hand and gaze movements were simultaneously initiated and gaze arrived at the goal ahead of the hand. When observing the actions, the initiation of the gaze shift was delayed relative to the observed hand movement in both infants and adults, but it still arrived at the goal ahead of the hand. For both the performance and observation of actions the proactiveness of gaze shifts was associated with saccades ahead of the velocity peak of the hand. The close similarity between adults' and infants' actions when performing the movements and the great advantage of the adults when observing them support the conclusion that one's own motor actions develop ahead of the ability to predict other people's actions.