Search

The investigation of human perception and movement kinematics during manipulation tasks provides insights that can be applied in the design of robotic systems in order to perform human-like manipulations in different contexts and with different performance requirements. In this paper we investigate control in a motor task, in which a tool is moved vertically until it touches a support surface. We evaluate how acoustic and haptic sensory information generated at the moment of contact modulates the kinematic parameters of the movement. Experimental results show differences in the achieved motor control precision and adaptation rate across conditions. We describe how the experimental results can be used in robotics applications in the fields of unsupervised learning, supervised learning from human demonstrators and teleoperations.

We present a system for sensorimotor audio-visual source localization on a mobile robot. We utilize a particle filter for the combination of audio-visual information and for the temporal integration of consecutive measurements. Although the system only measures the current direction of the source, the position of the source can be estimated because the robot is able to move and can therefore obtain measurements from different directions. These actions by the robot successively reduce uncertainty about the source's position. An information gain mechanism is used for selecting the most informative actions in order to minimize the number of actions required to achieve accurate and precise position estimates in azimuth and distance. We show that this mechanism is an efficient solution to the action selection problem for source localization, and that it is able to produce precise position estimates despite simplified unisensory preprocessing. Because of the robot's mobility, this approach is suitable for use in complex and cluttered environments. We present qualitative and quantitative results of the system's performance and discuss possible areas of application.

Sensory modalities are usually appropriately aligned in space: audition, vision, and proprioception each direct actions to the same spatial coordinates. Subjects wearing prism glasses that shift the visual input first miss the target in a pointing task, but quickly adapt to the new sensorimotor configuration. This adaptation may take place either in the visual or the proprioceptive pathway, ie, the internal visual representation is shifted back or the proprioceptive one is shifted towards the new configuration. Usually, the proprioceptive component is affected, probably due to the often observed dominance of vision over other modalities. This process is changed when auditory stimuli are presented during prism exposure inducing a shift of the visual representation contrary to the aforementioned results, maybe because a cortical mechanism performs statistical reliability estimation: Both audition and proprioception remain unaffected by prism exposure and therefore force vision to realign. We investigated the influence of sound-source-location on prism-adaptation by assessing the effects of displaced (accordingly to the prism offset), centered and unlocalized sound-sources. We discuss the influence of spatial properties on sensory calibration, its implications on the notion of motor action as the binding element between senses and its role in spatial adaptation processes.

Perception in natural environments is inseparably linked to motor action. In fact, we consider action an essential component of perceptual representation. But these representations are inherently difficult to investigate: Traditional experimental setups are limited by the lack of flexibility in manipulating spatial features. To overcome these problems, virtual reality (VR) experiments seem to be a feasible alternative, but these setups typically lack ecological realism due to the use of “unnatural” interface-devices (joystick). Thus, we propose an experimental apparatus which combines multisensory perception and action in an ecologically realistic way. The basis is a 10-foot hollow sphere (VirtuSphere) placed on a platform that allows free rotation. A subject inside can walk in any direction for any distance immersed into virtual environment. Both the rotation of the sphere and movement of the subject's head are tracked to process the subject's view within the VR-environment presented on a head-mounted display. Moreover, auditory features are dynamically processed taking greatest care of exact alignment of sound-sources and visual objects using ambisonic-encoded audio processed by a HRTF-filterbank. We present empirical data that confirm ecological realism of this setup and discuss its suitability for multi-sensory-motor research.