Search

Robot teleoperation, a control method allowing human operators to manipulate robotic systems remotely, has become increasingly popular in construction applications. A significant challenge is the disconnection between the robot sensor data and the human operator's sensory processes, creating a sensorimotor mismatch in motor-intensive activities. This disconnection is particularly challenging in motor-intensive activities that require accurate perception and response. Researchers have started investigating haptic interactions to enhance the control feedback loop, including simulating contacts, motions, and tactile input. However, although current methodologies have advanced the field, they often focused on certain aspects and could be further expanded to provide a more comprehensive simulation of the physical interaction that occurs in typical construction operations. This study designs and tests a comprehensive high-fidelity embodied teleoperation method that simulates complete real-world physical processes via the physics engine. The proposed method captures all categories of physical interaction in typical motor-intensive construction tasks, including weight, texture, inertia, impact, balance, rotation, and spring. A human-subject experiment shows that the proposed method substantially improves performance and human functions in a teleoperated pipe-fitting task. The results indicate that the proposed multisensory augmentation method significantly enhances performance and user experience, offering valuable insights for designing innovative robot teleoperation systems for future construction applications. [ABSTRACT FROM AUTHOR]

Robust remote teleoperation of high-degree-of-freedom manipulators is of critical importance across a wide range of robotics applications. Contemporary robot manipulation interfaces primarily utilize a free positioning pose specification approach to independently control each degree of freedom in free space. In this work, we present two novel interfaces, constrained positioning and point-and-click. Both novel approaches incorporate scene information from depth data into the grasp pose specification process, effectively reducing the number of 3D transformations the user must input. The novel interactions are designed for 2D image streams, rather than traditional 3D virtual scenes, further reducing mental transformations by eliminating the controllable camera viewpoint in favor of fixed physical camera viewpoints. We present interface implementations of our novel approaches, as well as free positioning, in both 2D and 3D visualization modes. In addition, we present results of a 90-participant user study evaluation comparing the effectiveness of each approach for a set of general object manipulation tasks, and the effects of implementing each approach in 2D image views versus 3D depth views. The results of our study show that point-and-click outperforms both free positioning and constrained positioning by significantly increasing the number of tasks completed and significantly reducing task failures and grasping errors, while significantly reducing the number of user interactions required to specify poses. In addition, we found that regardless of the interaction approach, the 2D visualization mode resulted in significantly better performance than the 3D visualization mode, with statistically significant reductions in task failures, grasping errors, task completion time, number of interactions, and user workload, all while reducing bandwidth requirements imposed by streaming depth data. [ABSTRACT FROM AUTHOR]

• A large-scale multi-view dataset that provides accurate annotations for hand poses. • A pipeline that improves the state-of-the-art results for 3D hand pose estimation. • We successfully applied our approach to robot hand teleoperation. In this paper, we present a novel deep learning-based architecture, which is under the scope of expert and intelligent systems, to perform accurate real-time tridimensional hand pose estimation using a single RGB frame as an input, so there is no need to use multiple cameras or points of view, or RGB-D devices. The proposed pipeline is composed of two convolutional neural network architectures. The first one is in charge of detecting the hand in the image. The second one is able to accurately infer the tridimensional position of the joints retrieving, thus, the full hand pose. To do this, we captured our own large-scale dataset composed of images of hands and the corresponding 3D joints annotations. The proposal achieved a 3D hand pose mean error of below 5 mm on both the proposed dataset and Stereo Hand Pose Tracking Benchmark, which is a public dataset. Our method also outperforms the state-of-the-art methods. We also demonstrate in this paper the application of the proposal to perform a robotic hand teleoperation with high success. [ABSTRACT FROM AUTHOR]

We introduce a novel magnetic angular rate gravity (MARG) sensor fusion algorithm for inertial measurement. The new algorithm improves the popular gradient descent (ʻMadgwick') algorithm increasing accuracy and robustness while preserving computational efficiency. Analytic and experimental results demonstrate faster convergence for multiple variations of the algorithm through changing magnetic inclination. Furthermore, decoupling of magnetic field variance from roll and pitch estimation is proven for enhanced robustness. The algorithm is validated in a human-machine interface (HMI) case study. The case study involves hardware implementation for wearable robot teleoperation in both Virtual Reality (VR) and in real-time on a 14 degree-of-freedom (DoF) humanoid robot. The experiment fuses inertial (movement) and mechanomyography (MMG) muscle sensing to control robot arm movement and grasp simultaneously, demonstrating algorithm efficacy and capacity to interface with other physiological sensors. To our knowledge, this is the first such formulation and the first fusion of inertial measurement and MMG in HMI. We believe the new algorithm holds the potential to impact a very wide range of inertial measurement applications where full orientation necessary. Physiological sensor synthesis and hardware interface further provides a foundation for robotic teleoperation systems with necessary robustness for use in the field. [ABSTRACT FROM AUTHOR]

This paper proposes a compensated PD-like controller for delayed bilateral teleoperation of a manipulator robot. The scheme has a PD-like remote controller, a damping into the master, and a compensation strategy. The proposed compensation removes part of potential energy of the user's command depending on the difference between the situation on the remote site and the situation as perceived by the human operator. In addition, the stability of the delayed teleoperation system is analyzed, and a comparison based on experiments is carried out in order to analyze the advantages of using the proposed compensation. Finally, results of a bilateral teleoperation including the proposed control scheme, where the master and slave exchange information by using a low-cost connection of mobile Internet, are shown. Copyright © 2014 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

This paper utilizes a human-robot interface system which incorporates Kalman filter (KF) and Adaptive Multi-space Transformation (AMT) to track the pose of the human hand for controlling the robot manipulator. This system employs a 3D camera (Kinect) to determine the orientation and the translation of the human hand. We use Camshift algorithm to track the hand. KF is used to estimate the translation of the human hand. Although a KF is used for estimating the translation, the translation error increases in a short period of time when the sensors fail to detect the hand motion. Therefore, a methodology to correct the translation error is required. What is more, to be subject to the perceptive limitations and the motor limitations, human operator is hard to carry out the high precision operation. This paper proposes an Adaptive Multi-space Transformation (AMT) method to assist the operator to improve the accuracy and reliability in determining the pose of the robot. The human-robot interface system was experimentally tested in a lab environment, and the results indicate that such a system can successfully control a robot manipulator. [ABSTRACT FROM AUTHOR]

Abstract: In Chopra et al (2008) [Chopra, N., Spong, M. W., & Lozano, R. (2008). Synchronization of bilateral teleoperators with time-delay. Automatica, 44(8), 2142–2148], an adaptive controller for teleoperators with time-delays, which ensures synchronization of positions and velocities of the master and slave manipulators, and does not rely on the use of the ubiquitous scattering transformation, is proposed. In this paper it is shown that this controller will tend to drive to zero the positions of the joints where gravity forces are non-zero. Hence, the scheme is, in general, applicable only to systems without gravity. We also prove in this paper that this limitation can be obviated, replacing the positions and velocities–that are used in the coordinating torques and the adaptation laws–by their errors. Simulation results illustrate the performance of both schemes. [Copyright &y& Elsevier]

Robot teleoperation systems have been limited in their utility due to the need for operator motion, lack of portability and limitation to singular input modalities. In this article, the design and construction of a dual-mode human-machine interface system for robot teleoperation addressing all these issues is presented. The interface is capable of directing robotic devices in response to tongue movement and/or speech without insertion of any device in the vicinity of the oral cavity. The interface is centered on the unique properties of the human ear as an acoustic output device. Specifically, we present: (1) an analysis of the sensitivity of human ear canals as acoustic output device; (2) the design of a new sensor for monitoring airflow in the aural canal; (3) pattern recognition procedures for recognition of both speech and tongue movement by monitoring aural flow across several human test subjects; and (4) a conceptual design and simulation of the machine interface system. We believe this work will lay the foundation for a new generation of human machine interface systems for all manner of robotic applications. [ABSTRACT FROM AUTHOR]

A novel 6 degrees of freedom (DOFs) wearable exoskeleton arm, ZJUESA, based on man-machine system is presented. It can be used in robot bilateral teleoperation as master arm with force-feedback. With introducing the revolute-prismatic-spherical (RPS) parallel mechanism and planetary gear mechanism, it is designed based on the anatomy of human upper-limb. With the sensors on this external mechanical structure, the human operator motion is detected and converted to the slave robot control command. Additionally the pneumatic system on it generates a force feedback by using hybrid fuzzy control. As a result, the human operator may have a feeling of doing the work directly. [ABSTRACT FROM AUTHOR]

To extend the choice of inertial motion-tracking systems freely available to researchers and educators, this paper presents an alternative open-source design of a wearable 7-DOF wireless human arm motion-tracking system. Unlike traditional inertial motion-capture systems, the presented system employs a hybrid combination of two inertial measurement units and one potentiometer for tracking a single arm. The sequence of three design phases described in the paper demonstrates how the general concept of a portable human arm motion-tracking system was transformed into an actual prototype, by employing a modular approach with independent wireless data transmission to a control PC for signal processing and visualization. Experimental results, together with an application case study on real-time robot-manipulator teleoperation, confirm the applicability of the developed arm motion-tracking system for facilitating robotics research. The presented arm-tracking system also has potential to be employed in mechatronic system design education and related research activities. The system CAD design models and program codes are publicly available online and can be used by robotics researchers and educators as a design platform to build their own arm-tracking solutions for research and educational purposes. [ABSTRACT FROM AUTHOR]