1. Virtual Kinesthetic Teaching for Bimanual Telemanipulation
- Author
-
Barry Lennox, Andrew Weightman, Hanlin Niu, Emily C. Collins, Inmo Jang, and Joaquin Carrasco
- Subjects
0209 industrial biotechnology ,Computer science ,Headset ,010401 analytical chemistry ,Kinesthetic learning ,02 engineering and technology ,Virtual reality ,Robot end effector ,01 natural sciences ,0104 chemical sciences ,law.invention ,Task (computing) ,020901 industrial engineering & automation ,law ,Human–computer interaction ,Systems architecture ,Task analysis ,Robot - Abstract
This paper proposes a novel telemanipulation system that enables a human operator to control a dual-arm robot. The operation provides kinesthetic teaching via a digital twin of the robot which the operator cyber-physically guides to perform a task. Its key enabler is the concept of a virtual reality interactive marker, which serves as a simplified end effector of the digital twin robot. In virtual reality, the operator can interact with the marker using bare hands, which are sensed by the Leap Motion on top of a virtual reality headset. Then, the status (e.g. position/orientation) of the marker is transformed to the corresponding joint space command to the remote robot so that its end effector can follow the marker. We provide the details of the system architecture, and implement the system based on commercial robots/devices (i.e. UR5, Robotiq gripper, Leap Motion), virtual reality, ROS, and Unity3D. Moreover, the paper discusses the technical challenges that we had to address, and the system’s potential benefits from a human-robot interaction perspective.
- Published
- 2021
- Full Text
- View/download PDF