1. Accurate and efficient 3D hand pose regression for robot hand teleoperation using a monocular RGB camera
- Author
-
Miguel Cazorla, Sergio Orts-Escolano, Francisco Gomez-Donoso, Universidad de Alicante. Departamento de Ciencia de la Computación e Inteligencia Artificial, Universidad de Alicante. Instituto Universitario de Investigación Informática, and Robótica y Visión Tridimensional (RoViT)
- Subjects
0209 industrial biotechnology ,Monocular ,business.industry ,Computer science ,Deep learning ,Frame (networking) ,General Engineering ,Ciencia de la Computación e Inteligencia Artificial ,02 engineering and technology ,Convolutional neural network ,Computer Science Applications ,Hand pose estimation ,020901 industrial engineering & automation ,Artificial Intelligence ,Teleoperation ,0202 electrical engineering, electronic engineering, information engineering ,RGB color model ,020201 artificial intelligence & image processing ,Computer vision ,Robot teleoperation ,Artificial intelligence ,business ,Pose - Abstract
In this paper, we present a novel deep learning-based architecture, which is under the scope of expert and intelligent systems, to perform accurate real-time tridimensional hand pose estimation using a single RGB frame as an input, so there is no need to use multiple cameras or points of view, or RGB-D devices. The proposed pipeline is composed of two convolutional neural network architectures. The first one is in charge of detecting the hand in the image. The second one is able to accurately infer the tridimensional position of the joints retrieving, thus, the full hand pose. To do this, we captured our own large-scale dataset composed of images of hands and the corresponding 3D joints annotations. The proposal achieved a 3D hand pose mean error of below 5 mm on both the proposed dataset and Stereo Hand Pose Tracking Benchmark, which is a public dataset. Our method also outperforms the state-of-the-art methods. We also demonstrate in this paper the application of the proposal to perform a robotic hand teleoperation with high success. This work has been supported by the Spanish Government TIN2016-76515R Grant, supported with Feder funds. This work has also been supported by a Spanish grant for PhD studies ACIF/2017/243
- Published
- 2019