Back to Search Start Over

Robust Hand Shape Features for Dynamic Hand Gesture Recognition Using Multi-Level Feature LSTM

Authors :
Guee-Sang Lee
Soo-Hyung Kim
Hyung-Jeong Yang
Nhu-Tai Do
Source :
Applied Sciences, Vol 10, Iss 6293, p 6293 (2020), Applied Sciences, Volume 10, Issue 18
Publication Year :
2020
Publisher :
MDPI AG, 2020.

Abstract

This study builds robust hand shape features from the two modalities of depth and skeletal data for the dynamic hand gesture recognition problem. For the hand skeleton shape approach, we use the movement, the rotations of the hand joints with respect to their neighbors, and the skeletal point-cloud to learn the 3D geometric transformation. For the hand depth shape approach, we use the feature representation from the hand component segmentation model. Finally, we propose a multi-level feature LSTM with Conv1D, the Conv2D pyramid, and the LSTM block to deal with the diversity of hand features. Therefore, we propose a novel method by exploiting robust skeletal point-cloud features from skeletal data, as well as depth shape features from the hand component segmentation model in order for the multi-level feature LSTM model to benefit from both. Our proposed method achieves the best result on the Dynamic Hand Gesture Recognition (DHG) dataset with 14 and 28 classes for both depth and skeletal data with accuracies of 96.07% and 94.40%, respectively.

Details

ISSN :
20763417
Volume :
10
Database :
OpenAIRE
Journal :
Applied Sciences
Accession number :
edsair.doi.dedup.....4c4434c924944a58f7b36da676a028d4
Full Text :
https://doi.org/10.3390/app10186293