Fine-grained hand gesture recognition based on active acoustic signal for VR systems

Hand gestures are the nature and dominant interaction interfaces for VR systems. The state of the art interaction mechanism for VR system either requires expensive sensing devices or suffers from accuracy issues thus hard to perform versatile interactions. In this paper, we leverage Ultragloves, a low cost interaction system using microphone-implanted gloves to extract the hand gestures. With specifically designed signals, we manage to get both the distance and the directions in a relatively accurate manner. We then design a CNN-LSTM like learning algorithm to extract the gestures. Furthermore, to improve the accuracy of recognition, we also design a filter algorithm to filter out noisy data. The implementation shows that our method can recognize four micro-gestures in the accuracy of 82% by combining phase and frequency features.