A novel system for automatic hand gesture spotting and recognition in stereo color image sequences

Automatic gesture spotting and recognition is a challenging task for locating the start and end points that correspond to a gesture of interest in Human-Computer Interaction. This paper proposes a novel gesture spotting system that is suitable for real-time implementation. The system executes gesture segmentation and recognition simultaneously without any time delay based on Hidden Markov Models. In the segmentation module, the hand of the user is tracked using mean-shift algorithm, which is a non-parametric density estimator that optimizes the smooth similarity function to find the direction of hand gesture path. In order to spot key gesture accurately, a sophisticated method for designing a non-gesture model is proposed, which is constructed by collecting the states of all gesture models in the system. The non-gesture model is a weak model compared to all trained gesture models. Therefore, it provides a good confirmation for rejecting the non-gesture pattern. To reduce the states of the non-gesture model, similar probability distributions states are merged based on relative entropy measure. Experimental results show that the proposed system can automatically recognize isolated gestures with 97.78% and key gestures with 93.31% reliability for Arabic numbers from 0 to 9.