1. Vision-based fingerspelling recognition for an assistive technology system.
- Author
-
Kapuscinski, Tomasz and Majcher, Marcel
- Subjects
ASSISTIVE technology ,ARTIFICIAL neural networks ,TELECOMMUNICATION systems ,SIGN language ,DEAF people - Abstract
In this paper, a vision-based method for fingerspelled acronyms recognition is proposed. First, the visual information carried by the image is reduced to the person's edges. The heterogeneous and changing background is removed by semantic segmentation of the person using a pre-trained R-CNN Mask. Then, the feature vector is generated by the encoder - part of the autoencoder, previously trained in an unsupervised manner. Finally, the sequence of feature vectors obtained in this way is fed into the input of the Bidirectional Long Short-Term Memory network used for classification. The method is tested on a demanding dataset, including the execution of acronyms fingerspelled by Deaf people who use sign language on a daily basis. Promising results suggest that the method can be incorporated into the Signed Communication System developed by the research team. [ABSTRACT FROM AUTHOR]
- Published
- 2024
- Full Text
- View/download PDF