1. Generation of Indian sign language by sentence processing and generative adversarial networks
- Author
-
Neel Vasani, Ruhina Karani, Samip Kalyani, and Pratik Autee
- Subjects
business.industry ,Computer science ,Index (typography) ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,02 engineering and technology ,computer.software_genre ,Notation ,Sentence processing ,03 medical and health sciences ,0302 clinical medicine ,Gesture recognition ,0202 electrical engineering, electronic engineering, information engineering ,Preprocessor ,020201 artificial intelligence & image processing ,Artificial intelligence ,030223 otorhinolaryngology ,business ,Representation (mathematics) ,computer ,Natural language processing ,Generative grammar ,Sentence - Abstract
This paper proposes generating Indian Sign Language from sentences using sentence preprocessing and Generative Adversarial Networks (GANs). The proposed method focuses on converting sentences to brief notations in the form of glosses and generating synthetic video frames for each gloss using a Generative Adversarial Network (GAN) architecture, thus resulting in a video representation for the input sentence. The glosses are mapped to a corresponding minimal skeletal representation, which is used as an input and a base image for the GAN model. The GAN outputs the corresponding video frames, which are further processed to produce a video. The dataset consists of video representations of words present in the Indian Sign Language. The proposed paper measures the performance of the method by using the SSIM index and a side-by-side comparison of real and synthetically generated video frames.
- Published
- 2020
- Full Text
- View/download PDF