Back to Search
Start Over
基于时域卷积网络的中文句子级唇语识别算法.
- Source :
-
Application Research of Computers / Jisuanji Yingyong Yanjiu . Sep2023, Vol. 40 Issue 9, p2596-2602. 7p. - Publication Year :
- 2023
-
Abstract
- Existing lip-reading algorithms for Chinese sentences are inadequate at feature extraction and visual ambiguity resolution, which leads to low accuracy. Aiming at this problem, this paper proposed a lip-reading algorithm for Chinese sentences based on temporal convolutional network and 3D convolutional neural network(3DT-CHLipNet).Firstly, it used a Temporal Convolutional Network to extract long-term features from lip dynamics sequences, which has a much larger receptive field than the Long Short Term Memory Network. Secondly, in order to minimize the visual ambiguity in Chinese lipreading, it adopted a Transformer model with the self-attention mechanism to capture the context information and improve the accuracy of sentence prediction. Finally, it introduced a temporal masking data augmentation strategy in the data preprocessing to further reduce the error rate of the algorithm. Comparison experiments on CMLR, the largest open-source Sequence-to-Sequence Chinese Mandarin Lip Reading dataset, show that the improvement in accuracy over representative lip reading algorithms for Chinese sentences ranges from 2.17%to 23.99%. [ABSTRACT FROM AUTHOR]
Details
- Language :
- Chinese
- ISSN :
- 10013695
- Volume :
- 40
- Issue :
- 9
- Database :
- Academic Search Index
- Journal :
- Application Research of Computers / Jisuanji Yingyong Yanjiu
- Publication Type :
- Academic Journal
- Accession number :
- 172372733
- Full Text :
- https://doi.org/10.19734/j.issn.1001-3695.2023.02.0051