1. Application of Fusion of Various Spontaneous Speech Analytics Methods for Improving Far-Field Neural-Based Diarization
- Author
-
Yuri Matveev, Sergei Astapov, Vladimir Kabarov, Vlada Kapranova, Marina Volkova, Elena Timofeeva, Aleksei Gusev, Valeriia Zaluskaia, Elena Evseeva, and Aleksei Logunov
- Subjects
Voice activity detection ,Artificial neural network ,business.industry ,Computer science ,General Mathematics ,Speech recognition ,speaker number estimation ,speaker diarization ,spontaneous speech processing ,voice activity detection ,overlapping speech detection ,speaker extractor models ,model fusion ,quality estimation ,distant speech processing ,artificial neural networks ,Speaker diarisation ,Analytics ,Computer Science (miscellaneous) ,QA1-939 ,Speech analytics ,Noise (video) ,Transcription (software) ,Cluster analysis ,business ,Engineering (miscellaneous) ,Mathematics - Abstract
Recently developed methods in spontaneous speech analytics require the use of speaker separation based on audio data, referred to as diarization. It is applied to widespread use cases, such as meeting transcription based on recordings from distant microphones and the extraction of the target speaker’s voice profiles from noisy audio. However, speech recognition and analysis can be hindered by background and point-source noise, overlapping speech, and reverberation, which all affect diarization quality in conjunction with each other. To compensate for the impact of these factors, there are a variety of supportive speech analytics methods, such as quality assessments in terms of SNR and RT60 reverberation time metrics, overlapping speech detection, instant speaker number estimation, etc. The improvements in speaker verification methods have benefits in the area of speaker separation as well. This paper introduces several approaches aimed towards improving diarization system quality. The presented experimental results demonstrate the possibility of refining initial speaker labels from neural-based VAD data by means of fusion with labels from quality estimation models, overlapping speech detectors, and speaker number estimation models, which contain CNN and LSTM modules. Such fusing approaches allow us to significantly decrease DER values compared to standalone VAD methods. Cases of ideal VAD labeling are utilized to show the positive impact of ResNet-101 neural networks on diarization quality in comparison with basic x-vectors and ECAPA-TDNN architectures trained on 8 kHz data. Moreover, this paper highlights the advantage of spectral clustering over other clustering methods applied to diarization. The overall quality of diarization is improved at all stages of the pipeline, and the combination of various speech analytics methods makes a significant contribution to the improvement of diarization quality.
- Published
- 2021
- Full Text
- View/download PDF