Search

Recognition of Hungarian conversational telephone speech is challenging due to the informal style and morphological richness of the language. Neural Network Language Models (NNLMs) can provide remedy for the high perplexity of the task; however, their high complexity makes them very difficult to apply in the first (single) pass of an online system. Recent studies showed that a considerable part of the knowledge of NNLMs can be transferred to traditional n-grams by using neural text generation based data augmentation. Data augmentation with NNLMs works well for isolating languages; however, we show that it causes a vocabulary explosion in a morphologically rich language. Therefore, we propose a new, morphology aware neural text augmentation method, where we retokenize the generated text into statistically derived subwords. We compare the performance of word-based and subword-based data augmentation techniques with recurrent and Transformer language models and show that subword-based methods can significantly improve the Word Error Rate (WER) while greatly reducing vocabulary size and memory requirements. Combining subword-based modeling and neural language model-based data augmentation, we were able to achieve 11% relative WER reduction and preserve real-time operation of our conversational telephone speech recognition system. Finally, we also demonstrate that subword-based neural text augmentation outperforms the word-based approach not only in terms of overall WER but also in recognition of Out-of-Vocabulary (OOV) words.

Transformer models have become to state-of-the-art in natural language understanding, their use for language modeling in Automatic Speech Recognition (ASR) is also promising. Albeit Transformer based language models were shown to improve ASR performance, their computational complexity makes their application in real-time systems quite challenging. It has been also shown that the knowledge of such language models can be transferred to traditional n-gram models, suitable for real-time decoding. This paper investigates the adaptation of this transfer approach to morphologically rich languages, and in a real time scenario. We propose a new method for subword-based neural text augmentation with a Transformer language model, which consists in retokenizing the training corpus into subwords, based on a statistical data-driven approach. We demonstrate that ASR performance can be augmented by yet reducing the vocabulary size and alleviating memory consumption.

Modeling the less constrained grammar and word order of conversational speech poses a great challenge to conventional back-off n-gram language models (BNLMs). Recurrent Neural Network Language Models (RNNLMs) can provide much better predictions, however, in real-time Automatic Speech Recognition (ASR) systems (e.g. speech dictation) the process delay due to two-pass decoding can not be tolerated. In this paper, we investigate n-gram-based language modeling techniques that can be applied in a single-pass ASR system to approximate the performance of a RNNLM. Perplexity and word error rate (WER) of BNLMs, BNLM approximation of RNNLMs (RNNBNLM) and RNN n-grams are compared on our Hungarian ASR task. Rich morphology of agglutinative languages (like Hungarian) is often handled by using subword language models, hence we evaluated subword BNLMs, RNN-BNLM and RNN ngrams, as well. It was found that a subword RNN-BNLM can approach the performance of a RNN 4-gram model, and recover roughly 40% of the RNNLM perplexity reduction. All in all, we managed to improve WER of our call center speech transcription system 8% relative without affecting its real-time operation.

This paper summarizes our recent efforts made to transcribe real-life Call Center conversations automatically with respect to non-verbal acoustic events, as well. Future Call Centers – as cognitive infocom systems – must respond automatically not only for well formed utterances but also for spontaneous and non-word speaker manifestations and must be robust against sudden noises. Conversational telephony speech transcription itself is a big challenge, primarily we address this issue on real-life (Bank and Insurance) tasks. In addition, we introduce several non-word acoustic modeling approaches and their integration to LVCSR (Large Vocabulary Continuous Speech Recognition). In the experiments, one and two channel (client and agent speech merged into one or left in two separate audio stream) transcription results, cross-task results and the handling of transcription data insufficiency are investigated – in parallel with the non-verbal acoustic event modeling. On the agent side less than 15% word error rate could be achieved and the best error rate reduction is 20% (relative) due to the inclusion of various written corpora and due to acoustic event handling.

Various morphological and acoustic modeling techniques are evaluated on a less resourced, spontaneous Hungarian large-vocabulary continuous speech recognition (LVCSR) task. Among morphologically rich languages, Hungarian is known for its agglutinative, inflective nature that increases the data sparseness caused by a relatively small training database. Although Hungarian spelling is considered as simple phonological, a large part of the corpus is covered by words pronounced in multiple, phonemically different ways. Data-driven and language specific knowledge supported vocabulary decomposition methods are investigated in combination with phoneme- and grapheme-based acoustic modeling techniques on the given task. Word baseline and morph-based advanced baseline results are significantly outperformed by using both statistical and grammatical vocabulary decomposition methods. Although the discussed morph-based techniques recognize a significant amount of out of vocabulary words, the improvements are due not to this fact but to the reduction of insertion errors. Applying grapheme-based acoustic models instead of phoneme-based models causes no severe recognition performance deteriorations. Moreover, a fully data-driven acoustic modeling technique along with a statistical morphological modeling approach provides the best performance on the most difficult test set. The overall best speech recognition performance is obtained by using a novel word to morph decomposition technique that combines grammatical and unsupervised statistical segmentation algorithms. The improvement achieved by the proposed technique is stable across acoustic modeling approaches and larger with speaker adaptation.

This article describes an objective speech quality assessment method developed for the Hungarian NMT-450 cellular mobile telephone system. The method is based on a psychoacoustic front end followed by a cognitive modeling component. Special problems of the NMT system, such as handover (HOV), the effects of automatic gain control (AGC), and the intrusion of signaling noise, are addressed in the cognitive module. Correlation between the subjective and objective quality measures is maximized by finding a transformation that linearizes their relationship. A correlation of 0.95 is achieved on an independent test set between the subjective speech quality and the proposed objective quality measure.

This article describes the problems encountered during the design and implementation of automatic speech recognition systems for the Hungarian language, proposes practical solutions for treating them, and evaluates their practicality using publicly available databases. The article introduces a rule-based system for modeling the phonological rules inside of words as well as at word boundaries and the notion of stochastic morphological analysis for the treatment of the vocabulary size problem. Finally, the implementation of the proposed methods by the FlexiVoice speech engine is described, and the results of the experimental evaluation on isolated and connected digit recognition, on a 2000-word recognition of Hungarian city names, and on inflected word recognition tasks are summarized.

Senior citizens are in the focus of current research in Europe. This paper investigates the usability aspects of synthetic voices intended for elderly people in Ambient Assisted Living (AAL) systems. The first topic of the study is the selection of an appropriate age of Personal Life Assistant's (PLA) voice intended for active seniors. The second topic is whether the user's own voice is feasible in personal messages. Third, the use of rather short speech corpora from elderly people for HMM speaker adaptation is studied. The question is whether adapted voice is categorized to the same age group by listeners as the original. Corpus based unit-selection TTS and adapted HMM-TTS voices were created from elderly speech samples and these are compared to other middle-aged and elderly voices. In listening tests the synthesized sentences were evaluated and compared to natural speech samples by elderly test subjects. The authors found that the TTS voices of more pleasant (younger) speakers are preferred, HMM-TTS adapted voices of elderly speakers retained age identification features of the original recordings and are suitable for personal messages.

This paper addresses two main issues. First, how to develop broadcast news transcription systems for Central-East European languages in a short time if only restricted language-specific knowledge is available; and second how to improve an already existing system by using on-line learning method. Accordingly, we present recognition results of two newly developed news transcription systems for Polish and Romanian languages, which are trained in fully data-driven manner based on only a few hours of manual transcriptions and web materials. Besides, an automatic language model updating method is also presented for our Hungarian transcription system. Continuous updating of the language model resulted in 2% relative WER (Word Error Rate) reduction measured on a 3 month long period primarily due to better language model parameter matching for IV (Intra Vocabulary) words and secondary due the reduction of OOV (Out Of Vocabulary) words. To the best of our knowledge, the first Romanian broadcast news recognition results are published in this study.

This paper focuses on the integration of non-word acoustic events into LVCSR (Large Vocabulary Continuous Speech Recognition). Non-word events may have an important role in cognitive, paraverbal infocommunication; however, they often are not modeled explicitly due to computational difficulties. In our experiments a serial and a loopback WFST (Weighted Finite State Transducer) architecture was built to recognize and/or print out certain non-word events on the output. We have used a Hungarian Broadcast News corpus to evaluate the results. No performance degradation was observed in terms of normal word recognition accuracy as compared to the baseline, where no non-word event modeling was applied. The non-word event recognition accuracy was, however, lower than expected. One of the most likely reasons may be the less consistent manual transcription as compared to the normal words. Nonetheless, some of the non-word events were mostly correctly recognized. The loopback architecture has higher memory requirement, but gives significantly better non-word event accuracies, without any increase of recognition time.

This paper examines the cognitive inofocommunications preferences of active senior citizens. The study is based on quantitative questionnaire surveys and qualitative workshops held in Budapest in Warsaw within the PAELIFE project [1]. Our findings show that both the user interface and the hardware and software limitations of currently widely used infocommunications tools and services raise barriers even for active Polish and Hungarian seniors of 60–70 years. Possible alternative solutions are proposed.

A crucial part of a speech recognizer is the acoustic feature extraction, especially when the application is intended to be used in noisy environment. In this paper we investigate several novel front-end techniques and compare them to multiple baselines. Recognition tests were performed on studio quality wide band recordings on Hungarian as well as on narrow band telephone speech including real-life noises collected in six languages: English, German, French, Italian, Spanish and Hungarian. The following baseline feature types were used with several settings: Mel Frequency Cepstral Coefficients (MFCC), Perceptual Linear Prediction (PLP) features implemented in HTK, SPHINX, or by ourselves. Novel methods include Perceptual Minimum Variance Distortionless Response (PMVDR) and multiple variations of the Power-Normalized Cepstral Coefficients (PNCC). Also, adaptive techniques are applied to reduce convolutive distortions. We have experienced a significant difference between the MFCC implementations, and there were major differences in the PNCC variations useful in the different bandwidths and noise conditions.

A novel noise robust voice activity detection approach is introduced. The novelty of the method that it uses noise suppressed spectrum of the input signal for spectral entropy calculation. As a result excellent end-pointing performance is observed based on predefined global entropy threshold and time constraints. The effect of frame dropping controlled by the proposed algorithm was investigated on the accuracy of automatic speech recognition. The experiments were performed on Hungarian publicly available noisy and normal telephony speech databases. The relative improvement due to dropping of non-speech frames was positive in all test configurations with a maximum of 29,5%. Besides, in average more than 50% of the frames were dropped.

A coupled acoustic- and language-modeling approach is presented for the recognition of spontaneous speech primarily in agglutinative languages. The effectiveness of the approach in large vocabulary spontaneous speech recognition is demonstrated on the Hungarian MALACH corpus. The derivation of morphs from word forms is based on a statistical morphological segmentation tool while the mapping of morphs into graphemes is obtained trivially by splitting each morph into individual letters. Using morphs instead of words in language modeling gives significant WER reductions in case of both phoneme- and grapheme-based acoustic modeling. The improvements are larger after speaker adaptation of the acoustic models. In conclusion, morphophonemic and the proposed morpho-graphemic ASR approaches yield the same best WERs, which are significantly lower than the word-based baselines but essentially without language dependent rules or pronunciation dictionaries in the latter case. Index Terms: spontaneous speech recognition, morphology.