Search

Adaptive training is a widely used technique for building speech recognition systems on nonhomogeneous training data. Recently, there has been interest in applying these approaches for situations where there is significant levels of background noise in the training data. Various schemes for adaptive training are based on noise-, or speaker-, specific transforms of features to yield estimates of the clean speech. However, when there are high levels of background noise, these clean speech estimates may be poor resulting in degradations in performance. In this paper, a new approach for adaptive training on noise-corrupted training data is presented. It extends a popular form of linear transform for model-based adaptation and adaptive training, constrained MLLR (CMLLR), to reflect additional uncertainty from noise-corrupted observations. This new form of adaptation transform is called noisy CMLLR (NCMLLR). NCMLLR uses a modified version of generative model between clean speech and noisy observation, similar to factor analysis (FA). However, in contrast to FA here the generative model describes an adaptation transform, rather than a covariance matrix structure. The use of NCMLLR for adaptive training using an expectation–maximization approach is described. Discriminative adaptive training with NCMLLR is also described based on the minimum phone error criterion. Experimental results comparing NCMLLR with standard adaptive training schemes are given on a noise-corrupted version of Resource Management, the ARPA 1994 CSRNAB Spoke 10 task, and in-car recorded data. [ABSTRACT FROM PUBLISHER]

This paper considers the problem of rapid and robust speaker adaptation in Automatic Speech Recognition (ASR) systems. We propose an approach using combination of eigenspace-based maximum likelihood linear regression (EMLLR) and Minimum Classification Error (MCE). MCE is used to estimate the coefficients of eigenvoices as an alternative to maximum likelihood. Experimental results on TIMIT database illustrate that using MCE leads to up to 1.9% improvement in phoneme error rate. Moreover, it is shown that increasing the amounts of adaptation data has less effect in proposed method than conventional EMLLR. Furthermore, the results achieved using genetic algorithm (GA) to estimate eigenvoices coefficients are compared to proposed method using MCE. The adaptation results using MCE show better performance than GA. [ABSTRACT FROM AUTHOR]

In conventional speech synthesis, large amounts of phonetically balanced speech data recorded in highly controlled recording studio environments are typically required to build a voice. Although using such data is a straightforward solution for high quality synthesis, the number of voices available will always be limited, because recording costs are high. On the other hand, our recent experiments with HMM-based speech synthesis systems have demonstrated that speaker-adaptive HMM-based speech synthesis (which uses an "average voice model" plus model adaptation) is robust to non-ideal speech data that are recorded under various conditions and with varying microphones, that are not perfectly clean, and/or that lack phonetic balance. This enables us to consider building high-quality voices on "non-TTS" corpora such as ASR corpora. Since ASR corpora generally include a large number of speakers, this leads to the possibility of producing an enormous number of voices automatically. In this paper, we demonstrate the thousands of voices for HMM-based speech synthesis that we have made from several popular ASR corpora such as the Wall Street Journal (WSJ0, WSJ1, and WSJCAM0), Resource Management, Globalphone, and SPEECON databases. We also present the results of associated analysis based on perceptual evaluation, and discuss remaining issues. [ABSTRACT FROM AUTHOR]

Abstract: Speaker-dependent modeling has a long history in speech recognition, but has received less attention in speech understanding. This study explores speaker-specific modeling for the task of automatic segmentation of speech into dialog acts (DAs), using a linear combination of speaker-dependent and speaker-independent language and prosodic models. Data come from 20 frequent speakers in the ICSI meeting corpus; adaptation data per speaker ranges from 5k to 115k words. We compare performance for both reference transcripts and automatic speech recognition output. We find that: (1) speaker adaptation in this domain results both in a significant overall improvement and in improvements for many individual speakers, (2) the magnitude of improvement for individual speakers does not depend on the amount of adaptation data, and (3) language and prosodic models differ both in degree of improvement, and in relative benefit for specific DA classes. These results suggest important future directions for speaker-specific modeling in spoken language understanding tasks. [Copyright &y& Elsevier]

This letter proposes a novel framework for speaker adaptation, using bilinear model-based maximum likelihood linear regression (MLLR) method. First, a set of speaker models is decomposed into the style factor identified as each speaker's characteristics and the common content factor across the speakers, by the bilinear model. Then, using some adaptation data from a new speaker, the speaker-specific model is generated by properly adjusting the dimensionality of the content factor and estimating a new style factor simultaneously. Experimental results show that the proposed framework outperforms MLLR with fewer number of parameters to be estimated. [ABSTRACT FROM AUTHOR]

Abstract: To automate assessments of beginning readers, especially those still learning English, we have investigated the types of knowledge sources that teachers use and have tried to incorporate them into an automated system. We describe a set of speech recognition and verification experiments and compare teacher scores with automatic scores in order to decide when a novel pronunciation is best viewed as a reading error or as dialect variation. Since no one classroom teacher is expected to be familiar with as many dialect systems as might occur in an urban classroom, making progress in automated assessments in this area can improve the consistency and fairness of reading assessment. We found that automatic methods performed best when the acoustic models were trained on both native and non-native speech, and argue that this training condition is necessary for automatic reading assessment since a child’s reading ability is not directly observable in one utterance. We also found assessment of emerging reading skills in young children to be an area ripe for more research! [Copyright &y& Elsevier]

Abstract: In recent years, various CALL systems which can evaluate a learner’s pronunciation using speech recognition technology have been proposed. In order to evaluate a learner’s utterances and point out problems with higher accuracy, speaker adaptation is a promising technology. However, many learners who use the CALL system often have very poor speaking ability in the target language (L2), so conventional speaker adaptation methods have problems because they require the learners’ correctly-pronounced L2 utterances for adaptation. In this paper, we propose two new types of speaker adaptation methods for the CALL system. The new methods only require the learners’ utterances in their native language (L1) for adapting the acoustic model for L2. The first method is an algorithm to adapt acoustic models using a bilingual speaker’s utterances. The speaker-independent acoustic models of L1 and L2 are adapted to the bilingual speaker once, then they are adapted to the learner again using the learner’s L1 utterances. Using this method, we obtained about 5-point higher phoneme recognition accuracy than the baseline method. The second method is a training algorithm of a set of acoustic models based on speaker adaptive training. It can robustly train bilinguals’ models using a few utterances in L1 and L2 uttered by bilingual speakers. Using this method, we obtained about 10-point higher phoneme recognition accuracy than the baseline method. [Copyright &y& Elsevier]

This paper proposes a nonlinear generalization of the popular maximum-likelihood linear regression (MLLR) adaptation algorithm using kernel methods. The proposed method, called maximum penalized likelihood kernel regression adaptation (MPLKR), applies kernel regression with appropriate regularization to determine the affine model transform in a kernel-induced high-dimensional feature space. Although this is not the first attempt of applying kernel methods to conventional linear adaptation algorithms, unlike most of other kernelized adaptation methods such as kernel eigenvoice or kernel eigen-MLLR, MPLKR has the advantage that it is a convex optimization and its solution is always guaranteed to be globally optimal. In fact, the adapted Gaussian means can be obtained analytically by simply solving a system of linear equations. From the Bayesian perspective, MPLKR can also be considered as the kernel version of maximum a posteriori linear regression (MAPLR) adaptation. Supervised and unsupervised speaker adaptation using MPLKR were evaluated on the Resource Management and Wall Street Journal 5K tasks, respectively, achieving a word error rate reduction of 23.6% and 15.5% respectively over the speaker-independently model. [ABSTRACT FROM AUTHOR]

This paper describes a speaker-adaptive HMM-based speech synthesis system. The new system, called "HTS-2007," employs speaker adaptation (CSMAPLR+MAP), feature-space adaptive training, mixed-gender modeling, and full-covariance modeling using CSMAPLR transforms, in addition to several other techniques that have proved effective in our previous systems. Subjective evaluation results show that the new system generates significantly better quality synthetic speech than speaker-dependent approaches with realistic amounts of speech data, and that it bears comparison with speaker-dependent approaches even when large amounts of speech data are available. In addition, a comparison study with several speech synthesis techniques shows the new system is very robust: It is able to build voices from less-than-ideal speech data and synthesize good-quality speech even for out-of-domain sentences. [ABSTRACT FROM AUTHOR]

Abstract: We introduce a strategy for modeling speaker variability in speaker adaptation based on maximum likelihood linear regression (MLLR). The approach uses a speaker-clustering procedure that models speaker variability by partitioning a large corpus of speakers in the eigenspace of their MLLR transformations and learning cluster-specific regression class tree structures. We present experiments showing that choosing the appropriate regression class tree structure for speakers leads to a significant reduction in overall word error rates in automatic speech recognition systems. To realize these gains in unsupervised adaptation, we describe an algorithm that produces a linear combination of MLLR transformations from cluster-specific trees using weights estimated by maximizing the likelihood of a speaker’s adaptation data. This algorithm produces small improvements in overall recognition performance across a range of tasks for both English and Mandarin. More significantly, distributional analysis shows that it reduces the number of speakers with performance loss due to adaptation across a range of adaptation data sizes and word error rates. [Copyright &y& Elsevier]

In realizing a speech recognition system robust to variation of speakers, a reliable adaptation algorithm is needed. Most adaptation techniques require a large amount of adaptation data from the target speaker to carry out the adaptation task. With the time needed to gather and transcribe adaptation utterances together with the time to execute adaptation, application to speech recognition is limited. We propose a rapid approach to speaker adaptation. We employ HMM-Sufficient Statistics in storing speaker-dependent subspaces. N-Closest speaker selection is employed in resolving the combinatorics of the speaker-dependent subspaces during recognition. This approach allows the adapted model to have a direct correspondence with the target speaker by using the target speakers’ utterance for the N-Closest speaker selection. The proposed method employs series of adaptation processes. First, the general model is trained, then adapted to broad gender/age classes, which are further adapted to speaker-specific data. Since HMM-Sufficient Statistics are pre-computed offline, little computation is needed in carrying out the adaptation task online. Moreover, the method requires only a single arbitrary utterance from the target speaker for adaptation. In this paper, we discuss the modification, expansion, and the improvement of rapid adaptation based on HMM-Sufficient Statistics in the framework of Baum-Welch and maximum likelihood linear regression (MLLR). Experimental results using the conventional MLLR, speaker-adaptive training, and CMLLR are evaluated and compared. We also tested for robustness in office, car, crowd and booth environments in several SNR conditions. [Copyright &y& Elsevier]

Abstract: Vocal tract length normalization (VTLN) for standard filterbank-based Mel frequency cepstral coefficient (MFCC) features is usually implemented by warping the center frequencies of the Mel filterbank, and the warping factor is estimated using the maximum likelihood score (MLS) criterion. A linear transform (LT) equivalent for frequency warping (FW) would enable more efficient MLS estimation. We recently proposed a novel LT to perform FW for VTLN and model adaptation with standard MFCC features. In this paper, we present the mathematical derivation of the LT and give a compact formula to calculate it for any FW function. We also show that our LT is closely related to different LTs previously proposed for FW with cepstral features, and these LTs for FW are all shown to be numerically almost identical for the sine-log all-pass transform (SLAPT) warping functions. Our formula for the transformation matrix is, however, computationally simpler and, unlike other previous LT approaches to VTLN with MFCC features, no modification of the standard MFCC feature extraction scheme is required. In VTLN and speaker adaptive modeling (SAM) experiments with the DARPA resource management (RM1) database, the performance of the new LT was comparable to that of regular VTLN implemented by warping the Mel filterbank, when the MLS criterion was used for FW estimation. This demonstrates that the approximations involved do not lead to any performance degradation. Performance comparable to front end VTLN was also obtained with LT adaptation of HMM means in the back end, combined with mean bias and variance adaptation according to the maximum likelihood linear regression (MLLR) framework. The FW methods performed significantly better than standard MLLR for very limited adaptation data (1 utterance), and were equally effective with unsupervised parameter estimation. We also performed speaker adaptive training (SAT) with feature space LT denoted CLTFW. Global CLTFW SAT gave results comparable to SAM and VTLN. By estimating multiple CLTFW transforms using a regression tree, and including an additive bias, we obtained significantly improved results compared to VTLN, with increasing adaptation data. [Copyright &y& Elsevier]

In this paper, we analyze the effects of several factors and configuration choices encountered during training and model construction when we want to obtain better and more stable adaptation in HMM-based speech synthesis. We then propose a new adaptation algorithm called constrained structural maximum a posteriori linear regression (CSMAPLR) whose derivation is based on the knowledge obtained in this analysis and on the results of comparing several conventional adaptation algorithms. Here, we investigate six major aspects of the speaker adaptation: initial models; the amount of the training data for the initial models; the transform functions, estimation criteria, and sensitivity of several linear regression adaptation algorithms; and combination algorithms. Analyzing the effect of the initial model, we compare speaker-dependent models, gender-independent models, and the simultaneous use of the gender-dependent models to single use of the gender-dependent models. Analyzing the effect of the transform functions, we compare the transform function for only mean vectors with that for mean vectors and covariance matrices. Analyzing the effect of the estimation criteria, we compare the ML criterion with a robust estimation criterion called structural MAP. We evaluate the sensitivity of several thresholds for the piecewise linear regression algorithms and take up methods combining MAP adaptation with the linear regression algorithms. We incorporate these adaptation algorithms into our speech synthesis system and present several subjective and objective evaluation results showing the utility and effectiveness of these algorithms in speaker adaptation for HMM-based speech synthesis. [ABSTRACT FROM AUTHOR]

This work presents a fuzzy logic control (FLC) mechanism for the widely adopted vector field smoothing (VFS) speaker adaptation scheme. The proposed mechanism, called FLC-VFS, regulates the influence of VFS adaptation when the training data from a new speaker is inadequate, specifically by taking the amount of adaptation data into account at the stage of transfer vectors interpolation, and thus ensures the robustness of VFS adaptation against data scarcity. The proposed mechanism is conceptually simple and effective. Experimental results indicate that FLC-VFS outperforms conventional VFS, particularly when the adaptation data are scarce. [ABSTRACT FROM AUTHOR]

Abstract: Past research has established that listeners can accommodate a wide range of talkers in understanding language. How this adjustment operates, however, is a matter of debate. Here, listeners were exposed to spoken words from a speaker of an American English dialect in which the vowel /æ/ is raised before /g/, but not before /k/. Results from two experiments showed that listeners’ identification of /k/-final words like back (which are unaffected by the dialect) was facilitated by prior exposure to their dialect-affected /g/-final counterparts, e.g., bag. This facilitation occurred because the competition between interpretations, e.g., bag or back, while hearing the initial portion of the input [bæ], was mitigated by the reduced probability for the input to correspond to bag as produced by this talker. Thus, adaptation to an accent is not just a matter of adjusting the speech signal as it is being heard; adaptation involves dynamic adjustment of the representations stored in the lexicon, according to the characteristics of the speaker or the context. [Copyright &y& Elsevier]

Abstract: In this paper, the use of discriminative linear transforms (DLT) is investigated to construct speaker adaptive speech recognition systems, where a discriminative criterion rather than ML is used for transform parameter estimation. The minimum phone error (MPE) criterion is investigated for DLT estimation, by making use of a so-called weak-sense auxiliary function to derive the estimation formulae. An implementation based on lattices is used for DLT statistics accumulation, where the use of a weakened language model allows more confusion data to be included. To improve DLT estimation for unsupervised adaptation, a method of incorporating word correctness information of the supervision into transform estimation is developed. The confidence scores calculated by confusion network decoding are used to represent the word correctness and weight the numerator statistics during DLT estimation. This makes the DLT estimation less sensitive to errors in the supervision. Experiments on transcription of read newspaper data and on conversational telephone speech transcription have shown the improvements of DLT over MLLR for both supervised and unsupervised adaptation, and the effectiveness of confidence scores for improving both normal and DLT-based MLLR adaptation. [Copyright &y& Elsevier]

Speaker space-based adaptation methods for automatic speech recognition have been shown to provide significant performance improvements for tasks where only a few seconds of adaptation speech is available. However, these techniques are not widely used in practical applications because they require large amounts of speaker-dependent training data and large amounts of computer memory. The authors propose a robust, low-complexity technique within this general class that has been shown to reduce word error rate, reduce the large storage requirements associated with speaker space approaches, and eliminate the need for large numbers of utterances per speaker in training. The technique is based on representing speakers as a linear combination of clustered linear basis vectors and a procedure is presented for maximum-likelihood estimation of these vectors from training data. Significant word error rate reduction was obtained using these methods relative to speaker independent performance for the Resource Management and Wall Street Journal task domains. Rapid Speaker Adaptation Using Clustered Maximum-Likelihood Linear Basis With Sparse Training Data. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a fuzzy control mechanism for conventional maximum likelihood linear regression (MLLR) speaker adaptation, called FLC-MLLR, by which the effect of MLLR adaptation is regulated according to the availability of adaptation data in such a way that the advantage of MLLR adaptation could be fully exploited when the training data are sufficient, or the consequence of poor MLLR adaptation would be restrained otherwise. The robustness of MLLR adaptation against data scarcity is thus ensured. The proposed mechanism is conceptually simple and computationally inexpensive and effective; the experiments in recognition rate show that FLC-MLLR outperforms standard MLLR especially when encountering data insufficiency and performs better than MAPLR at much less computing cost. [Copyright &y& Elsevier]

Spectral mismatch between training and testing utterances can cause significant degradation in the performance of automatic speech recognition (ASR) systems. Speaker adaptation and speaker normalization techniques are usually applied to address this issue. One way to reduce spectral mismatch is to reshape the spectrum by aligning corresponding formant peaks. There are various levels of mismatch in formant structures. In this paper, regression-tree-based phoneme- and state-level spectral peak alignment is proposed for rapid speaker adaptation using linearization of the vocal tract length normalization (VTLN) technique. This method is investigated in a maximum-likelihood linear regression (MLLR)-like framework, taking advantage of both the efficiency of frequency warping (VTLN) and the reliability of statistical estimations (MLLR). Two different regression classes are investigated: one based on phonetic classes (using combined knowledge and data-driven techniques) and the other based on Gaussian mixture classes. Compared to MLLR, VTLN, and global peak alignment, improved performance can be obtained for both supervised and unsupervised adaptations for both medium vocabulary (the RM1 database) and connected digits recognition (the TIDIGITS database) tasks. Performance improvements are largest with limited adaptation data which is often the case for ASR applications, and these improvements are shown to be statistically significant. [ABSTRACT FROM AUTHOR]

This paper focuses on the adaptation of Automatic Speech Recognition systems using Hybrid models combining Artificial Neural Networks (ANN) with Hidden Markov Models (HMM). Most adaptation techniques for ANNs reported in the literature consist in adding a linear transformation network connected to the input of the ANN. This paper describes the application of linear transformations not only to the input features, but also to the outputs of the internal layers. The motivation is that the outputs of an internal layer represent discriminative features of the input pattern suitable for the classification performed at the output of the ANN. In order to reduce the effect due to the lack of adaptation samples for some phonetic units we propose a new solution, called Conservative Training. Supervised adaptation experiments with different corpora and for different types of adaptation are described. The results show that the proposed approach always outperforms the use of transformations in the feature space and yields even better results when combined with linear input transformations. [Copyright &y& Elsevier]

This paper examines the susceptibility of a dictation system to various types of mismatches between the training and testing conditions. With these experiments we intend to find the best training configuration for the system and also to evaluate the efficiency of the speaker adaptation algorithm we use. The paper first presents the components of the dictation system, and then describes a set of training and recognition experiments where we vary the microphones and create gender-dependent and speaker-dependent models. In each case we examine how much the recognition performance can be improved further by speaker adaptation. We conclude that the best and most reliable scores can be obtained by using gender-dependent phone models in combination with speaker adaptation. Speaker adaptation results in great improvements in almost every case. However, our results do not confirm the assumption that the use of one microphone is better than the use of several. [ABSTRACT FROM AUTHOR]

This paper addresses the issue of speaker variability and session variability in text-independent Gaussian mixture model (GMM)-based speaker verification. A speaker model adaptation procedure is proposed which is based on a joint factor analysis approach to speaker verification. It is shown in this paper that this approach facilitates the implementation of a progressive unsupervised adaptation strategy which is able to produce an improved model of speaker identity while minimizing the influence of channel variability. The paper also deals with the interaction between this model adaptation approach and score normalization strategies which act to reduce the variation in likelihood ratio scores. This issue is particularly important in establishing decision thresholds in practical speaker verification systems since the variability of likelihood ratio scores can increase as a result of progressive model adaptation. These adaptation methods have been evaluated under the adaptation paradigm defined under the NIST 2005 Speaker Recognition Evaluation Plan, which is based on conversation sides derived from telephone speech utterances. It was found that when target speaker models were trained from a single conversation, an equal error rate (EER) of 4.5% was obtained under the NIST unsupervised speaker adaptation scenario. [ABSTRACT FROM AUTHOR]

The maximum likelihood linear regression (MLLR) technique is widely used in speaker adaptation due to its effectiveness and computational advantages. When the adaptation data are sparse, MLLR performance degrades because of unreliable parameter estimation. In this paper, a robust MLLR speaker adaptation approach via weighted model averaging is investigated. A variety of transformation structures is first chosen and a general form of maximum likelihood (ML) estimation of the structures is given. The minimum description length (MDL) principle is applied to account for the compromise between transformation granularity and descriptive ability regarding the tying patterns of structured transformations with a regression tree. Weighted model averaging across the candidate structures is then performed based on the normalized MDL scores. Experimental results show that this kind of model averaging in combination with regression tree tying gives robust and consistent performance across various amounts of adaptation data. [ABSTRACT FROM AUTHOR]

In this paper, we present a formulation of minimum classification error linear regression (MCELR) for the adaptation of Gaussian mixture continuous-density hidden Markov model (CDHMM) parameters. Two optimization approaches, namely generalized probabilistic descent (GPD) and Quickprop are studied and compared for the optimization of the MCELR objective function. The effectiveness of the proposed MCELR technique is confirmed via a series of supervised speaker adaptation experiments on a task of continuous Putonghua (Mandarin Chinese) speech recognition. [ABSTRACT FROM AUTHOR]

Abstract: This paper discusses and optimizes an HMM/GMM based User-Customized Password Speaker Verification (UCP-SV) system. Unlike text-dependent speaker verification, in UCP-SV systems, customers can choose their own passwords with no lexical constraints. The password has to be pronounced a few times during the enrollment step to create a customer dependent model. Although potentially more “user-friendly”, such systems are less understood and actually exhibit several practical issues, including automatic HMM inference, speaker adaptation, and efficient likelihood normalization. In our case, HMM inference (HMM topology) is performed using hybrid HMM/MLP systems, while the parameters of the inferred model, as well as their adaptation, will use GMMs. However, the evaluation of a UCP-SV baseline system shows that the background model used for likelihood normalization is the main difficulty. Therefore, to circumvent this problem, the main contribution of the paper is to investigate the use of multiple reference models for customer acoustic modeling and multiple background models for likelihood normalization. In this framework, several scoring techniques are investigated, such as Dynamic Model Selection (DMS) and fusion techniques. Results on two different experimental protocols show that an appropriate selection criteria for customer and background models can improve significantly the UCP-SV performance, making the UCP-SV system quite competitive with a text-dependent SV system. Finally, as customers’ passwords are short, a comparative experiment using the conventional GMM–UBM text-independent approach is also conducted. [Copyright &y& Elsevier]

We present an overview of the most common techniques used in automatic speech recognition to adapt a general system to a different environment (known as cross-task adaptation) such as in an air traffic control system (ATC). The conditions present in ATC are very specific: very spontaneous, the presence of noise, and high speed speech. So, with a typical speech recognizer the recognition results are unsatisfactory. We have to decide on the best option for the modeling: to develop acoustic models specific to those conditions from scratch using the data available for the new environment, or to carry out cross-task adaptation starting from reliable HMM models (usually requiring less data in the target domain). We begin with a description of the main techniques considered for cross-task adaptation, namely maximum a posteriori (MAP), maximum likelihood linear regression (MLLR), and the two together. We have applied each in two speech recognizers for air traffic control tasks, one for spontaneous speech and the other for a command interface. We show the performance of these techniques and compare them with the development of a new system from scratch. We also show the results obtained for speaker adaptation using a variable amount of adaptation data. The main conclusion is that MLLR can outperform MAP when a large number of transforms is used, and MLLR followed by MAP is the best option. All of these techniques are better than developing a new system from scratch, showing the effectiveness of mean and variance adaptation [ABSTRACT FROM PUBLISHER]

The objective of voice conversion algorithms is to modify the speech by a particular source speaker so that it sounds as if spoken by a different target speaker. Current conversion algorithms employ a training procedure, during which the same utterances spoken by both the source and target speakers are needed for deriving the desired conversion parameters. Such a (parallel) corpus, is often difficult or impossible to collect. Here, we propose an algorithm that relaxes this constraint, i.e., the training corpus does not necessarily contain the same utterances from both speakers. The proposed algorithm is based on speaker adaptation techniques, adapting the conversion parameters derived for a particular pair of speakers to a different pair, for which only a nonparallel corpus is available. We show that adaptation reduces the error obtained when simply applying the conversion parameters of one pair of speakers to another by a factor that can reach 30%. A speaker identification measure is also employed that more insightfully portrays the importance of adaptation, while listening tests confirm the success of our method. Both the objective and subjective tests employed, demonstrate that the proposed algorithm achieves comparable results with the ideal case when a parallel corpus is available. [ABSTRACT FROM AUTHOR]

We present a new discriminative linear regression adaptation algorithm for hidden Markov model (HMM) based speech recognition. The cluster-dependent regression matrices are estimated from speaker-specific adaptation data through maximizing the aggregate a posteriori probability, which can be expressed in a form of classification error function adopting the logarithm of posterior distribution as the discriminant function. Accordingly, the aggregate a posteriori linear regression (AAPLR) is developed for discriminative adaptation where the classification errors of adaptation data are minimized. Because the prior distribution of regression matrix is involved, AAPLR is geared with the Bayesian learning capability. We demonstrate that the difference between AAPLR discriminative adaptation and maximum a posteriori linear regression (MAPLR) adaptation is due to the treatment of the evidence. Different from minimum classification error linear regression (MCELR), AAPLR has closed-form solution to fulfill rapid adaptation. Experimental results reveal that AAPLR speaker adaptation does improve speech recognition performance with moderate computational cost compared to maximum likelihood linear regression (MLLR). MAPLR, MCELR and conditional maximum likelihood linear regression (CMLLR). These results are verified for supervised adaptation as well as unsupervised adaptation for different numbers of adaptation data. [ABSTRACT FROM AUTHOR]

Abstract: Avatars in many applications are constructed manually or by a single speech-driven model which needs a lot of training data and long training time. It is essential to build up a user-dependent model more efficiently. In this paper, a new adaptation method, called the partial linear regression (PLR), is proposed and adopted in an audio-driven talking head application. This method allows users to adapt the partial parameters from the available adaptive data while keeping the others unchanged. In our experiments, the PLR algorithm can retrench the hours of time spent on retraining a new user-dependent model, and adjust the user-independent model to a more personalized one. The animated results with adapted models are 36% closer to the user-dependent model than using the pre-trained user-independent model. [Copyright &y& Elsevier]

In speech recognition, speaker adaptation refers to the range of techniques whereby a speech recognition system is adapted to the acoustic features of a specific user using a small sample of utterances from that user. In recent years the practical development of speaker-independent speech recognition systems using continuous density hidden Markov models has seen significant progress; however, the recognition performance of these systems has not yet reached that of speaker-dependent speech recognition systems in which a user's speech is registered beforehand. Much hope has therefore been placed on the establishment of speaker adaptation techniques that can bring performance of a speaker-independent system up to that of a speaker-dependent one using the smallest amounts of data. In this paper we present a survey of previous research into speaker adaptation techniques focusing particularly on three important approaches in this area: maximum a posteriori (MAP) parameter estimation, maximum likelihood linear regression (MLLR), and eigenvoices. We also discuss approaches that combine these techniques in a lateral fashion. © 2005 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 88(12): 25–42, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ ecjc.20207 [ABSTRACT FROM AUTHOR]

A novel model-based speaker adaptation algorithm, support speaker weighting (SSW), was proposed for rapid speaker adaptation in speech recognition systems. It realizes the specific speaker selection and dimensionality reduction by computing the support speaker subsets from many reference speakers. This method yields major improvements in performance for tiny amounts of adaptation data while greatly reducing the memory requirement. The experiments on the supervised adaptation demonstrate that the relative error rate reduction of 5.82 % is achieved when only one adaptation sentence is available. In comparison with other rapid speaker adaptation algorithms, SSW is more effective. [ABSTRACT FROM AUTHOR]

Regarding the drawbacks of eigenvoice speaker adaptation, a novel adaptation algorithm was proposed, which is called discriminative structural eigenvoice (DSEV). This algorithm can adjust the adaptation parameters with the available amount of adaptation data and discriminatively estimate them under the maximum mutual information (MMI) criterion, which further improves the adaptation performance. It can be shown from the experimental results of supervised adaptation that this algorithm achieves 6.7% relative reduction in error rate with only one adaptation sentence. The proposed method also shows better gradual improvement than eigenvoice with increased amount of adaptation data. [ABSTRACT FROM AUTHOR]

In order to achieve high-precision speech recognition in real environments phone model adaptation procedures that can rapidly account for a wide range of different speakers and acoustic noise conditions are required. In this paper we propose an unsupervised speaker adaptation method that extends an unsupervised speaker and environment adaptation method based on sufficient statistics from HMMs by performing spectral subtraction and then adding a known noise to the input. Existing methods assume that a model is trained to match each of the different types of background noise that will be the object of recognition and do not consider variations in the signal-to-noise ratio or changes in the background noise for given inputs. In contrast, our method constrains the noise of the input data using an estimation of the noise spectra and then adds a known stable noise to the bleached noise that remains in the input, thereby smoothing out differences between background noises and enabling us to perform recognition with a single set of acoustic models. In addition, with regard to speaker adaptation, we select the set of closest speakers from our database on the basis of a single arbitrary utterance from the test speaker and retrain the acoustic models using the sufficient statistics of those speakers. By combining these two methods we are able to rapidly and accurately adapt to a new speaker. In recognition experiments with a signal-to-noise ratio of 20 dB and in a variety of noise conditions, the proposed method resulted in a recognition rate of 2 percent more than a speaker-independent model matched to the test noise environment for each noise environment, achieving an average recognition performance of 85.1 percent overall. In addition, we conducted a comparison of our method with a standard supervised adaptation technique: maximum likelihood linear regression (MLLR). © 2005 Wiley Periodicals, Inc. Electron Comm Jpn Pt 2, 88(8): 30–41, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjb.20199 [ABSTRACT FROM AUTHOR]

In conventional speaker recognition tasks, the amount of training data is almost the same for each speaker, and the speaker model structure is uniform and specified manually according to the nature of the task and the available size of the training data. In real-world speech data such as telephone conversations and meetings, however, serious problems arise in applying a uniform model because variations in the utterance durations of speakers are large, with numerous short utterances. We therefore propose a flexible framework in which an optimal speaker model (GMM or VQ) is automatically selected based on the Bayesian Information Criterion (BIC) according to the amount of training data available. The framework makes it possible to use a discrete model when the data is sparse, and to seamlessly switch to a continuous model after a large amount of data is obtained. The proposed framework was implemented in unsupervised speaker indexing of a discussion audio. For a real discussion archive with a total duration of 10 hours, we demonstrate that the proposed method has higher indexing performance than that of conventional methods. The speaker index is also used to adapt a speaker-independent acoustic model to each participant for automatic transcription of the discussion. We demonstrate that speaker indexing with our method is sufficiently accurate for adaptation of the acoustic model. [ABSTRACT FROM AUTHOR]

Eigenvoice techniques have been proposed to provide rapid speaker adaptation with very limited adaptation data, but the performance may be saturated when more adaptation data become available. This is because in these techniques an eigenspace with reduced dimensionality is established by properly utilizing the a priori knowledge from the large quantity of training data. The reduced dimensionality of the eigenspace requires less adaptation data to estimate the model parameters for the new speaker, but also makes it less easy to obtain more precise models with more adaptation data. In this paper, a new segmental eigenvoice approach is proposed, in which the eigenspace can be further segmented into N subeigenspaces by properly classifying the model parameters into N clusters. These N subeigenspaces can help to construct a more delicate eigenspace and more precise models when more adaptation data are available. It will be shown that there can be at least mixture-based, model-based and feature-based segmental eigenvoice approaches. Not only improved performance can be obtained, but these different approaches can be properly integrated to offer better performance. Two further approaches leading to improved segmental eigenvoice techniques with even better performance are also proposed. The experiments were performed with both a large vocabulary and a small vocabulary recognition tasks. [ABSTRACT FROM AUTHOR]

We derive an exact solution to the problem of maximum likelihood estimation of the supervector covariance matrix used in extended MAP (or EMAP) speaker adaptation and show how it can be regarded as a new method of eigenvoice estimation. Unlike other approaches to the problem of estimating eigenvoices in situations where speaker-dependent training is not feasible, our method enables us to estimate as many eigenvoices from a given training set as there are training speakers. In the limit as the amount of training data for each speaker tends to infinity, it is equivalent to cluster adaptive training. [ABSTRACT FROM AUTHOR]

We describe a new method of estimating speaker-dependent hidden Markov models for speakers in a closed population. Our method differs from previous approaches in that it is based on an explicit model of the correlations between all of the speakers in the population, the idea being that if there is not enough data to estimate a Gaussian mean vector for a given speaker then data from other speakers can be used provided that we know how the speakers are correlated with each other. We explain how to estimate inter-speaker correlations using a Kullback-Leibler divergence minimization technique which can be applied to the problem of estimating the parameters of all of the hyperdistributions that are currently used in Bayesian speaker adaptation. [ABSTRACT FROM AUTHOR]

A transformation matrix linear interpolation (TMLI) approach for speaker adaptation is proposed. TMLI uses the transformation matrixes produced by MLLR from selected training speakers and the testing speaker. With only 3 adaptation sentences, the performance shows a 12.12% word error rate reduction. As the number of adaptation sentences increases, the performance saturates quickly. To improve the behavior of TMLI for large amounts of adaptation data, the TMLI+MAP method which combines TMLI with MAP technique is proposed. Experimental results show TMLI+MAP achieved better recognition accuracy than MAP and MLLR+MAP for both small and large amounts of adaptation data. [ABSTRACT FROM AUTHOR]

This paper describes a technique for synthesizing speech with an arbitrary speaker's voice using speaker-independent speech units, which we call “average voice”models. The proposed method is based on an HMM-based text-to-speech synthesis system. In the HMM-based speech synthesis system, the spectrum and the pitch parameter are modeled simultaneously using an HMM based on the multi-space probability distribution (MSD). By appropriately transforming the HMM parameters, the voice characteristics and the prosodic features of synthesized speech can be transformed. In this paper, we derive an extension of the MLLR algorithm to apply it to MSD-HMMs. It is shown that by speaker adaptation for the pitch and applying the spectrum model, not only the voice characteristics, but also the prosodic features, can be adapted by using a small number of sentences uttered by a target speaker. Through the subjective evaluation, it is shown that by applying simultaneous adaptation of pitch and spectrum, synthetic speech generated from adapted models from the average voice model using a small number of sentences is very close to that from speaker-dependent models. © 2004 Wiley Periodicals, Inc. Syst Comp Jpn, 35(11): 59–68, 2004; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10332 [ABSTRACT FROM AUTHOR]

In speech recognition research, transformation-based adaptation algorithms provide an effective way of adapting acoustic models to improve the recognition accuracy. However, when only limited amounts of adaptation data are available, the transformation is often poorly estimated, which may cause performance degradation. This paper presents the Markov Random Field Linear Regression (MRFLR) algorithm, which constrains the transformation-based adaptation by the correlations among acoustic parameters. The Markov Random Field theory is used to model the correlations. The correlations are estimated from the training corpus and hypothesized as prior knowledge of acoustic models. By explicitly incorporating them into adaptation, robust and fast adaptation can be achieved. The hypothesis is tested by comparing MRFLR with MLLR (Maximum Likelihood Linear Regression), a widely used trans formation-based adaptation algorithm. Experimental results show that MRFLR outperforms MLLR when adaptation data are sparse, and converges to the MLLR performance when more adaptation data are available. [ABSTRACT FROM AUTHOR]

Modern speech recognition systems are based on the hidden Markov model (HMM) and employ cepstral features to represent input speech. In speaker normalization, the cepstral features of speech from a given speaker are transformed to match the speaker independent HMM. In speaker adaptation, the means of the HMM are transformed to match the input speech. Vocal tract length normalization (VTLN) is a popular normalization scheme wherein the frequency axis of the short-time spectrum is rescaled prior to the extraction of cepstral features. In this work, we develop novel speaker adaptation schemes by exploiting the fact that frequency domain transformations similar to that inherent in VTLN can be accomplished entirely in the cepstral domain through the use of conformal maps. We describe two classes of such maps: rational all-pass transforms (RAPTs) which are well-known in the signal processing literature, and sine-log all-pass transforms (SLAPTs) which are novel in this work. For both classes of maps, we develop the relations necessary to perform maximum likelihood estimation of the relevant transform parameters using enrollment data from a new speaker. We also propose the means by which an HMM may be trained specifically for use with this type of adaptation. Finally, in a set of recognition experiments conducted on conversational speech material from the Switchboard Corpus as well as the English Spontaneous Scheduling Task, we demonstrate the capacity of APT-based speaker adaptation to achieve word error rate reductions superior to those obtained with other popular adaptation techniques, and moreover, reductions that are additive with those provided by VTLN. [Copyright &y& Elsevier]

This paper presents a novel approach to rapid speaker adaptation based on the speaker space projection paradigm in which the adapted model is constrained to lie on a specific subspace spanned by a small number of basis vectors. In order to select the basis vectors that form the speaker space, we apply probabilistic principal component analysis (PPCA) technique to a set of training speaker models represented by a number of hidden Markov models (HMMs). The PPCA incorporates a probability model to the conventional principal component analysis (PCA) method, and finds the speaker space model by means of the expectation maximization (EM) algorithm which is computationally efficient. The PPCA model provides the information of correlation among different speech units as well as the prior probability density function (pdf) associated with each HMM parameter, which can be directly applied to the maximum a posteriori (MAP) adaptation framework. Through a series of supervised adaptation experiments on the tasks of connected digit and large vocabulary recognition, we show that the proposed approach not only achieves a good performance for a small amount of adaptation data but also guarantees a consistent estimate as the data size grows. [Copyright &y& Elsevier]

Handling non-native speech in automatic speech recognition (ASR) systems is an area of increasing interest. The majority of systems are tailored to native speech only and as a consequence performance for non-native speakers often is not satisfactory. One way to approach the problem is to adapt the acoustic models to the new speaker. Another important means to improve performance for non-native speakers is to consider non-native pronunciations in the dictionary. The difficulty here lies in the generation of the non-native variants, especially if various accents are to be considered. Traditional approaches to model pronunciation variation either require phonetic expertise or extensive speech databases. They are too costly, especially if a flexible modelling of several accents is desired. We propose to exclusively use native speech databases to derive non-native pronunciation variants. We use an English phoneme recogniser to generate English pronunciations for German words and use these to train decision trees that are able to predict the respective English-accented variant from the German canonical transcription. Furthermore we combine this approach with online, incremental weighted MLLR speaker adaptation. Using the enhanced dictionary and the speaker adaptation alone improved the word error rate of the baseline system by 5.2% and 16.8%, respectively. When both methods were combined, we achieved an improvement of 18.2%. [Copyright &y& Elsevier]

This paper presents a technical speaker adaptation method called FCMAP, which incorporated an appropriate fuzzy logic controller into maximum a posteriori (MAP) to adapt the hidden Markov model parameters. Experimental results show that the speaker adaptation system using the FCMAP adaptation method outperforms that of the conventional MAP, especially when only few utterances from a new speaker are available for adaptation. [Copyright &y& Elsevier]

A robust speech recognizer is developed to tackle the inevitable mismatch between training and testing environments. Because the realistic environments are uncertain and nonstationary, it is necessary to characterize the uncertainty of speech hidden Markov models (HMMs) for recognition and trace the uncertainty incrementally to catch the newest environmental statistics. In this paper, we develop a new Bayesian predictive classification (BPC) for robust decision and online environmental learning. The BPC decision is adequately established by modeling the uncertainties of both the HMM mean vector and precision matrix using a conjugate prior density. The frame-based predictive distributions using multivariate t distributions and approximate Gaussian distributions are herein exploited. After the recognition, the prior density is pooled with the likelihood of the current test sentence to generate the reproducible prior density. The hyperparameters of the prior density are accordingly adjusted to meet the newest environments and apply for the recognition of upcoming data. As a result, an efficient online unsupervised learning strategy is developed for HMM-based speech recognition without needing adaptation data. In the experiments, the proposed approach is significantly better than conventional plug-in maximum a posteriori (MAP) decision on the recognition of connected Chinese digits in hands-free car environments. This approach is economical in computation. [Copyright &y& Elsevier]

This paper describes a new unsupervised, on-line and incremental speaker adaptation technique that improves the performance of speech recognition systems when there are frequent changes in speaker identity and each speaker utters a series of several sentences. The speaker change is detected using speaker-independent (SI) and speaker-adaptive (SA) Gaussian mixture models (GMMs), and both phone hidden Markov model (HMM) and GMM are adapted by maximum likelihood linear regression (MLLR) transformation. Using this method, the word error rate of a broadcast news transcription task was reduced by 10.0% relative to the results using the SI models. [Copyright &y& Elsevier]