Search

In the era of advanced text-to-speech (TTS) systems capable of generating high-fidelity, human-like speech by referring a reference speech, voice cloning (VC), or zero-shot TTS (ZS-TTS), stands out as an important subtask. A primary challenge in VC is maintaining speech quality and speaker similarity with limited reference data for a specific speaker. However, existing VC systems often rely on naive combinations of embedded speaker vectors for speaker control, which compromises the capture of speaking style, voice print, and semantic accuracy. To overcome this, we introduce the Two-branch Speaker Control Module (TSCM), a novel and highly adaptable voice cloning module designed to precisely processing speaker or style control for a target speaker. Our method uses an advanced fusion of local-level features from a Gated Convolutional Network (GCN) and utterance-level features from a gated recurrent unit (GRU) to enhance speaker control. We demonstrate the effectiveness of TSCM by integrating it into advanced TTS systems like FastSpeech 2 and VITS architectures, significantly optimizing their performance. Experimental results show that TSCM enables accurate voice cloning for a target speaker with minimal data through both zero-shot or few-shot fine-tuning of pretrained TTS models. Furthermore, our TSCM-based VITS (TSCM-VITS) showcases superior performance in zero-shot scenarios compared to existing state-of-the-art VC systems, even with basic dataset configurations. Our method's superiority is validated through comprehensive subjective and objective evaluations. A demonstration of our system is available at https://great-research.github.io/tsct-tts-demo/, providing practical insights into its application and effectiveness. [ABSTRACT FROM AUTHOR]

Intelligent speech recognition is increasingly used in embedded systems, which is also seriously threatened by malicious speech spoofing attacks. Different from the conventional methods, this article proposes a segment-based anti-spoofing detection (SASD) method for the quick detection of spoofed speeches against embedded speech recognition, which focuses on the anti-spoofing features rather than the contexts of speeches and the voiceprints of speakers. The speeches are divided into word segments and silent segments. Based on constant $Q$ cepstral coefficients (CQCCs), a word CQCC (WCQCC) extraction is first designed for the word segments of speeches. Then, based on short-term zero crossing rate (ZCR), an average ZCR (AZCR) extraction is devised for the silent segments. Combining the WCQCC of word segments and AZCR of silent segments, a biased decision strategy is proposed to quickly determine whether a speech is spoofed. Based on ASVspoof 2021 datasets, extensive experiments are conducted to evaluate the effectiveness of the proposed method. Specifically, our SASD can improve the accuracy of anti-spoofing detection by up to 33.47% and save up to 69.10% of time overhead on embedded devices compared with the existing methods. [ABSTRACT FROM AUTHOR]

Speech data publishing breaches users5 data privacy, thereby causing more privacy disclosure. Existing work sanitizes content, voice, and voiceprint of speech data without considering the consistence among these three features, and thus is susceptible to inference attacks. To address the problem, we design a privacy-preserving protocol for speech data publishing (P3 S2) that takes the corrections among the three factors into consideration. To concrete, we first propose a three-dimensional sanitization that uses feature learning to capture characteristics in each dimension and then sanitize speech data using the learned features. As a result the correlations among the three dimensions of the sanitized speech data are guaranteed. Furthermore, the (sy 8) -differential privacy is used to theoretically prove both the data privacy preservation and the data utility guarantee of P3S2,filling the gap of algorithm design and performance evaluation. Finally, simulations on two real world datasets have demonstrated both the data privacy preservation and the data utility guarantee. [ABSTRACT FROM AUTHOR]

A multi-scale analysis method, called Empirical Mode Decomposition (EMD), has been proposed for analysis of nonlinear and non stationary data. The empirical mode decomposition is a method initiated by Huang et al. as an alternative technique to the traditional Fourier and wavelet techniques for examining signals. It decomposes a signal into several components called intrinsic mode functions. This paper deals with this new tool to detect usable speech in co-channel speech. We applied empirical mode decomposition to decompose the co-channel speech signal into intrinsic oscillatory modes. Detected usable speech segments are organized into speaker streams, which are applied to speaker identification system. The system is evaluated on co-channel speech across various Targets to Interferer Ratios (TIR). Performance evaluation has shown that empirical mode decomposition performs better than linear multi-scale decomposition by discrete wavelet for usable speech detection. [ABSTRACT FROM AUTHOR]

The objective of this work is to investigate the benefit of discrete wavelet transform combined with LPC, for speaker identification system applied for Algerian Berber language, compared to the traditional Mel frequency analysis. We've developed a speaker identification system for Algerian Berber language. The corpus concerns two dataset, the first one concerns eight isolated words and the second is dedicated for continuous speech repeated by Algerian native Berber. We've used MFCC feature, their first and second derivatives and discrete wavelet transform (DWT) followed by linear predictive coding (LPC) to ameliorate the parameterization phase. Mahalanobis distance, ascendant classification and pitch analysis were used for characterizing our speech signals. We evaluate the performance of DWT-LPC feature for clean and additive noisy speech. The multilayer perceptron classifier was used for this purpose, efficiency was improved for DWT combined with LPC feature vectors. [ABSTRACT FROM AUTHOR]

Classification of speech is one of the most vital problems in speech processing. Although there have been many studies on the classification of speech, the results are still limited. Firstly, most of the speech classification approaches requiring input data have the same dimension. Secondly, all traditional methods must be trained before classifying speech signal and must be retrained when having more training data or new class. In this paper, we propose an approach for speech classification using Scale-invariant Feature Transform (SIFT) features on spectrogram images of speech signal combination with Local naïve Bayes nearest neighbor. The proposed approach allows using feature vectors to have different sizes. With this approach, the achieved classification results are satisfactory. They are 73, 96, 95, 97 %, and 97 % on the ISOLET, English Isolated Digits, Vietnamese Places, Vietnamese Digits, JVPD databases, respectively. Especially, in a subset of the TMW database, the accuracy is 100 %. In addition, in our proposed approach, non-retraining is needed for additional training data after the training phase. The experiment shows that the more features are added to the model, the more is the accuracy in performance. [ABSTRACT FROM AUTHOR]

Whispered vowels, produced with no vocal fold vibration, lack the periodic temporal fine structure which in voiced vowels underlies the perceptual attribute of pitch (a salient auditory cue to speaker sex). Voiced vowels possess no temporal fine structure at very short durations (below two glottal cycles). The prediction was that speaker-sex discrimination performance for whispered and voiced vowels would be similar for very short durations but, as stimulus duration increases, voiced vowel performance would improve relative to whispered vowel performance as pitch information becomes available. This pattern of results was shown for women's but not for men's voices. A whispered vowel needs to have a duration three times longer than a voiced vowel before listeners can reliably tell whether it's spoken by a man or woman (~30ms vs. ~10ms). Listeners were half as sensitive to information about speaker-sex when it is carried by whispered compared with voiced vowels. [ABSTRACT FROM AUTHOR]

Robust distant speech recognition (DSR) is necessary in many speech technology applications using multiple microphones but has received only limited treatment in the literature. In this paper, we work on communicating with vehicle voice-controlled system which is one of the applications of DSR. Two approaches for DSR are i) signal-level combination using beamforming followed by automatic speech recognition (ACR), and ii) word hypothesis-level combination using several speech recognition engines followed by confusion network combination or followed by recognizer output voting error reduction (ROVER). In addition to these approaches, it is possible to examine training-level combination by training the recognizer on audio signals from multiple channels (microphones). In this paper, the authors investigate how these methods can be leveraged for in-vehicle ACR using the CU-Move corpus. The authors propose various combinations of these three methods to find an optimum structure for in-vehicle ACR. The authors also investigate the effect of speaker adaptation (SA). The author's experience shows that applying SA on individual channels and merging the results with ROVER reduces the negative effects of SA reported by others in the field, and illustrates the overall improvement obtained with front-end enhancement techniques in DSR. [ABSTRACT FROM PUBLISHER]

The state-of-the-art speaker-recognition systems suffer from significant performance loss on degraded speech conditions and acoustic mismatch between enrolment and test phases. Past international evaluation campaigns, such as the NIST speaker recognition evaluation (SRE), have partly addressed these challenges in some evaluation conditions. This work aims at further assessing and compensating for the effect of a wide variety of speech-degradation processes on speaker-recognition performance. We present an open-source simulator generating degraded telephone, VoIP, and interview-speech recordings using a comprehensive list of narrow-band, wide-band, and audio codecs, together with a database of over 60 h of environmental noise recordings and over 100 impulse responses collected from publicly available data. We provide speaker-verification results obtained with an $i$-vector-based system using either a clean or degraded PLDA back-end on a NIST SRE subset of data corrupted by the proposed simulator. While error rates increase considerably under degraded speech conditions, large relative equal error rate (EER) reductions were observed when using a PLDA model trained with a large number of degraded sessions per speaker. [ABSTRACT FROM PUBLISHER]

We analyze the i-vector extraction from the perspective of the prior distribution exerted on the mean supervector of Gaussian mixture model (GMM). To this end, we start off with the analysis of the subspace prior which leads to the compressed representation in the standard i-vector extraction. We then propose the use of quasi-factorial prior and show how it impacts the total variability space and its application for i-vector extraction. The quasi-factorial prior could be used in a standalone manner, or in combination with a subspace prior. In the latter context, we found that the performance of the standard i-vector can be greatly improved with the use of quasi-factorial prior followed by a subspace prior. This assertion is confirmed through experiments conducted on the NIST 2010 Speaker Recognition Evaluation (SRE10) dataset. [ABSTRACT FROM PUBLISHER]

The development of the i-vector framework for generating low dimensional representations of speech utterances has led to considerable improvements in speaker recognition performance. Although these gains have been achieved in periodic National Institute of Standards and Technology (NIST) evaluations, the problem of domain mismatch, where the system development data and the application data are collected from different sources, remains a challenging one. The impact of domain mismatch was a focus of the Johns Hopkins University (JHU) 2013 speaker recognition workshop, where a domain adaptation challenge (DAC13) corpus was created to address this problem. This paper proposes an approach to domain mismatch compensation for applications where in-domain development data is assumed to be unavailable. The method is based on a generalization of data whitening used in association with i-vector length normalization and utilizes a library of whitening transforms trained at system development time using strictly out-of-domain data. The approach is evaluated on the 2013 domain adaptation challenge task and is shown to compare favorably to in-domain conventional whitening and to nuisance attribute projection (NAP) inter-dataset variability compensation. [ABSTRACT FROM PUBLISHER]

Speaker recognition is the process of automatically recognizing who is speaking on the basis of speaker specific characteristics included in the speech signal. These speaker specific characteristics are called features. Over the past decades, extensive research has been carried out on various possible speech signal features obtained from signal in time or frequency domain. The objective of this paper is to introduce two-dimensional information entropy as a new text-independent speaker recognition feature. Computations are performed in time domain with real numbers exclusively. Experimental results show that the two-dimensional information entropy is a speaker specific characteristic, useful for speaker recognition. [ABSTRACT FROM AUTHOR]

This paper describes an approach to robust signal analysis using iterative parameter re-estimation of a mixture autoregressive (AR) model. The model's focus can be adjusted by initialization of the target and non-target states. The variant examined in this study uses an i.i.d. mixture AR model and is designed to tackle the spectral biasing effect caused by the voice excitation in speech signals with variable fundamental frequency. In our speaker verification experiments, this method performed competitively against standard spectrum analysis techniques in non-mismatch conditions and showed significant improvements in vocal effort mismatch conditions. [ABSTRACT FROM AUTHOR]

Hemispheric specialization for linguistic prosody is a controversial issue. While it is commonly assumed that linguistic prosody and emotional prosody are preferentially processed in the right hemisphere, neuropsychological work directly comparing processes of linguistic prosody and emotional prosody suggests a predominant role of the left hemisphere for linguistic prosody processing. Here, we used two functional magnetic resonance imaging (fMRI) experiments to clarify the role of left and right hemispheres in the neural processing of linguistic prosody. In the first experiment, we sought to confirm previous findings showing that linguistic prosody processing compared to other speech-related processes predominantly involves the right hemisphere. Unlike previous studies, we controlled for stimulus influences by employing a prosody and speech task using the same speech material. The second experiment was designed to investigate whether a left-hemispheric involvement in linguistic prosody processing is specific to contrasts between linguistic prosody and emotional prosody or whether it also occurs when linguistic prosody is contrasted against other non-linguistic processes (i.e., speaker recognition). Prosody and speaker tasks were performed on the same stimulus material. In both experiments, linguistic prosody processing was associated with activity in temporal, frontal, parietal and cerebellar regions. Activation in temporo-frontal regions showed differential lateralization depending on whether the control task required recognition of speech or speaker: recognition of linguistic prosody predominantly involved right temporo-frontal areas when it was contrasted against speech recognition; when contrasted against speaker recognition, recognition of linguistic prosody predominantly involved left temporo-frontal areas. The results show that linguistic prosody processing involves functions of both hemispheres and suggest that recognition of linguistic prosody is based on an inter-hemispheric mechanism which exploits both a right-hemispheric sensitivity to pitch information and a left-hemispheric dominance in speech processing. [ABSTRACT FROM AUTHOR]

Purpose: investigate coincidences and divergences between audio transcription and textualization in order to check for statistical evidence which may be a justification as to the best procedure to be applied. Methods: retrospective study. 30 audios were selected randomly among the 239 available audios, proceeding from telephone intercepts of the same lawsuit. We considered: the number of words and time in minutes for each audio, and the comparative analysis for maintaining the main content highlights. Three Speech Pathologists transcribed and textualized different parts of the file, ensuring independence. A Speech Pathologists, who did not attend the previous step, conducted content analysis. For statistical analysis we used Wilcoxon-Mann Whitney test in R environment, with Tinn R interface. Significance level 5% (0.05). CEP: 274-742. Results: the mean number of words used in of audio file transcription was 27% greater than the number of words used in textualization, p=0.52. The mean time in minutes required to perform the transcription was significantly higher p=0.013. In the comparative analysis as for the maintenance of the main content highlights, we found that on average 93% of highlights were kept, p=0.61%. Conclusion: the similarities among the of transcription and textualization processes were compared with the median number of words and the maintenance as for the median number of content highlights. There was divergence as for the implementation time, significantly lower in textualization. Considering the data obtained in this study, textualization process proved to be the most suitable in audio de-recording. [ABSTRACT FROM AUTHOR]

Most speaker recognition systems rely on short-term acoustic cepstral features for extracting the speaker-relevant information from the signal. But phonetic discriminant features, extracted by a bottle-neck multi-layer perceptron (MLP) on longer stretches of time, can provide a complementary information and have been adopted in speech transcription systems. We compare the speaker verification performance using cepstral features, discriminant features, and a concatenation of both followed by a dimension reduction. We consider two speaker recognition systems, one based on maximum likelihood linear regression (MLLR) super-vectors and the other on a state-of-the-art i-vector system with two session variability compensation schemes. Experiments are reported on a standard configuration of NIST SRE 2008 and 2010 databases. The results show that the phonetically discriminative MLP features retain speaker-specific information which is complementary to the short-term cepstral features. The performance improvement is obtained with both score domain and feature domain fusion and the speaker verification equal error rate (EER) is reduced up to 50% relative, compared to the best i-vector system using only cepstral features. [ABSTRACT FROM AUTHOR]

In this letter, we apply Phone Log-Likelihood Ratio (PLLR) features to the task of speaker recognition. PLLRs, which are computed on the phone posterior probabilities provided by phone decoders, convey acoustic-phonetic information in a sequence of frame-level vectors, and therefore can be easily plugged into traditional acoustic systems, just by replacing the Mel-Frequency Cepstral Coefficients (MFCC) or an alternate representation. To study the performance of the proposed features, MFCC-based and PLLR-based systems are trained under an i-vector-PLDA approach. Results on the NIST 2010 and 2012 Speaker Recognition Evaluation databases show that, despite yielding lower performance than the acoustic system, the system based on PLLR features does provide significant gains when both systems are fused, which reveals a complementarity among features, and provides a suitable and effective way of using higher level phonetic information in speaker recognition systems. [ABSTRACT FROM AUTHOR]

In order to develop a novel voice sensor to detect human voices, the use of features which are more robust to noise is an important issue. Voice sensor is also called voice activity detection (VAD). Due to that the inherent nature of the formant structure only occurred on the speech spectrogram (well-known as voiceprint), Wu et al. were the first to use band-spectral entropy (BSE) to describe the characteristics of voiceprints. However, the performance of VAD based on BSE feature was degraded in colored noise (or voiceprint-like noise) environments. In order to solve this problem, we propose the two-dimensional part-band energy entropy (TD-PBEE) parameter based on two variables: part-band partition number upon frequency index and long-term window size upon time index to further improve the BSE-based VAD algorithm. The two variables can efficiently represent the characteristics of voiceprints on each critical frequency band and use long-term information for noisy speech spectrograms, respectively. The TD-PBEE parameter can be regarded as a PBEE parameter over time. First, the strength of voiceprints can be partly enhanced by using four entropies applied to four part-bands. We can use the four part-band energy entropies for describing the voiceprints in detail. Due to the characteristics of non-stationary for speech and various noises, we will then use long-term information processing to refine the PBEE, so the voice-like noise can be distinguished from noisy speech through the concept of PBEE with long-term information. Our experiments show that the proposed feature extraction with the TD-PBEE parameter is quite insensitive to background noise. The proposed TD-PBEE-based VAD algorithm is evaluated for four types of noises and five signal-to-noise ratio (SNR) levels. We find that the accuracy of the proposed TD-PBEE-based VAD algorithm averaged over all noises and all SNR levels is better than that of other considered VAD algorithms. [ABSTRACT FROM AUTHOR]

Modern communication relies increasingly more on the verbal communication between a machine and a human, aiming to govern certain resources and robots, increase the security of certain means, initiate certain processing protocols, faster financial transactions... This paper illustrates the possibility of using the voice biometrics in modern living, from simple examples, such as starting the motor of a vehicle, through opening security gates, to proving fraud and embezzlement. Special emphasis has been put on the systems of automatic speaker identification and forensic speaker recognition. [ABSTRACT FROM AUTHOR]