Search

With the continuing growth of voice-controlled devices, voice metrics have been widely used for user identification. However, voice biometrics is vulnerable to replay attacks and ambient noise. We identify that the fundamental vulnerability in voice biometrics is rooted in its indirect sensing modality (e.g., microphone). In this paper, we present VocalPrint, a resilient mmWave interrogation system that directly captures and analyzes the vocal vibrations for user authentication. Specifically, VocalPrint exploits the unique disturbance of the skin-reflect radio frequency (RF) signals around the near-throat region of the user, caused by the vocal vibrations. The complex ambient noise is isolated from the RF signal using a novel resilience-aware clutter suppression approach for preserving fine-grained vocal biometric properties. Afterward, we extract the vocal tract and vocal source features and input them into an ensemble classifier for authentication. VocalPrint is practical as it allows the effortless transition to a smartphone while having sufficient usability due to its non-contact nature. Our experimental results from 41 participants with different interrogation distances, orientations, and body motions show that VocalPrint achieves over 96% authentication accuracy even under unfavorable conditions. We demonstrate the resilience of our system against complex noise interference and spoof attacks of various threat levels.

The aim of this paper is perform the synthesis of sung Spanish vowels considering the soprano vocal category of lyrical singers, including variation of sustained pitches with vibrato and tremolo effects, considering sounds from Spanish language. The Fant source-filter theory is used to model the production of the sung vowels: the source is based on the Rosenberg glottal pulse model and the filter (the vocal tract) is composed by an all-pole filter model with formant frequencies and bandwidths from the vowels of the Spanish language, obtained through experimental voice signals from two soprano singers. All the sounds synthesized are available to be accessed and they were submitted to a group of listeners which gave a very good evaluation with respect to the intelligibility and naturalness of the sounds.

Speech analysis plays a pivotal role in the exploitation of forensic recordings in resolving a wide range of questions. Although this topic may include a large set of methodologies based on varied digital features (MFCC, Centroid, Harmonicity, VOT, etc., cf. [1]), this work approaches the theme from a phonetic perspective, taking into consideration the vowel formants, and focusing on their variability and difficulty of measurement. Formants correspond to the resonant frequencies of the vocal tract and are, therefore, sensible to specific speaker-related variations such as age and sex. In this respect, formants' variation contributes to characterizing the subjective timbre of the person. For this reason, formants' values are largely used in forensics [2], with all the practical problems that come with them, and in particular when dealing with speaker recognition or discrimination [3], [4]. Indeed, formants' values correspond to specific frequencies of the sound signal and are usually reported in Hertz. They are, however, affected by numerous internal and external variables, so that although on average they are characteristic of the individual speaker, they always vary within frequency bands that cannot be defined in absolute terms [5]. How, then, is it possible to provide reliable answers to forensic questions? In essence, it is up to the specialist to determine if there are the conditions to carry out an analysis, and to understand, for example, whether the differences between formants' values could be ascribed to two different speakers or the difference is too subtle to justify this claim. What should be measurable today, and must be defined at the level of jurisprudence, is therefore the professionalism of the expert. However, this is still heterogeneous in different countries, although several codes of practice such as the International Association for Forensic Phonetics or engineering society ones have been validated.

The growing number of voice-enabled devices and applications consider automatic speaker verification (ASV) a fundamental component. However, maximum outreach for ASV in critical domains e.g., financial services and health care, is not possible unless we overcome security breaches caused by voice cloning algorithms and replayed audios. Therefore, to overcome these vulnerabilities, a secure ASV (SASV) system based on the novel sign modified acoustic local ternary pattern (sm-ALTP) features and asymmetric bagging-based classifier-ensemble with enhanced attack vector is presented. The proposed audio representation approach clusters the high and low frequency components in audio frames by normally distributing frequency components against a convex function. Then, the neighborhood statistics are applied to capture the user specific vocal tract information. The proposed SASV system simultaneously verifies the bonafide speakers and detects the voice cloning attack, cloning algorithm used to synthesize cloned audio (in the defined settings), and voice-replay attacks over the ASVspoof 2019 dataset. In addition, the proposed method detects the voice replay and cloned voice replay attacks over the VSDC dataset. Both the voice cloning algorithm detection and cloned-replay attack detection are novel concepts introduced in this paper. The voice cloning algorithm detection module determines the voice cloning algorithm used to generate the fake audios. Whereas, the cloned voice replay attack detection is performed to determine the SASV behavior when audio samples are simultaneously contemplated with cloning and replay artifacts.

An algorithm for a potentially non-obtrusive speech production system was developed and characterized. The algorithm is primarily based on the articulation of the human tongue referred as tongue articulatory system (TAS) and was cascaded with a previously developed laryngeal model. We developed and optimized statistical formulae for formants of vowels and consonants and studied the model for different ages and genders. The difference between the formant frequencies obtained using both the established vocal tract system and proposed cascaded system was found to be

The technique of transforming voices in order to hide the real identity of a speaker is called voice disguise, among which automatic voice disguise (AVD) by modifying the spectral and temporal characteristics of voices with miscellaneous algorithms are easily conducted with softwares accessible to the public. AVD has posed great threat to both human listening and automatic speaker verification (ASV). In this paper, we have found that ASV is not only a victim of AVD but could be a tool to beat some simple types of AVD. Firstly, three types of AVD, pitch scaling, vocal tract length normalization (VTLN) and voice conversion (VC), are introduced as representative methods. State-of-the-art ASV methods are subsequently utilized to objectively evaluate the impact of AVD on ASV by equal error rates (EER). Moreover, an approach to restore disguised voice to its original version is proposed by minimizing a function of ASV scores w.r.t. restoration parameters. Experiments are then conducted on disguised voices from Voxceleb, a dataset recorded in real-world noisy scenario. The results have shown that, for the voice disguise by pitch scaling, the proposed approach obtains an EER around 7% comparing to the 30% EER of a recently proposed baseline using the ratio of fundamental frequencies. The proposed approach generalizes well to restore the disguise with nonlinear frequency warping in VTLN by reducing its EER from 34.3% to 18.5%. However, it is difficult to restore the source speakers in VC by our approach, where more complex forms of restoration functions or other paralinguistic cues might be necessary to restore the nonlinear transform in VC. Finally, contrastive visualization on ASV features with and without restoration illustrate the role of the proposed approach in an intuitive way., accepted for publication

Affective flattening is a typical negative symptom in schizophrenia that causes a diminution of normal behaviors and functions in schizophrenic patients. In this work, an automatic algorithm of schizophrenia detection is proposed. This algorithm includes three newly proposed features. These features establish a representation for the production and perception of schizophrenic speech, called the K-Sf (kurtosis skewness fraction), SPI (sound perception indicator), and EBMC (enhanced bilateral matching coefficient). The K-Sf is proposed to reflect the overall distribution of speech segments. The SPI evaluates the relation between the speech composition and the sound perception. The EBMC feature is proposed based on the theory of air pressure oscillations which could reflect the details of air modulation in the vocal tract. Experiments evaluating the discriminative capabilities of the three features are conducted using a speech dataset that is collected from 56 participants (28 schizophrenic patients and 28 healthy controls) and an ensemble classifier. Comparative experiments with SVM classifiers are also conducted. The discrimination accuracies of patients and control subjects using the K-Sf, SPI, EBMC, and the ensemble classifier are in the range of 76.8-92.9%, 82.1-92.8%, and 80.4-91.1%, respectively. When the three features are combined, the discrimination results range from 89.3% to 94.6%. The experimental results indicate that the three features have stronger robustness and better discrimination capability than those previous features relating to the detection of flat affect.

Differences in vocal tract lengths (VTLs) in individual speakers cause variations in acoustic features of phonemes. In this paper, a simple method to estimate speaker-specific VTLs and to quantitatively evaluate some speaker-normalization effects of the VTLs is proposed. We employed accumulated means of formant trajectories to estimate the VTLs of speakers ranging from children to adults. For the formant estimation, the inverse-filter control (IFC) system was used. In the system, the decision of analysis order, which means number of formants to be estimated, is automated. Moreover, to evaluate the speaker-normalization effect of VTLs, we proposed the data reduction method, which can reasonably find dense areas of ellipses from distributions in the formant space. Using these ellipse areas, we evaluated the three normalization effects of VTLs: normalization by the mean of all VTLs as the standard, by speaker-categorical means of VTLs, and by individual VTLs. The area reduced from the standard area of the original data by 39.5% and 46.6% in the case of the categorical means and individual VTLs, respectively. As a result, our proposed method was used to provide a “normalized vowel map (NVM)” that visualizes universal vowel-distributions as a core image of linguistic information. Finally, we compared the estimated VTLs with those by another method based on magnetic resonance imaging (MRI) data, using the proposed methods.

International audience; Whisper-to-speech conversion is motivated by laryngeal disorders, in which malfunction of the vocal folds leads to loss of voicing. Many patients with laryngeal disorders can still produce functional whispers, since these are characterised by the absence of vocal fold vibration. Whispers therefore constitute a common ground for speech rehabilitation across many kinds of laryngeal disorder. Whisper-to-speech conversion involves recreating natural-sounding speech from recorded whispers, and is a non-invasive and non-surgical rehabilitation that can maintain a natural method of speaking, unlike the existing methods of rehabilitation. This paper proposes a new rule-based method for whisper-to-speech conversion that replaces the noisy whisper sound source with a synthesised speech-like harmonic source, while maintaining the vocal tract component unaltered. In particular, a novel glottal source generator is developed in which whisper information is used to parameterise the excitation through a high-quality glottis model. Evaluation of the system against the standard pulse train excitation method reveals significantly improved performance. Since our method is glottis-based, it is potentially compatible with the many existing vocal tract component adaptation systems.

Clinical diagnosis of voice disorder and evaluation of therapy outcome heavily rely on accurate quantification of voice quality, which is closely tied to the physiology and function of the laryngeal mechanism. Considering the evaluation methodology of the voice, two main categories of auditory-perceptual assessment and acoustic analysis can be identified. This article presents a new approach for acoustic analysis of voice quality, which brings several advantages to the field. The proposed approach is non-parametric in the sense that it does not require the estimation of the fundamental frequency or spectral response of the vocal tract. This reduces the computational complexity of the measurement and reduces the possible errors due to inaccurate estimation of those parameters. Additionally, the method does not make any assumption about the phonetic context and hence has the potential to be applied to connected speech. The proposed method benefits from the multiresolution structure of the wavelet analysis for estimating the noisy component of a voice in the spectro-temporal domain. The informativeness of the estimated noise for voice quality distinction is examined based on different noise-quantification approaches. It is shown that deviation from the model of the human auditory system (HAS) leads to performance improvement. Through several analyses, it is argued that using models of HAS for quantification of the noise leads to significant loss of information relevant to voice quality. Findings from this article suggest that perception-based measures of voice quality are highly restricted in capturing important aspects of acoustic that could assist with voice quality distinctions. This characteristic is inherent to HAS and cannot be alleviated, highlighting a significant limitation of perception-based measures.

Dynamic magnetic resonance imaging (MRI) can be used to scan a wide range of dynamic processes within the body, including the motion of internal organs, tissue-level nuclear magnetic resonance (NMR) relaxation, and dynamic contrast enhancement (DCE) of dye agents. The ability of MRI to safely provide unique soft-tissue contrast and comprehensive functional information has made dynamic MRI central to a number of imaging exams for cardiac, interventional, vocal tract, cancer, and gastrointestinal applications, among others. Unfortunately, MRI is a notoriously slow imaging modality due to fundamental physical and physiological limitations. These limitations result in tradeoffs between spatial and temporal resolutions, spatial coverage, and the signal-to-noise ratio and have made dynamic MRI a challenging technical goal.

Recent advances in real-time Magnetic Resonance Imaging (rtMRI) provide an invaluable tool to study speech articulation. In this paper, we present an effective deep learning approach for supervised detection and tracking of vocal tract contours in a sequence of rtMRI frames. We train a single input multiple output deep temporal regression network (DTRN) to detect the vocal tract (VT) contour and the separation boundary between different articulators. The DTRN learns the non-linear mapping from an overlapping fixed-length sequence of rtMRI frames to the corresponding articulatory movements, where a blend of the overlapping contour estimates defines the detected VT contour. The detected contour is refined at a post-processing stage using an appearance model to further improve the accuracy of VT contour detection. The proposed VT contour tracking model is trained and evaluated over the USC-TIMIT dataset. Performance evaluation is carried out using three objective assessment metrics for the separating landmark detection, contour tracking and temporal stability of the contour landmarks in comparison with three baseline approaches from the recent literature. Results indicate significant improvements with the proposed method over the state-of-the-art baselines., NA

Problems in speech and basic emission of vocal sounds are commonly observed in children, however the task of identifying the origin of such disability is complex and prolonged, due to the generic aspect, if by irregular learning or neurobiological disorder, thus causing, in some situations, constant social exclusion. Therefore, it is proposed to use Tracking Algorithms consolidated in specific literature (ACF, Boosting, MIL, Median Flow, KCF, KLT, CRST) for a feasibility analysis in complex computational implementations in the anatomy of human vocal tract from sagittal images. These algorithms are used for understanding the kinematic configuration of multiple dynamic objects and their movements, represented by the upper and lower lips (consequently the “lip distance” at the moment of speech), using Brazilian Portuguese language, for the extraction and recognition of patterns automatically. The patterns can be used in innovative applications in therapy for children with poor articulation in the vocal tract, based on “gaps” and specific difficulties in speech.

The synthesis of diphthongs in three-dimensions (3D) involves the simulation of acoustic waves propagating through a complex 3D vocal tract geometry that deforms over time. Accurate 3D vocal tract geometries can be extracted from Magnetic Resonance Imaging (MRI), but due to long acquisition times, only static sounds can be currently studied with an adequate spatial resolution. In this work, 3D dynamic vocal tract representations are built to generate diphthongs, based on a set of cross-sections extracted from MRI-based vocal tract geometries of static vowel sounds. A diphthong can then be easily generated by interpolating the location, orientation and shape of these cross-sections, thus avoiding the interpolation of full 3D geometries. Two options are explored to extract the cross-sections. The first one is based on an adaptive grid (AG), which extracts the cross-sections perpendicular to the vocal tract midline, whereas the second one resorts to a semi-polar grid (SPG) strategy, which fixes the cross-section orientations. The finite element method (FEM) has been used to solve the mixed wave equation and synthesize diphthongs [${\alpha i}$] and [${\alpha u}$] in the dynamic 3D vocal tracts. The outputs from a 1D acoustic model based on the Transfer Matrix Method have also been included for comparison. The results show that the SPG and AG provide very close solutions in 3D, whereas significant differences are observed when using them in 1D. The SPG dynamic vocal tract representation is recommended for 3D simulations because it helps to prevent the collision of adjacent cross-sections. QC 20211129 EUNISON

Accent conversion AC aims to transform non-native utterances to sound as if the speaker had a native accent. This can be achieved by mapping source speech spectra from a native speaker into the acoustic space of the target non-native speaker. In prior work, we proposed an AC approach that matches frames between the two speakers based on their acoustic similarity after compensating for differences in vocal tract length. In this paper, we propose a new approach that matches frames between the two speakers based on their phonetic rather than acoustic similarity. Namely, we map frames from the two speakers into a phonetic posteriorgram using speaker-independent acoustic models trained on native speech. We thoroughly evaluate the approach on a speech corpus containing multiple native and non-native speakers. The proposed algorithm outperforms the prior approach, improving ratings of acoustic quality 22% increase in mean opinion score and native accent 69% preference while retaining the voice quality of the non-native speaker. Furthermore, we show that the approach can be used in the reverse conversion direction, i.e., generating speech with a native speaker's voice quality and a non-native accent. Finally, we show that this approach can be applied to non-parallel training data, achieving the same accent conversion performance.

Epochs are the instants of significant excitation to vocal tract system. Existing methods can extract epochs accurately from clean speech signals. However, identification of epoch locations from band-limited telephonic speech is challenging due to the attenuation of fundamental frequency component and degradation caused by channel effect. This letter proposes an epoch extraction method that can accurately extract epochs from clean as well as telephonic speech signals. In the proposed method, the significant impulse-like discontinuities are extracted directly from the speech signal using continuous wavelet transform. The performance of the proposed method is evaluated using three speakers, namely, SLT, BDL, and JMK from CMU Arctic database. The clean speech is simulated using G.191 software tools to obtain telephonic speech. Experimental results show that the epoch identification rate of proposed method is significantly better than the state-of-the-art methods for the telephone quality speech.

The widely used method to estimate glottal source waveform and vocal tract shape is to process speech signal using inverse filter and then to fit residual signal using glottal source model. However, since source-tract interactions, estimation accuracy is reduced. In this paper, we propose a method to estimate glottal source waveform and vocal tract shape simultaneously based on analysis-by-synthesis approach with a source-filter model constructed with an auto-regressive eXogenous (ARX) model combined with the Lilijencrant-Fant (LF) model. Since the optimization of multiple parameters makes simultaneous estimation difficult, there are two steps: the glottal source parameters are initialized using the inverse filter method, then the accurate parameters of the glottal source and the vocal tract shape are estimated simultaneously using an analysis-by-synthesis approach. Experimental results with synthetic and real speech signals showed the higher estimation accuracy of the proposed method than inverse filter.

The Mandarin electrolaryngeal EL speech is suffering from severe fricative confusion due to improper EL source in EL speech production and abnormal physiological structure of vocal tract in the laryngectomized condition. To reduce the fricative confusions, this paper proposes a hybrid noise source by combining the typical natural fricative sources and compensation sources that consider the acoustic defects in the frequency domain caused by the truncated vocal tract and abnormal source location in EL speech production. All parameters of the model are fricative-specific and the parameters of the compensation sources are determined by analyzing the vocal tract transfer functions before and after the laryngectomy. All five Mandarin fricatives are produced by laryngectomized subjects with an experimental EL system loading the hybrid noise source and the wideband noise source. The acoustic and perceptual features of these reconstructed EL fricatives are analyzed and evaluated by comparing with the conventional EL fricatives and normal fricatives. The results indicate that the hybrid noise source successfully improves the acoustic properties of the EL fricatives by forming better spectral shapes, raising the frequencies of average energy concentration, and producing better spectral skewness and kurtosis. Finally, due to these improvements of acoustic properties, the hybrid noise sources achieve much larger intelligibility for EL fricatives than the wideband noise source and the conventional EL source. Thus, the hybrid noise source is an effective, feasible, and promising method of reducing the severe fricative confusions and improving the intelligibility of EL speech.

Besides the recognition of audible speech, there is currently an increasing interest in the recognition of silent speech, which has a range of novel applications. A major obstacle for a wide spread of silent-speech technology is the lack of measurement methods for speech movements that are convenient, non-invasive, portable, and robust at the same time. Therefore, as an alternative to established methods, we examined to what extent different phonemes can be discriminated from the electromagnetic transmission and reflection properties of the vocal tract. To this end, we attached two Vivaldi antennas on the cheek and below the chin of two subjects. While the subjects produced 25 phonemes in multiple phonetic contexts each, we measured the electromagnetic transmission spectra from one antenna to the other, and the reflection spectra for each antenna (radar), in a frequency band from 2–12 GHz. Two classification methods ( k -nearest neighbors and linear discriminant analysis) were trained to predict the phoneme identity from the spectral data. With linear discriminant analysis, cross-validated phoneme recognition rates of 93% and 85% were achieved for the two subjects. Although these results are speaker- and session-dependent, they suggest that electromagnetic transmission and reflection measurements of the vocal tract have great potential for future silent-speech interfaces.

Articulatory speech synthesis based on aero-acoustic simulations of the vocal tract is computationally expensive and, therefore, requires simple yet precise models. Modeling the one-dimensional vocal tract area function directly instead of a higher dimensional vocal tract model is an efficient way to minimize the computational overhead of the simulations. In this paper, we propose a new parametric vocal tract model that is controlled by six points and capable of modeling a large variety of vocal tract shapes. We geometrically and perceptually evaluated the model on a set of 22 reference area functions corresponding to German vowels and consonants. The model was able to geometrically approximate the reference area functions with a minimum root-mean-square error of 0.302 cm$^2$, a maximum error of 1.142 cm$^2$, and a median error of 0.891 cm$^2$. After optimizations, a perceptual evaluation of the synthesis using our model in combination with a state-of-the-art aero-acoustic simulation achieved a vowel recognition rate of 90.7% and a consonant recognition rate of 73.2%.

In social interactions, Aggression is a behavior towards another individual with the motive of causing physical or psychological damage. In humans, anger or disgust due to obstructs in achieving certain goals causes aggression. This article proposes an automatic method for detection of aggression using features extracted from pressure distribution in vocal tract during voiced speech. The variations in air pressure distribution across different sections of vocal tract have been computed from the speech signal by inverse estimation. A Hidden Markov Model has been trained to classify these air pressure variations as Aggression or Calm. The system has been tested on a set of audio clips extracted from interviews of political personalities on television shows. A total of 120 audio clips with an overall duration of around 30 minutes were collected from three different speakers based on human perception of Aggression and Calm. The clips were rated by human raters to discard perceptually ambiguous clips. The speaker dependent system was trained using 40 percent of the data of each speaker and was tested with the remaining 60 percent of the data of the same speaker. The system was able to detect aggression with about 93 percent accuracy.

Articulatory speech synthesis has the potential to offer more natural sounding synthetic speech than established concatenative or parametric synthesis methods. Time-domain acoustic models are particularly suited to the dynamic nature of the speech signal, and recent work has demonstrated the potential of dynamic vocal tract models that accurately reproduce the vocal tract geometry. This paper presents a dynamic 3D digital waveguide mesh (DWM) vocal tract model, capable of movement to produce diphthongs. The technique is compared to existing dynamic 2D and static 3D DWM models, for both monophthongs and diphthongs. The results indicate that the proposed model provides improved formant accuracy over existing DWM vocal tract models. Furthermore, the computational requirements of the proposed method are significantly lower than those of comparable dynamic simulation techniques. This paper represents another step toward a fully functional articulatory vocal tract model which will lead to more natural speech synthesis systems for use across society.

Human behavior plays a major role in improving human-machine communication. The performance must be affected by abnormal behavior as systems are trained using normal utterances. The abnormal behavior is often associated with a change in the human emotional state. Different emotional states cause physiological changes in the human body that affect the vocal tract. Fear, anger, or even happiness we recognize as a deviation from a normal behavior. The whole spectrum of human-machine application is susceptible to behavioral changes. Abnormal behavior is a major factor, especially for security applications such as verification systems. Face, fingerprint, iris, or speaker verification is a group of the most common approaches to biometric authentication today. This paper discusses human normal and abnormal behavior and its impact on the accuracy and effectiveness of automatic speaker verification (ASV). The support vector machines classifier inputs are Mel-frequency cepstral coefficients and their dynamic changes. For this purpose, the Berlin Database of Emotional Speech was used. Research has shown that abnormal behavior has a major impact on the accuracy of verification, where the equal error rate increase to 37 %. This paper also describes a new design and application of the ASV system that is much more immune to the rejection of a target user with abnormal behavior. Web of Science 6 40127 40120

This study examined the effects of three speech conditions (voiced, whispered, and mouthed) on global average measures of articulatory movement during sentence production. Participants were 20 adults (ten males and ten females) with no history of speech, language, or hearing disorders. They produced six target utterances in the three speaking conditions. Movements of the following articulators were recorded with an electromagnetic articulograph: Mid tongue, front of tongue, jaw, lower lip, and upper lip. The kinematic metrics were averages derived from movement strokes defined by minima in the speed record of each articulator. These measures revealed a number of significant changes between the voiced and mouthed conditions, with relatively few differences between voiced and whispered speech. Significant increases in sentence duration, articulatory stroke count, and stroke duration as well as significant decreases in peak stroke speed, stroke distance, and hull areas were found in the mouthed condition. These findings suggest that both laryngeal activation and auditory feedback play an important role in the production of normally articulated vocal tract movements, and that the absence of these may account for the significant changes in articulation between the voiced and mouthed conditions.

Three-dimensional 3-D vocal tract acoustic modeling has the potential to generate high quality and natural voice sounds, but at the price of a large computational cost. Alternatively, 2-D models based on tuned vocal tracts have shown to provide similar results to the 3-D ones but with less computational demands. However, they are currently limited to the synthesis of static vowel sounds. In this paper, the tuned 2-D approach is extended by considering moving vocal tracts to generate dynamic vowel sounds, like diphthongs. Four tuning steps are followed to build a dynamic 2-D vocal tract model that can recover, to a large extent, the formant locations, bandwidths, and energies of a 3-D vocal tract with circular cross section, set in a spherical baffle representing the human head. Acoustic waves propagating through the time evolving vocal tract and radiating to free-field are simulated using the finite element method in the time-domain. As examples, the diphthongs [Ai] and [Au] have been generated using the tuning approach and compared, by means of objective and subjective evaluations, to those resulting from 3-D and conventional 2-D simulations.

In this paper, a noise robust formant frequency estimation scheme is developed based on a spectral model matching algorithm. Considering the vocal tract as an autoregressive system, a spectral model of repeated autocorrelation function RACF of band-limited speech signal is proposed. It is shown that because of the repeated autocorrelation operation on band-limited signal, the proposed model can exhibit prominent formant characteristics. First from given noisy speech observations, an adaptive band selection criterion is developed. Next, on each resulting band-limited noisy speech signal, a repeated autocorrelation operation is carried out, which not only reduces the effect of noise but also strengthens the dominant poles corresponding to the formant frequencies. Finally, spectrum of the RACF is computed and instead of direct spectral peak picking, a model fitting scheme is introduced to find out model parameters which lead to formant estimation. The proposed algorithm has been tested on natural vowels as well as some naturally spoken sentences in the presence of different environmental noises. It is found that the proposed scheme provides better formant estimation accuracy in comparison to some of the existing methods at low levels of signal-to-noise ratio.

This study proposes an approach for glottal inverse filtering of acoustic speech signals using quadratic programming (QPR). The method aims to jointly model the effect of vocal tract and lip radiation with a single filter whose coefficients are optimized using QPR. This optimization is based on the principles of closed phase analysis, where the contribution of the glottal source is attenuated in optimizing the inverse model of the vocal tract. By expressing the optimization problem in terms of the output of a filter, we can apply physically motivated optimization such as flatness of the closed phase. The proposed method was objectively evaluated using a synthetic Liljencrants–Fant model based test set of sustained vowels, as well as a real speech test set where the glottal flow estimates’ closed phases were compared in terms of their flatness. The results based on synthetic speech indicate that the proposed method is robust to changes in $f_0$ , and state-of-the-art quality results were obtained for high-pitched voices, when $f_0$ is in the range 330–450 Hz. The results based on real speech indicate that the proposed method produces glottal flow estimates that have flatter closed phases with less formant ripple in comparison to estimates computed with known reference methods.

In this paper we propose a method for extracting the fundamental frequency fo from degraded speech signals using single frequency filtering SFF approach. The SFF of frequency-shifted speech signal gives high signal-to-noise ratio SNR segments at some frequencies and hence the SFF approach can be exploited for fo extraction using autocorrelation function of those segments. Since the fo is computed from the envelope of a single frequency component of the signal, the vocal tract resonances do not affect the fo extraction. The use of the high SNR frequency component in a given segment helps in overcoming the effects of degradations in the speech signal, without explicitly estimating the characteristics of noise. The proposed method of fo extraction is shown to give better performance for several types of real and simulated degradations, in comparison with some of the methods reported recently in the literature.

Epoch extraction from speech involves the suppression of vocal tract resonances, either by linear prediction based inverse filtering or filtering at very low frequency. Degradations due to channel effect and significant attenuation of low frequency components ( $ 300 Hz) create challenges for the epoch extraction from telephone quality speech. An epoch extraction method is proposed that considers the vertical striations present in the time-frequency representation of voiced speech as the representative candidates for the epochs. Time-frequency representation with better localized vertical striations is estimated using single pole filter based filter bank. The time marginal of time-frequency representation is computed to locate the epochs. The proposed algorithm is evaluated on the database of five speakers, which provide simultaneous speech and electroglottographic recordings. Telephone quality speech is simulated using G.191 software tools. The identification rate of the state-of-the-art methods degrades substantially for the telephone quality speech whereas that of the proposed method remains the same, comparable to that of clean speech.

Ultrasound tongue imaging (UTI) provides a convenient way to visualize the vocal tract during speech production. UTI is increasingly being used for speech therapy, making it important to develop automatic methods to assist various time-consuming manual tasks currently performed by speech therapists. A key challenge is to generalize the automatic processing of ultrasound tongue images to previously unseen speakers. In this work, we investigate the classification of phonetic segments (tongue shapes) from raw ultrasound recordings under several training scenarios: speaker-dependent, multi-speaker, speaker-independent, and speaker-adapted. We observe that models underperform when applied to data from speakers not seen at training time. However, when provided with minimal additional speaker information, such as the mean ultrasound frame, the models generalize better to unseen speakers., Comment: 5 pages, 4 figures, published in ICASSP2019 (IEEE International Conference on Acoustics, Speech and Signal Processing, 2019)

A systematic variation of length and cross-sectional area of specific segments of the vocal tract (trachea to lips) was conducted computationally to quantify the effects of source–filter interaction. A one-dimensional Navier–Stokes (transmission line) solution was used to compute peak glottal airflow, maximum flow declination rate, and formant ripple on glottal flow for Level 1 (aero-acoustic) interactions. For Level 2 (tissue movement) interaction, peak glottal area, phonation threshold pressure, and deviation in $f_{{{o}}}$ were quantified. Results show that the ventricle, the false-fold glottis, the conus elasticus entry, and the laryngeal vestibule are the regions to which acoustic variables are most sensitive. Generally, any narrow section of the vocal tract increases the degree of interaction, both in terms of its length and its cross-sectional area. The closer the narrow section is to the vocal folds, the greater the effect.

In this letter, we propose a novel fusion feature for detection of whispered speech in noisy environment using a group-delay-based instantaneous spectrum analysis. The fusion feature involves two individual components, namely, subband modulation spectrum (SMS)-based features and subband correntropy (SCE) features, both extracted from the instantaneous spectrum. The instantaneous spectrum estimation involves zero-time windowing for improved temporal resolution and group-delay computation for improved spectral resolution, as compared to the traditional discrete-Fourier-transform-based spectrum estimation. The SMS features capture the spectral representation of the subband energy time trajectories, while the SCE features model the fluctuations in the subband energy time trajectories. The SMS captures both the short-term as well as long-term spectral characteristics of whispered speech and is known to provide good separation between speech and noise components. The correntropy features help capture the dynamics of the vocal tract system to discriminate noisy whisper from noise. Whisper speech detection experiments using support vector machine models and the proposed features indicate promising performance under low signal-to-noise conditions.

This paper presents a new technique for glottal inverse filtering using a distributed model of the vocal tract. A discrete state space model has been constructed for the speech production system by combining the concatenated tube model of the vocal tract and Liljencrants–Fant (LF) model of the glottal flow derivative waveform. An adaptive system identification technique, based on extended Kalman filtering, has been used for estimation of the states and model parameters from continuous speech. The glottal signal, represented by the LF model, is piecewise differentiable in one glottal cycle. Hence, the hybrid system has been characterized by separate models during two different modes. Multiple model estimation has been performed by switching between the two models at the mode jumps. The open phase of the glottal cycle has been considered as Mode 1; whereas, the return phase and closed phase combined has been taken as Mode 2. The starting point of Mode 1, also known as glottal opening instant, was estimated by observing formant modulation, which remains negligible during closed phase, and starts to increase at the onset of opening. The starting point of Mode 2, also known as the glottal closing instant, was computed by peak-picking from linear prediction (LP) residual signal. The proposed method estimates the glottal waveform as well as changes in flow occurring at different sections of the vocal tract during speech production. This technique has been found to be accurate and robust to variations in pitch as compared to other LP-based methods in the literature. The method also estimates the air pressure distribution at different sections of the vocal tract.

In this paper, the speech signal recorded from the desired speaker close to microphone in natural environment is regarded as foreground speech and rest of the interfering sources as background noise . The proposed paper exploits speech production features like glottal closure instants in time domain and vocal tract information in spectral domain to segment the desired speaker's speech and to further enhance it. The foreground speech is perceptually enhanced using the auditory perception feature in mel-frequency domain using mel-cepstral coefficients and its inversion using mel log spectrum approximation filter. The focus is on enhancing the production and perceptual features of foreground speech rather than relying on modeling the interfering sources. The speech data are collected in different natural environments from different speakers in order to evaluate the proposed method. The enhanced speech signals derived at three different stages of the proposed method are evaluated with state-of-the-art methods in terms of subjective and objective measures. The proposed method provides improved performance compared to the considered state-of-the-art methods. In terms of the proposed objective measure foreground to background Ratio , the enhancement approach presented in this paper gives an average improvement of 12 dB as opposed to existing spectral subtraction-based method which provides 3 dB. Moreover, subjective evaluation using 24 different subjects corroborates the objective test results.

This letter presents a method to perform the classification of speech under stress based on physical characteristics. A physical model is proposed to model airflow patterns in the physiological system in order to represent the process of speech production under psychological stress, and physical parameters characterizing airflow variations in the vocal folds, the vocal tract, and laryngeal ventricle are explored. Experimental evaluations show that the physical parameters are effective for the classification of stressed speech.

X-ray imaging is an effective technique toobtain the continuous motions of the vocal tract duringspeech, and Active appearance model (AAM) is a usefultool to analyze the X-ray images. However, for the task oftongue tracking in X-ray images, the accuracy of AAM fitting is insufficient. AAM aims to minimize the residual error between the model appearance and the input image. Itoften fails to accurately converge to the true landmarks. Toimprove the tracking accuracy, we propose a fitting methodby combining Constrained local model (CLM) into AAM.In our method, we first combine the objective functionsof AAM and CLM into a single objective function. Then,we project out the texture variation and derive a gradient based method to optimize the objective function. Ourmethod effectively incorporates not only the shape priorand global texture, but also local texture around each landmark. Experiments demonstrate that the proposed methodsignificantly reduces the fitting error. We also show that realistic 3D tongue animation can be created by using tonguetracking results of the X-ray images.

In this paper we present a data driven vocal tract area function (VTAF) estimation using Deep Neural Networks (DNN). We approach the VTAF estimation problem based on sequence to sequence learning neural networks, where regression over a sliding window is used to learn arbitrary non-linear one-to-many mapping from the input feature sequence to the target articulatory sequence. We propose two schemes for efficient estimation of the VTAF; (1) a direct estimation of the area function values and (2) an indirect estimation via predicting the vocal tract boundaries. We consider acoustic speech and phone sequence as two possible input modalities for the DNN estimators. Experimental evaluations are performed over a large data comprising acoustic and phonetic features with parallel articulatory information from the USC-TIMIT database. Our results show that the proposed direct and indirect schemes perform the VTAF estimation with mean absolute error (MAE) rates lower than 1.65 mm, where the direct estimation scheme is observed to perform better than the indirect scheme., NA

This work proposes evidence using zero frequency filtering (ZFF) that gives an approximate measure of vocal tract constriction in terms of the low frequency component present in the speech signal. The vocal tract is completely closed in the case of voice bars and nasals and is wide open for low vowels. Intermediate cases are for high vowels, semivowels, laterals, voiced fricatives and other sounds. Vocal tract constriction affects the spectrum by reducing the first formant and attenuating the amplitude of the spectrum. The attenuation is relatively high in higher frequencies resulting in an increase in the low frequency component. The proposed method exploits the sinusoid like nature of ZFF signal (ZFFS) to obtain the evidence. Epoch synchronous analysis is performed and the ZFFS between successive epochs is compared with the corresponding speech segment using a cosine kernel. The low frequency dominant voiced regions match closely with ZFFS as compared to other regions and hence give higher value. This evidence when used as a feature gives relatively higher performance for the constricted phones in an HMM-based phoneme recognizer.

This paper proposes an analysis method to separate the glottal source and vocal tract components of speech that is called Glottal Spectral Separation (GSS). This method can produce high-quality synthetic speech using an acoustic glottal source model. In the source-filter models commonly used in speech technology applications it is assumed the source is a spectrally flat excitation signal and the vocal tract filter can be represented by the spectral envelope of speech. Although this model can produce high-quality speech, it has limitations for voice transformation because it does not allow control over glottal parameters which are correlated with voice quality. The main problem with using a speech model that better represents the glottal source and the vocal tract filter is that current analysis methods for separating these components are not robust enough to produce the same speech quality as using a model based on the spectral envelope of speech. The proposed GSS method is an attempt to overcome this problem, and consists of the following three steps. Initially, the glottal source signal is estimated from the speech signal. Then, the speech spectrum is divided by the spectral envelope of the glottal source signal in order to remove the glottal source effects from the speech signal. Finally, the vocal tract transfer function is obtained by computing the spectral envelope of the resulting signal. In this work, the glottal source signal is represented using the Liljencrants-Fant model (LF-model). The experiments we present here show that the analysis-synthesis technique based on GSS can produce speech comparable to that of a high-quality vocoder that is based on the spectral envelope representation. However, it also permit control over voice qualities, namely to transform a modal voice into breathy and tense, by modifying the glottal parameters.

The digital waveguide mesh has been shown to be capable of reproducing the acoustic impulse response of cylindrical vocal tract analogs. This study extends the same methodology to three-dimensional simulation of the acoustic response of graphical models of the vocal tract obtained from magnetic resonance imaging for a group of trained subjects. By such simulation of the vocal tract transfer function and convolution with an appropriate source waveform, basic phonemes are resynthesized and compared with benchmark audio recordings. The technologies and techniques used for simulation are described, alongside the protocol for image capture and the process for collection of benchmark audio. The results of simulation and acoustic recording are then evaluated and compared. The value of three-dimensional simulation in comparison to existing lower-dimensionality equivalents is assessed. It is found that while three-dimensional simulation provides a strong representation of the low frequency vocal tract transfer function, at higher frequencies its performance becomes geometry-dependent. MRI imaging and benchmark audio is provided for future studies and to permit comparison with comparable means of acoustic simulation.

In this work, we present a joint source-filter optimization approach for separating voiced speech into vocal tract (VT) and voice source components. The presented method is pitch-synchronous and thereby exhibits a high robustness against vocal jitter, shimmer and other glottal variations while covering various voice qualities. The voice source is modeled using the Liljencrants-Fant (LF) model, which is integrated into a time-varying auto-regressive speech production model with exogenous input (ARX). The non-convex optimization problem of finding the optimal model parameters is addressed by a heuristic, evolutionary optimization method called differential evolution. The optimization method is first validated in a series of experiments with synthetic speech. Estimated glottal source and VT parameters are the criteria used for comparison with the iterative adaptive inverse filter (IAIF) method and the linear prediction (LP) method under varying conditions such as jitter, fundamental frequency (f0) as well as environmental and glottal noise. The results show that the proposed method largely reduces the bias and standard deviation of estimated VT coefficients and glottal source parameters. Furthermore, the performance of the source-filter separation is evaluated in experiments using speech generated with a physical model of speech production. The proposed method reliably estimates glottal flow waveforms and lower formant frequencies. Results obtained for higher formant frequencies indicate that research on more accurate voice source models and their interaction with the VT is necessary to improve the source-filter separation. The proposed optimization approach promises to be a useful tool for future research addressing this topic.

Pattern recognition tasks often face the situation that training data are not fully representative of test data. This problem is well-recognized in speech recognition, where methods like cepstral mean normalization (CMN), vocal tract length normalization (VTLN) and maximum likelihood linear regression (MLLR) are used to compensate for channel and speaker differences. Speech emotion recognition (SER) is an important emerging field in human-computer interaction and faces the same data shift problems, a fact which has been generally overlooked in this domain. In this paper, we show that compensating for channel and speaker differences can give significant improvements in SER by modelling these differences as a covariate shift. We employ three algorithms from the domain of transfer learning that apply importance weights (IWs) within a support vector machine classifier to reduce the effects of covariate shift. We test these methods on the FAU Aibo Emotion Corpus, which was used in the Interspeech 2009 Emotion Challenge. It consists of two separate parts recorded independently at different schools; hence the two parts exhibit covariate shift. Results show that the IW methods outperform combined CMN and VTLN and significantly improve on the baseline performance of the Challenge. The best of the three methods also improves significantly on the winning contribution to the Challenge.

Electrolarynx (EL) is a medical speech-recovery device designed for patients who have lost their original voice box due to laryngeal cancer. As a substitute for human larynx, the current commercial EL voice source cannot reconstruct natural EL speech under laryngectomy conditions. To eliminate the abnormal acoustic properties of EL speech, a supraglottal voice source with compensation of vocal tract characteristics was proposed and provided through an experimental EL(SGVS-EL) system. The acoustic analyses of simulated EL speech and reconstructed EL speech produced with different voice sources were performed in the normal subject and laryngectomee. The results indicated that the supraglottal voice source was successful in improving the acoustic properties of EL speech by enhancing low- frequency energy, correcting the shifted formants to normal range, and eliminating the visible spectral zeros. Both normal subject and laryngectomee also produced more natural vowels using SGVS-EL than commercial EL, even if the vocal tract parameter was substituted and the supraglottal voice source was biased to a certain degree. Therefore, supraglottal voice source is a feasible and effective approach to improving the acoustic quality of EL speech.

It has long been posited that the space of speech sounds is inherently low dimensional, the result of a relatively small number of degrees of freedom involved in the human vocal apparatus. We attempt to formalize this notion by analyzing a simple physical model of the vocal tract and demonstrating that it produces transfer functions whose spectra are restricted to low dimensional manifolds embedded in an infinite dimensional space of square integrable functions. While source convolution and channel distortion precludes analytic recovery of the articulatory configuration from the observed signal, we present a data-driven unsupervised learning algorithm called Intrinsic Spectral Analysis designed to recover from a stream of unannotated and unsegmented audio a set of nonlinear basis functions for the speech manifold. Projecting a traditional spectrogram onto this nonlinear basis defines a novel acoustic representation that is demonstrated to have phonological significance, improved phonetic separability, inherent speaker independence, and complementarity with standard acoustic front-ends.

3D time-domain acoustic modeling techniques have the potential to produce more accurate simulation of the vocal tract than previously implemented 1D or 2D solutions, although the variability of human voice renders it a problematic benchmark for validation of its resynthesis. This study uses acoustic measurement of acrylic cylindrical vocal tract models derived from X-Ray data to assess the validity of comparable 3D digital waveguide mesh simulations. It is found that for more simple structures the 3D digital waveguide mesh is able to reproduce the acoustic behavior up to 10 kHz with only slight errors in resonant frequencies. As the simulated structures become more geometrically complex, this shifting becomes more severe.

This paper proposes a novel feature-space VTLN (vocal tract length normalization) method that models frequency warping as a linear interpolation of contiguous Mel filter-bank energies. The presented technique aims to reduce the distortion in the Mel filter-bank energy estimation due to the harmonic composition of voiced speech intervals and DFT (discrete Fourier transform) sampling when the central frequency of band-pass filters is shifted. This paper also proposes an analytical maximum likelihood (ML) method to estimate the optimal warping factor in the cepstral space. The presented interpolated filter-bank energy-based VTLN leads to relative reductions in WER (word error rate) as high as 11.2% and 7.6% when compared with the baseline system and standard VTLN, respectively, in a medium-vocabulary continuous speech recognition task. Also, the proposed VTLN scheme can provide significant reductions in WER when compared with state-of-the-art VTLN methods based on linear transforms in the cepstral feature-space. The warping factor estimated with the proposed VTLN approach shows more dependence on the speaker and more independence of the acoustic-phonetic content than the warping factor resulting from standard and state-of-the-art VTLN methods. Finally, the analytical ML-based optimization scheme presented here achieves almost the same reductions in WER as the ML grid search version of the technique with a computational load 20 times lower.

We propose a concatenative synthesis approach to the problem of foreign accent conversion. The approach consists of replacing the most accented portions of nonnative speech with alternative segments from a corpus of the speaker's own speech based on their similarity to those from a reference native speaker. We propose and compare two approaches for selecting units, one based on acoustic similarity [e.g., mel frequency cepstral coefficients (MFCCs)] and a second one based on articulatory similarity, as measured through electromagnetic articulography (EMA). Our hypothesis is that articulatory features provide a better metric for linguistic similarity across speakers than acoustic features. To test this hypothesis, we recorded an articulatory-acoustic corpus from a native and a nonnative speaker, and evaluated the two speech representations (acoustic versus articulatory) through a series of perceptual experiments. Formal listening tests indicate that the approach can achieve a 20% reduction in perceived accent, but also reveal a strong coupling between accent and speaker identity. To address this issue, we disguised original and resynthesized utterances by altering their average pitch and normalizing vocal tract length. An additional listening experiment supports the hypothesis that articulatory features are less speaker dependent than acoustic features.

This paper considers 2-D Fourier analysis of local time-frequency regions of wideband spectrograms, a representation we refer to as the wideband Grating Compression Transform (WGCT). We develop frequency-dependent, speech-production-based models of speech signals for the WGCT, building on previous work in modeling narrowband-based GCT representations (NGCT). Comparisons show important distinctions, including dual behavior, between the wideband and narrowband models, and distinct ways in which vocal tract/formant content is distributed redundantly throughout the NGCT and WGCT spaces. Our results motivate a novel taxonomy of speech-signal behavior as an interpretative framework (i.e., in relation to speech-production characteristics) for 2-D processing of speech using the GCT, as well as for other 2-D approaches and time-frequency distributions such as the auditory spectrogram. We demonstrate and evaluate the ability of the model to represent real speech content through demodulation techniques for analysis/synthesis of wideband spectrograms. Finally, we develop a co-channel speaker separation method, using prior and estimated pitch information, based on the WGCT, as well as through fusion with the NGCT. These GCT-based separation systems are compared against and further fused with a reference sinusoidal separation system.

Traditional approaches in speech-signal processing analyze short-time frames of the signal (e.g., the short-time Fourier transform). Findings from auditory neurophysiology coupled with image processing principles, however, have motivated an alternative 2-D processing framework in which 2-D analysis is performed on the time-frequency distribution itself. This paper develops a 2-D model of speech in local time-frequency regions of narrowband spectrograms using sinusoidal-series-based modulation. Our model is shown to distribute vocal tract and onset/offset content based on source information (e.g., noise and voicing) in a transformed 2-D space, thereby explicitly representing different classes of energy modulations commonly observed in spectrograms. We demonstrate the model's ability to represent speech sounds by developing and evaluating algorithms for analysis/synthesis of spectrograms. As an example application, we demonstrate the utility of the model for co-channel speaker separation using prior pitch information of two overlapping speakers. Finally, our separation scheme based on 2-D modeling is compared against a reference (frame-based) sinusoidal separation system using both prior and estimated pitch.