Search

Modern speech recognition systems are powerful, but it is not simple to understand the mechanisms by which they model speech, or precisely how they make use of acoustic cues. Moreover, a system for the explicit detection of individual acoustic cues may be useful not only for speech recognition but for understanding the principles that govern sub-phonemic patterns of surface-phonetic variation, and how specific cue patterns may reflect speech disorders. This report describes a transparent, acoustic cue-based nasalization detection module, that can be used to find not only nasal consonants but also nasalization wherever it appears, e.g., within a vocalic region. We lay out a framework for extracting measurements that are key to detecting nasalization, in particular, the nasality-related spectral peaks P0 and P1, and confirm the utility of these measurements for nasality detection in spoken words. In particular, we present a pre-processing and cue-value extraction framework, and propose a Gaussian Mixture Model-based approach to detect the regions of nasalization in a speech signal. This work on a detection module for nasalization is part of a larger effort to develop a speech recognition system that is based on landmarks and other acoustic cues.

A given word or sound can take on a wide range of acoustic forms in conversational speech, and these patterns of variation are systematic with context. This substantial variability poses a problem for modeling human speech perception, but it can also be highly informative if it can be measured quantitatively; surface phonetic patterns can be used to characterize different speakers, dialects, speaking styles and speech disorders, as well as context-driven variation within a speaker. Adopting the view that individual acoustic cues provide a useful vocabulary for describing these variation patterns, we are developing an algorithm for tabulating the range of acoustic cue production patterns for each phoneme, in a corpus of recorded speech, with the potential for extension to words and phrases. This algorithm will enable the determination of key factors that govern systematic speech variation, including segmental, prosodic and lexical contexts, which can be used to create a profile for a speaker or defined population of speakers, i.e., a Cue Production Profile.

Since the publication of Speaking in 1989, with its extraordinary goal of modelling the entire process of human speech generation from message conceptualisation to articulation, encompassing result...

Irregular pitch periods (IPPs) are associated with grammatically, pragmatically, and clinically significant types of nonmodal phonation, but are challenging to identify. Automatic detection of IPPs is desirable because accurately hand-identifying IPPs is time-consuming and requires training. The authors evaluated an algorithm developed for creaky voice analysis to automatically identify IPPs in recordings of American English conversational speech. To determine a perceptually relevant threshold probability, frame-by-frame creak probabilities were compared to hand labels, yielding a threshold of approximately 0.02. These results indicate a generally good agreement between hand-labeled IPPs and automatic detection, calling for future work investigating effects of linguistic and prosodic context.

Two types of consonant gemination characterize Italian: lexical and syntactic. Italian lexical gemination is contrastive, so that two words may differ by only one geminated consonant. In contrast, syntactic gemination occurs across word boundaries, and affects the initial consonant of a word in specific contexts, such as the presence of a monosyllabic morpheme before the word. This study investigates the acoustic correlates of Italian lexical and syntactic gemination, asking if the correlates for the two types are similar in the case of stop consonants. Results confirmed previous studies showing that duration is a prominent gemination cue, with a lengthened consonant closure and a shortened pre-consonant vowel for both types. Results also revealed the presence, in about 10-12% of instances, of a double stop-release burst, providing strong support for the biphonematic nature of Italian geminated stop consonants. Moreover, the timing of these bursts suggests a different planning process for lexical vs. syntactic geminates. The second burst, when present, is accommodated within the closure interval in syntactic geminates, while lexical geminates are lengthened by the extra burst. This suggests that syntactic gemination occurs during a post-lexical phase of production planning, after timing has already been established., Under revision at The Journal of the Acoustical Society of America

It has long been recognized that speakers organize the words of a spoken utterance into phrases, and accent some words and syllables rather than others. For many years these characteristics were regarded as reflections of the syntax structure of the sentence, signalled by acoustic cues to grouping and prominence, such as fundamental frequency (f0), duration, and amplitude. In the past decades, however, it has become clear that a second set of structures, parallel to but independent of the syntactic structure of a sentence, governs the context-specific surface characteristics of any particular spoken utterance of a sentence, and that the cues signalling these prosodic structures are many. These hierarchical prosodic structures are reflected in the intonation and timing contour of the utterance, and also in the surface phonetic shapes of its words. Consequently, models of speech production planning have begun to incorporate prosodic structure at the phrase level and higher as a governing framework.

This chapter presents the outline of a model of speech-production planning, based on symbolic phonology and the specification of surface-timing patterns using general-purpose timekeeping mechanisms. This phonology-extrinsic-timing-based, three-component (XT/3C) model includes a Phonological Planning Component, to set and prioritize the goals for an utterance; a Phonetic Planning Component, to quantitatively specify the acoustic targets and the movements to achieve them; and a Motor-Sensory Implementation component, to track the planned movements and adjust them to ensure that the targets are reached on time. This approach addresses some of the gaps in earlier speech-production models based on abstract symbolic phonology, by proposing a mechanism for the specification of context-appropriate surface phonetic variation, including timing. In this way it provides an alternative to the Task-Dynamics-based approach embodied in Articulatory Phonology.

This chapter summarizes the basic mechanisms of the Articulatory Phonology model, currently the most thoroughly worked-out model in the literature, with a focus on its system-intrinsic mechanisms used to account for systematic variation in speech timing. Key features of the model are reviewed, and oscillator-based mechanisms are described for timing control for articulatory gestures, control of inter-gestural coordination, prosodic timing control, and the control of overall speech rate. Strengths of the AP/TD approach are discussed, which include facts that are well-accounted-for within this model, such as the predominance of CV syllables within the world’s languages, as well as characteristics of processing within the model that are assumed to be advantageous, such as the avoidance of the need to explicitly plan the details of articulatory movement when planning an utterance. This presentation forms the basis of the evaluation presented in subsequent chapters.

This chapter addresses the nature of the general-purpose timekeeping mechanisms that are assumed in phonology-extrinsic-timing models of speech production. The first part of the chapter discusses some current questions about the nature of these mechanisms. The second part of the chapter presents Lee’s General Tau theory (Lee 1998, 2009), a theory of the temporal guidance of action in voluntary movement. This theory provides a crucial component for our phonology-extrinsic-timing-based, three-component model of speech production because its tau-coupling mechanism provides a way to plan movements with appropriate velocity profiles, as well as endpoint-based movement coordination. In doing so, it provides a general-purpose, phonology-extrinsic alternative to AP/TD’s use of oscillators for the control of the time-course of articulatory movement and coordination.

This chapter presents evidence that challenges models in which phonological representations are temporal in nature, and where timing mechanisms are phonology-specific and intrinsic to the phonology. For example, evidence for separate representations of 1) movement targets vs. other parts of movement, and 2) spatial vs. temporal aspects, is difficult to account for in phonology-intrinsic timing approaches, where all parts of movement are determined by the same, spatiotemporal phonological representation. In the AP/TD phonology-intrinsic-timing approach, surface time is not represented, specified, or tracked, but instead emerges from the phonological system. Evidence that speakers do in fact represent surface time motivates the consideration of a phonology-extrinsic-timing-based approach. This evidence comes from actors’ interactions with perceived events, and also from speech: 1) constraints on the amounts of structure-related lengthening to maintain phonological length contrasts in quantity languages, and 2) different articulatory strategies for producing a given duration pattern. Finally, evidence for general-purpose timekeeping mechanisms in speech and non-speech movements is discussed.

This chapter introduces a theoretical framework, Optimal Control Theory, which will enable a phonology-extrinsic-timing-based, three-component model to determine values of controlled variables, and to model the influence of multiple factors on these parameter values. Key features of Optimal Control Theory models are discussed, as well as evidence for types of movement costs (including the cost of time) used in the models, and predictions of the models for the coordination of multiple effectors and hierarchical control. Finally, the chapter reviews Optimal Control Theory models currently used to account for timing phenomena in speech.

Effects of prosodic structure on surface phonetics are modeled in AP/TD in two ways: 1) via a set of PI and MuT adjustment mechanisms used to model lengthening effects at boundaries and on prominent syllables, and 2) via a hierarchy of coupled syllable, cross-word foot, and phrase oscillators, used to model poly-subconstituent shortening effects, and to control overall speech rate. These mechanisms are challenged by 1) findings presented in previous chapters that suggest that longer durations associated with boundaries and prominences are due to longer surface duration specifications, 2) findings presented here that show that polysyllabic shortening does not affect all words in an utterance, inconsistent with an oscillator-based mechanism that controls all aspects of any produced utterance, and 3) findings relating to speech rate presented in previous chapters which suggest that speech rate specifications relate to surface durations, rather than to planning oscillator frequencies. Patterns of speech timing presented in this chapter thus suggest that there are reasons to be uncertain whether periodicity is a major factor in speech motor control in typical speaking circumstances, and therefore call into question the use of suprasegmental oscillators.

This chapter begins to motivate the development of an alternative approach to speech production by pointing out three potential difficulties with the highly-successful Articulatory Phonology/Task Dynamics approach. First, it discusses the extensive nature of modifications to AP/TD default specifications required to account for the wide variety of surface phonetic forms. The need for a large number of adjustments in AP/TD raises questions about the appropriateness of the AP/TD default-adjustment approach, which would have been more appropriate if the default, non-prominent, phrase-medial, normal-speech-rate specifications could be used most of the time. Second, it discusses the lack of a principled explanation for behaviors described by Fitts’ law. While the theory can accommodate some aspects of Fitts’ law, others are not explained or accommodated. Finally, it suggests that AP/TD’s gestural score architecture raises the risk of spatial interference among overlapping, independent gestures. These three challenges taken together set the stage for the discussion of additional challenges in Chapter 4, which further motivate consideration of phonology-extrinsic-timing-based approaches to speech motor control.

Evidence presented in previous chapters suggests consideration of an alternative to the coupled-oscillator approach to modeling human speech planning and production processes. One alternative approach is based on the symbolic phonemic representations of Generative Phonology. This approach requires a separate mechanism to translate these symbolic representations into the quantitative context-appropriate specifications that can produce the surface form of an utterance. To date models of this type have not included comprehensive versions of components that complete this symbol-to-quantification translation process. This chapter reviews the evidence that supports a three-component model of this process, with an abstract symbolic Phonological Planning Component to specify the goals of the utterance at the level of contrastive feature cues; a Phonetic Planning Component to translate these goals into acoustic targets with their articulatory realizations; and a Motor-Sensory Implementation Component to track and adjust the articulatory movements which create the vocal-tract configurations that realize these goals. Development of such a three-component model would allow direct comparison with performance of the oscillator-based AP/TD model.

This is a book about the architecture of the speech-production planning process and speech motor control. It is written in reaction to a debate in the literature about the nature of phonological representations, which are proposed to be spatiotemporal by some, and symbolic (atemporal) by others. Making this choice about the nature of phonological representation has several fundamental implications for the architecture of the speech-production planning system, notably with regard to the number of planning components and the type of timing mechanisms. In systems with symbolic phonological representations, a separate phonetic planning component is required for speakers to plan the details of surface timing and spatial characteristics for each context. In contrast, the Articulatory Phonology system, which proposes spatiotemporal phonological representations, has a very different architecture, with fewer components. These contrasting assumptions about the spatiotemporal vs. symbolic nature of phonological representations have important consequences for how these two approaches deal with timing issues. This is because time is intrinsic to phonological representations in Articulatory Phonology, but is not part of symbolic phonology. These two proposals are evaluated in light of existing literature on speech and non-speech timing behavior. Evidence that challenges the Articulatory Phonology model inspired a sketch of a new model of the production process, based on symbolic phonological representations and a separate phonetic planning component to specify surface-timing details. This approach provides an appropriate account of what is known about motor timing in general and speech timing in particular. Keywords

This volume compares two very different approaches to modeling speech planning: Articulatory Phonology, with quantitative phonological representations and a set of phonology-intrinsic timing mechanisms, and XT/3C, an alternative model with non-quantitative symbolic phonological representations and general-purpose phonology-extrinsic timing mechanisms. It argues that the motor-control literature for both speech and non-speech supports the XT/3C approach, which expands on earlier models based on Generative Phonology to include a Phonological Planning Component to set the symbolic goals for an utterance, a separate Phonetic Planning Component to provide the quantitative target specifications for the utterance, and a Motor-Sensory Implementation Component to track and adjust the movements required to reach those targets on time. It preserves the insights of a symbol-based phonology, while also providing a comprehensive account of systematic phonetic variation, including timing.