Search

Corpus linguistics (i.e. research on language based on written or oral linguistic material that has been collected and saved) has been considerably developed over the last decades. This development has occurred using more numerous and larger corpora. The increased size of corpora has required the development of automatic tools for their analysis, but also a real reflection on the nature and the objectives of the annotation of corpora. The increased size of the corpora has required the development of automatic tools for their analysis, but also a real reflection on the nature and the objectives of the annotation of the corpora. The enrichment of corpora by a set of specific annotations has emerged, in most cases, as a preliminary to any linguistic analysis.Annotating a corpus consists in adding relevant information for its exploitation. The interest of having annotated corpora (i.e. enriched at different linguistic levels) is to be able to study each annotated levels and the mutual links between them. The work carried out at the LPL on these issues of corpus enrichment was initially meant to make possible the study of multimodality, such as the finest levels of granularity (phonemes) up to the mimo-gestural levels, passing through the syntactic, discursive, prosodic, and interactional levels. It was therefore necessary to think about annotation early on, at the level of information representation. A global annotation scheme allows to consider all these levels in a single formal approach that facilitates their subsequent interrogation.Whatever the level of annotation, several questions have arisen: on the one hand there were questions about the labels used (e.g. decomposition, typology, function, gradual/categorical nature); on the other hand there were questions about the temporary embedment of these labels (location and boundaries). For certain levels of annotation, it will be necessary to describe the levels of dependence between the different labels. These questions must be considered in relation to the research objectives. The work within each annotation level is then relatively similar. It is a question of establishing an annotation scheme that allows the most consistent and robust annotation possible. This scheme is established based on theoretical knowledge and in order to answer research questions. Once the annotation scheme is established, it is also possible to build an annotation guide for potential annotators (expert/naive). Most often, annotations are performed using several annotators to make possible an evaluation of the consistency (inter-annotator agreements). The transversal issue of heterogeneity in human annotations will be addressed in this chapter.In this chapter, we develop some of the main annotation steps that have been performed to annotate corpora manually or automatically, as well as the research issues associated with them. These steps are listed below: - Automatic search of IPUs and orthographic transcriptionFrom the collected primary data, we automatically search for IPUs - Inter-Pausal Units - which allow us to obtain a segmentation into silence blocks versus sound blocks. We then perform the orthographic transcription within these IPUs. This transcription step is crucial as it constitutes the tier from which the other annotation levels will be developed. Here again, the choices made in terms of transcription (chosen convention) have an impact on the links between annotation levels. Once the orthographic transcription is done - and aligned with the signal on IPUs - many annotations can be obtained, either manually, automatically, or semi-automatically. - Phonetic and lexical annotationWe develop, distribute, and regularly enrich an automatic annotation software -SPPAS, which also allows to normalize the transcribed text, which means to obtain the tokens. From these tokens within the IPUs, SPPAS can perform the grapheme-phoneme conversion based on a grammar of the possible pronunciations of each IPU. Finally, SPPAS provides the temporal alignment of phonemes which is now rarely performed manually. However, the manual and automatic aspects of phonetic annotation are different but complementary processes. Thus, spontaneous speech generates phonetic realizations (reductions) that are difficult to manage at the level of automatic alignment. Consequently 1/it may be necessary to manually correct some parts of the automatic alignment: 2/it is possible to use the alignment errors to locate these specific phonetic realizations. In this chapter, we will address the issues related to these two aspects. Other annotations can then be obtained from this phoneme segmentation. They allow to automatically obtain the alignment of tokens; a rule-based system allows to group phonemes into syllables. - Syntactic annotationSyntactic annotation is based on tokens. If there are automatic syntactic analyzers available for written language, syntactic analysis of spoken French remains a challenge. We present here the methodology we have adopted to adapt our writing tagger to handle spontaneous spoken transcripts. If the performances of our MarsaTag tagger are already acceptable, the improvement of our tool will require a multi-level modeling including the phenomena of disfluencies (see below) and the more precise treatment of discourse markers. - Annotation of disfluenciesOral utterances contain many variations in verbal fluency at several levels (e.g. the rate of pronunciation of words, phrases, or clauses). But these variations can also occur at the acoustic and phonetic levels. On the morphological and syntactic levels, some of these variations are translated by real self-interruptions which suspend the syntagmatic flow in the verbal emission. In our corpus analyses, we have planned to keep (in addition to filled or unfilled pauses, discourse elements, interjections) the evidence of the discourse elaboration which are, among other things, initiations or fragments of words and the syntagms’ breaks. This strategy made it possible to envisage a detailed and exhaustive description of these phenomena designated under the term of “disfluency”. - Annotation of speech and interactionsFrom the speech signal and its transcription, it is also possible to consider an annotation of several pragmatic levels such as the thematic organization of conversational interactions. Several levels of annotation will be described in this chapter: the annotation of conversational themes, thematic transitions (i.e. conversational movements that allow to go from one topic to another), and the phases of these transitions. Other phenomena will also be described, such as feedback items and humorous sequences. We will present the annotation protocol associated with these different phenomena as well as the evaluation methods chosen to assess the reliability of these annotations. - Mimogestual annotation From the video signal, it is possible to consider a mimogestual annotation (facial expressions or coverbal manual gestures for example). This can be done either manually or semi-automatically. In this chapter, first we will present the semi-automatic annotation protocol of smiles that we have developed in order to annotate two conversational corpora. We will present the SMAD tool which allows to automatically annotate smiles. Then, we will describe the protocol of correction of these annotations. Finally, we will discuss the evaluation method chosen to assess the robustness of the annotated data. We will also present the manual annotation of coverbal gestures as well as the inherent methodological issues such as annotation schemes and guides, typologies and segmentation. We will give examples of studies carried out at LPL that propose different approaches for gesture annotation.

The research in Clinical Phonetics aims to improve our knowledge of speech pathologies by comparing phonetic methods and research with clinical data and clinicians’ diagnoses. In France, Clinical Phonetics has developed over the last thirty years bringing together phoneticians, linguists, computer scientists, engineers, physicians, speech therapists and clinicians.The “Laboratoire Parole et Langage” (LPL) has a long experience in this field through many collaborations with local hospitals and other partners at the national level. These collaborations have led to the development of several projects dealing with a wide variety of pathologies (dysphonia, Parkinson’s disease, ALS, multiple sclerosis, cancers of the oral cavity and oropharynx, apraxia, etc.). Traditional phonetic methodologies (analysis of speech production and perception) have been adapted to the specific constraints of pathological speech. In particular, the LPL has strongly questioned and improved aerophonometry, speech intelligibility measurements, transcription of pathological speech, and acoustic analysis.There are many research questions in clinical phonetics, but they can be structured around two main axis: (1) what are the contributions of phonetics research to the assessment of speech pathologies?; (2) how can speech pathologies deepen our understanding of the complex mechanisms underlying speech production and perception? The goal of this work is twofold: first, to assist clinicians in the evaluation of the severity of a disorder, either for a punctual assessment or in a therapeutic evaluation (rehabilitation, surgical, pharmacological, electrophysiological treatment); secondly, to improve our knowledge on speech by characterizing the productions considered as “atypical”.Without being exhaustive, this chapter offers an overview of research in clinical phonetics at the LPL. After a brief history recalling the emergence of clinical phonetics at the LPL, we organized the chapter around the current work carried out at the LPL on laryngeal pathologies, motor disturbances, speech intelligibility and speech planning disorders.The first studies at LPL combining phonetics and clinical studies date back to the mid-1970s–1980s. These studies became a research focus at the LPL in the 1990s, thanks to formal collaborations with the ENT department of the Timone University Hospital in Marseilles. At that time, the main research interest for the ENT department was voice disorders analysis. This first research has continued to develop and the field of investigation has been extended to speech disorders, particularly those of neurological origin, such as motor or speech planning disorders, in which collaborations with the neurology department of the Pays d’Aix Hospital have been decisive. These clinical collaborations have also supported the valorization and dissemination of this research in the socio-economic world, in particular with the invention, manufacture and marketing of the Assisted Voice Evaluation (EVA) device.Laryngeal pathologies were the first area explored and have led to many publications. The establishment of vocal assessment, both at perceptual and instrumental level, has monopolized most of the efforts. Nevertheless, and in spite of the contribution of automatic speech processing, the instrumental approach derived from phonetics remains little applied in clinical practice for various reasons discussed in the chapter. Some perspectives are proposed and may open new horizons for clinical phonetics. Laryngeal pathologies have also addressed questions to the linguistic functioning and the phonological representation of language units, in particular with regard to the voicing feature, which is the basis of a basic and almost universal lexical contrast in the languages of the world.Motor disturbances whose symptoms on speech are grouped under the term dysarthria have also been the object of important studies at the laboratory, particularly in the context of Parkinson’s disease. These studies have contributed to the pathophysiological model of the disease by measuring various parameters of speech in the context of monitoring the disease. For example, the therapeutic effects (drug or electrophysiological) were studied via acoustic and physiological (aerodynamic) instrumental analyses. The prosodic dimension was also given an important place. This work has allowed us to highlight the characteristics of the speech of Parkinson’s patients by integrating the interactions between the pathological disorders and the linguistic structure at different levels of organization.The measurement of speech intelligibility, which is a recurrent theme in the laboratory’s research, as developed strongly and has been integrated into a clinical perspective, first with the work on Parkinson’s disease, then in the context of the after-effects of cancers of the oral cavity and oropharynx. Biases related to speech perception have led to various alternatives to intelligibility test batteries allowing to propose linguistically justified solutions adapted to clinical practice. They have also made it possible to study more precisely the articulation between the various perceptual levels involved in decoding and understanding speech.More recently, questions about the planning of speech production have emerged in the laboratory. Speech pathology and clinical phonetics provide a framework for testing these questions. Research comparing healthy subjects vs. those with cognitive deficits related to multiple sclerosis is being conducted at LPL to explore how cognitive constraints influence speech parameters of planning. Finally, we think that an epistemological and multidisciplinary challenge is now facing clinical phonetics. Clinicians and phoneticians have learned to research and work together on complex problems that have a fundamental research issue but also a societal and public health issue. The challenge is therefore to promote interaction and understanding of the issues that move these two communities. Phoneticians must be able to propose fields of application of their knowledge adapted to the real and practical problems of clinicians; and conversely, clinicians must be able to better propose observations that question and nourish reflections on the models of production, perception and understanding of speech and language.

This chapter deals with the linguistic manifestations of conversation. For a long time, linguists have been interested in the levels of oral language (phonology, lexicon, syntax, discourse) separately, particularly based on prepared and non-interactive speech corpora. Research on conversational speech highlights what modern linguistic theories seek to integrate: the interdependence of these dimensions. Moreover, beyond the description of variants, this work allows us to study in a specific way the processes of language production and comprehension, including by considering them jointly, by relying on the most recent theories of interaction.In this chapter, we propose to first discuss the particularities of conversational language and their impact on the units and phenomena generally studied by linguists: phonemes, morphemes, syntagms, prosodic units, propositions, discourse units. These particularities stem from common parameters to most conversational communication situations: temporal pressure, co-presence of participants, fleeting nature of these conversational productions whose form is forgotten as soon as the message has been transmitted. Temporal pressure provides strong characteristics to conversational language linked to time management (lexical and phonetic reductions, disfluencies, etc.). The transience of productions allows the use of particular strategies (repetitions, redundancies, reformulations) which are generally avoided in other language uses. Finally, co-presence allows the use of all the modalities involved in an interaction, and on which the multiple mechanisms of alignment between participants are based. Moreover, the study of these particularities is profoundly impacted by the notions of context and shared knowledge that underlie an interaction. All these particularities due to conversational language suggest adjustments at all levels of production, but in a specific way for each of them. These adjustments are not always described at each level, but even more so, they are rarely analysed together. An important point is to put into perspective the different types of linguistic modifications linked to conversation.Moreover, understanding these general issues requires a precise description of conversation phenomena (cf. conversational analysis and interactional linguistics). These phenomena include turns of speech, listening signals and conversational feedback, or co-produced utterances.These specific phenomena of conversation must be put into perspective with the question of the interaction existing between units. We know, for example, that phonetic reductions more specifically affect sequences with a low semantic and informational load. It is therefore necessary to describe the conditions under which these interactions are possible. In particular, it is necessary to explain them by putting them in perspective with the question of comprehension.The participants in a conversation seek to achieve goals that are external to the communicative activity, but which involve the achievement of these participants’ communicative goals. In order to achieve these goals, the participants must articulate informative elements through the traditionally studied structures of language which allow the construction and articulation of representations. Conversation is a privileged place to observe these phenomena. Interactions between participants are generally built on an exchange of information. This is not a fixed mechanism as linguistics has long explained. It can be based on very superficial processing during which each participant produces and perceives bits of information. The way in which these are aggregated during a conversation is probably not always based on a compositional process, but rather consists in allowing all participants to construct a shared body of knowledge. In this chapter, we will describe the observable evidence of this mechanism, addressing in particular the question of the alignment of linguistic units that we observe during a conversation. This approach to interaction provides a theoretical framework for explaining the conversational phenomena mentioned above.This chapter will be structured as follows: 1/ conversation will be discussed as the articulation between representation and communication through language as a social action; 2/ a second part will aim at drawing up an inventory of the “traces” of the activity of conversation (communicative goal, temporal pressure, co-presence, cognitive load, social pressure) on the different linguistic units (phonemes, lexicon, syntax, prosody); 3/ we will then address the phenomena specific to conversation which are not identifiable in other types of speech (speech turns, disfluencies, conversational feedback, co-produced utterances, non-phrastic utterances, non-lexical utterances, etc.)); 4/ we will finally develop some perspectives related to all these works.

Abstract: This study aims to identify certain phonetic and lexical determinants of major durational variations in spontaneous speech. Three duration samples (SHORT, STANDARD and LONG) were extracted from a corpus of French conversational speech. With high concentrations of liquids and glides in SHORT, vowels show strong temporal distortion towards lengthening. Conversely, fricatives and plosives represent stable temporal supports. Some function words tend to promote reduction while the lengthening of phonemes is due, in large part, to their presence in some frequent items in conversation (discourse markers, interjections). The duration of specific phonemes (/l/, /ʀ/, /w/, /m/) is strongly conditioned by their lexical or communicative context. Phonemic durations are therefore determined, among other things, by the nature of phonemes and by the properties of lexicon and its use in conversation.

Abstract: Most of the time, analyzing the phonetic entities of speech requires the alignment of the speech recording with its phonetic transcription. However, studies on automatic segmentation have predominantly been carried out on read speech or on prepared speech while spontaneous speech refers to a more informal activity, without any preparation. As a consequence, in spontaneous speech numerous phenomena occur such as hesitations, repetitions, feedback, backchannels, non-standard elisions, reduction phenomena, truncated words, and more generally, non-standard pronunciations. Events like laughter, noises and filled pauses are also very frequent in spontaneous speech. This paper aims to compare read speech and spontaneous speech in order to evaluate the impact of speech style on a speech segmentation task. This paper describes the solution implemented into the SPPAS software tool to automatically perform speech segmentation of read and spontaneous speech. This solution consists mainly in two sorts of things: supporting an Enriched Orthographic Transcription for an optimization of the grapheme-to-phoneme conversion and allowing the forced-alignment of the following events: filled pauses, laughter and noises. Actually, these events represent less than 1 % of the tokens in read speech and about 6 % in spontaneous speech. They occur in a maximum of 3 % of the Inter-Pausal Units of a read speech corpus and from 20 % up to 36 % of the Inter-Pausal Units in the spontaneous speech corpora. The UBPA measure – Unit Boundary Positioning Accuracy, of the proposed forced-alignment system is 96.09 % accurate as regards read speech and 96.48 % for spontaneous speech with a delta range of 40 ms. Keywords: spontaneous speech; forced-alignment; paralinguistic events. Resumo: Na maior parte dos casos, a análise de entidades fonéticas da fala exige o alinhamento da gravação da fala com sua transcrição fonética. Entretanto, os estudos sobre segmentação automática têm sido predominantemente desenvolvidos com amostras de fala lida ou fala preparada, uma vez que a fala espontânea refere-se a uma atividade mais informal, sem qualquer preparação. Como consequência, na fala espontânea numerosos fenômenos ocorrem, tais como: hesitações, repetições, feedback, backchannels, elisões não-padrão, fenômenos de redução, palavras truncadas, e mais comumente, pronúncias não-padrão. Eventos como o riso, ruídos e pausas preenchidas também são muito comuns na fala espontânea. Este artigo objetiva comparar a fala lida e a fala espontânea a fim de avaliar o impacto do estilo de fala numa tarefa de segmentação da fala. O artigo descreve a solução implementada no programa SPPAS para a segmentação automática da fala lida e da fala espontânea. Essa solução consiste de principalmente dois aspectos: suporte para uma Transcrição Ortográfica Enriquecida para a otimização da conversão grafema-para-fonema e permissão para o alinhamento forçado (forced-alignment) dos seguintes eventos: pausas preenchidas, riso e ruídos. Tais eventos representam menos de 1% das ocorrências na fala lida e cerca de 6% na fala espontânea. Eles ocorrem com um máximo de 3% nas Unidades Entre-Pausas de um corpus de fala lida e de 20% a 36% nas Pausas Entre-Unidades de corpora de fala espontânea. As medidas APFU – Acurácia no Posicionamento de Fronteiras de Unidade, do sistema de alinhamento forçado (forced-alignment system) proposto são de 96% de acerto no que diz respeito à fala lida e 96,48% para a fala espontânea, com uma variação delta de 4 ms. Palavras-chave: fala espontânea; sistema de alinhamento forçado (forced alignment system); eventos paralinguísticos