Search

International audience; The present work focuses on how signers and speakers demonstrate forms of engagement through visible cues (gaze, body orientation, postures, suspension of actions) with or without providing visual or vocal feedback in multi-party conversations. The dinners involve complex participation frameworks (Goffman,1981, Goodwin, 1981) and coordination of activities (Haddington et al., 2014).Analyses are carried out on the DinLang Corpus (Morgenstern et al., 2021) which includes video recordings of French middle-class speaking and signing families during dinner at home.The specificity of family dinners is that participants are constantly alternating between languaging (Linell, 2009) and eating, which requires a finely tuned orchestration of their bodies. How do participants thus manage to signal their engagement while managing these different activities simultaneously? We presentmultimodal analyses of both signers and speakers from 4 signing families and 4 speaking families (two dinners per family) with children between 3 and 12 years old. Three major trends emerge: participants may display signs of active participation without providing backchanneling while suspending their current activity, or they may do both simultaneously; in other cases, however, participants may withdraw from the current participation arrangement by focusing on a singleactivity. By observing the subtle coordination of bodies in those complex multiparty interactions, this study illustrates the subtle impact that addresses' and overhearers’ engagement and backchanneling have on complex participation frameworks and on interactive dynamics, as participants constantly navigatebetween different activities and modalities.

International audience; Cet article présente une expérimentation de traduction automatique de texte vers la langue des signes (LS). Comme nous ne disposons pas de corpus aligné de grande taille, nous avons exploré une approche à base d'exemples, utilisant AZee, une représentation intermédiaire du discours en LS sous la forme d'expressions hiérarchisées

International audience

International audience; This paper is a contribution to sign language (SL) modeling. We focus on the hitherto imprecise notion of "Multiplicity", assumed to express plurality in French Sign Language (LSF), using AZee approach. AZee is a linguistic and formal approach to modeling LSF. It takes into account the linguistic properties and specificities of LSF while respecting constraints linked to a modeling process. We present the methodology to extract AZee production rules. Based on the analysis of strong form-meaning associations in SL data (elicited image descriptions and short news), we identified two production rules structuring the expression of multiplicity in LSF. We explain how these newly extracted production rules are different from existing ones. Our goal is to refine the AZee approach to allow the coverage of a growing part of LSF. This work could lead to an improvement in SL synthesis and SL automatic translation.

International audience; In natural language settings, many interactions include more than two speakers, and real-life interpretation is based on all types of information available in all modalities. This constitutes a challenge for corpus-based analyses because the information in the audio and visual channels must be included in the coding. The goal of the DINLANG project is to tackle that challenge and analyze spontaneous interactions in family dinner settings (two adults and two to three children). The families use either French, or LSF (French sign language). Our aim is to compare how participants share language across the range of modalities found in vocal and visual languaging in coordination with dining. In order to pinpoint similarities and differences, we had to find a common coding tool for all situations (variations from one family to another) and modalities. Our coding procedure incorporates the use of the ELAN software. We created a template organized around participants, situations, and modalities, rather than around language forms. Spoken language transcription can be integrated, when it exists, but it is not mandatory. Data that has been created with another software can be injected in ELAN files if it is linked using time stamps. Analyses performed with the coded files rely on ELAN's structured search functionalities, which allow to achieve fine-grained temporal analyses and which can be completed by using spreadsheets or R language.

This article presents an original method for Text-to-Sign Translation. It compensates data scarcity using a domain-specific parallel corpus of alignments between text and hierarchical formal descriptions of Sign Language videos in AZee. Based on the detection of similarities present in the source text, the proposed algorithm recursively exploits matches and substitutions of aligned segments to build multiple candidate translations for a novel statement. This helps preserving Sign Language structures as much as possible before falling back on literal translations too quickly, in a generative way. The resulting translations are in the form of AZee expressions, designed to be used as input to avatar synthesis systems. We present a test set tailored to showcase its potential for expressiveness and generation of idiomatic target language, and observed limitations. This work finally opens prospects on how to evaluate translation and linguistic aspects, such as accuracy and grammatical fluency.

To date, comparisons between sign languages (SL) and co-speech gestures (CSG) still face major challenges.One of them is the difficulty to choose, among all the existing transcription systems (TranSys), a single TranSys that could allow researchers to take into account the peculiar characteristics of SL and CSG. Even if the researchers of these two fields describe the same “gesturing body”, they have developed their own representation models (with their own form and scope) to code the features they are interested in. In fact, all these systems adopt a perceptual point of view , i.e. a visuo-spatial representation of the gesture, and are therefore subject to different insight (e.g., in the division of space {1}). This can lead to a fragmentation of the descriptive modalities, limiting the identification of formal and functional common characteristics between SL and CVG. Doing so, researchers currently ignore a rallying perspective allowing to describe both SL and CSG: the body structure. To fill this gap, we have developed Typannot {2}, a typographic TranSys that aims at representing gestures from an articulatory standpoint, using a descriptive approach rooted in the body: the kinesiological approach {3}.Designed for SL but suitable for CSG too, Typannot is a formal TranSys that reproduces the articulatory characteristics of each segment of the body (fingers, hand, forearm, arm; shoulder, torso and hip; neck, head, mouth and eyes). It describes their possibilities of organization from the body articulatory perspective: the positions and the transformations of the degrees of freedom. In Typannot, each articulator (upper-limb, bust, mouth and eyes segments) is associated with a different OpenType font; every character (in Unicode sense) of a Typannot font contains only one specific information (i.e., the degree of extension or abduction) necessary to describe all the features of a segment. An advanced system of typographic ligature allows the user to see a single “holistic” glyph containing every feature, thus assuring searchability in a still readable way.Another challenge faced by LS and CSG TranSys is the time required to transcribe gestural phenomena, which drastically restricts the size of multimodal corpus and therefore the possibility of generalizations. Here again, Typannot deals with this issue by facilitating and automating the work. On one hand, the input interface has been designed to help transcribers produce transcription easily; on the other hand, the articulatory model used in this TranSys can easily be associated with Motion Capture (MoCap) data. This correspondence makes it possible to envisage a (semi)automatic transcription of body structure, drastically minimizing the transcription time.Typannot has been used within the ANR-LexiKHuM project (2021-2025) {4} to transcribe the body postures present in the multimodal and parallel corpus DEGELS1 {5}, consisting of conversations in LSF (≃ 30 min) and in spoken French (≃ 30 min). One of the objectives of LexiKHuM is to identify the epistemic markers common to SL and to CSG, in particular those that are not brachial. Thanks to Typannot, an articulatory description of the upper trunk and of the facial expressions could be implemented out in a coherent manner between the data in SL and CSG. The corpus was transcribed using the ELAN software {6} and the annotation was carried out manually and with the sensor-free MoCap software AlphaPose {7}, allowing a semi-automatic annotation in Typannot characters.In this talk, we will present the kinesiological approach and the Typannot typographic font system; some transcribed data and preliminary results of LexiKHuM will be exposed, also focusing on the time used for manual and semi-automatic transcription.******This presentation is dedicated to the memory of Dominique Boutet (1966-2020), who was co-author of our proposal in 2020 and co-coordinator of the Typannot and the LexiKHuM projects.Bibliography{1} McNeill, D. (1992). Hands and mind: what gestures reveal about thought. Chicago: University of Chicago Press.{2} Danet C., Boutet D., Doan P., Bianchini C.S., Contesse A., Chevrefils L., Rébulard M., Thomas C. & Dauphin J.-F. (2021). Transcribing sign languages with Typannot: a typographic system which retains and displays layers of information. Grapholinguistics and its Applications, 5(2), 1009–1037. https://doi.org/10.36824/2020-graf-dane{3} Boutet D. (2018). Pour une approche kinésiologique de la gestualité: synthèse. Habilitation à diriger des recherches, Université́ de Rouen-Normandie.{4} Catteau F., Morgenstern A. & Bianchini C.S. (2022). From gestures to kinesthetic modality: how to express epistemicity in a haptic device for human-machine interaction. 9th Intl Conf. of Intl. Soc. for Gesture Studies [ISGS 2022].{5} Braffort A. & Boutora L. (2012), DEGELS1: a comparable corpus of French Sign Language and co-speech gestures, Proc. 8th Intl. Conf. on Language Resources and Evaluation [LREC 2012].{6} Wittenburg P., Brugman H., Russel A., Klassmann A. & Sloetjes H. (2006). ELAN: a professional framework for multimodality research. Proc. 5th Intl. Conf. on Language Resources and Evaluation [LREC 2006].{7} Fang H.-S., Xie S., Tai Y.-W. & Lu C. (2018). RMPE: Regional Multi-person Pose Estimation. arXiv:161200137 [cs].

Research on sign languages (SLs) requires dedicated, efficient and comprehensive transcription systems to analyze and compare the sign parameters; at present, many transcription systems focus on manual parameters, relegating the non-manual component to a lesser role. This article presents Typannot, a formal transcription system, and in particular its application to mouth gestures: 1) first, exposing its kinesiological approach, i.e. an intrinsic articulatory description anchored in the body; 2) then, showing its conception to integrate linguistic, graphic and technical aspects within a typeface; 3) finally, presenting its application to a corpus in French Sign Language (LSF) recorded with motion capture., La recherche sur les langues des signes (LS) nécessite des systèmes de transcription dédiés, efficaces et complets pour analyser et comparer les paramètres des signes; à l'heure actuelle, de nombreux systèmes de transcription se concentrent sur les paramètres manuels, reléguant la composante non manuelle à un rôle moindre. Cet article présente Typannot, un système de transcription formelle, et en particulier son application aux gestes de la bouche: 1) d'abord, il expose son approche kinésiologique, c'est-à-dire une description articulatoire intrinsèque ancrée dans le corps; 2) puis, il montre sa conception pour intégrer les aspects linguistiques, graphiques et techniques au sein d'une police de caractères; 3) enfin, il présente son application à un corpus en Langue des Signes Française (LSF) enregistré en motion capture., La ricerca sulle lingue dei segni (SL) richiede sistemi di trascrizione dedicati, efficienti e completi per analizzare e confrontare i parametri dei segni; attualmente molti sistemi di trascrizione si concentrano sui parametri manuali, relegando la componente non manuale ad un ruolo minore. Questo articolo presenta Typannot, un sistema di trascrizione formale, e in particolare la sua applicazione ai gesti della bocca: 1) in primo luogo, esponendo il suo approccio kinesiologico, cioè una descrizione articolatoria intrinseca ancorata al corpo; 2) poi, mostrando la sua concezione per integrare gli aspetti linguistici, grafici e tecnici all'interno di un carattere tipografico; 3) infine, presentando la propria applicazione ad un corpus in Lingua dei Segni Francese (LSF) registrato con motion capture.

Facial actions are among the least documented parameters in sign languages (SL). Yet facial signs have a key role in SL, operating either as inseparable parts of a given manual sign or carrying its own meaning independently. One could argue that a major obstacle to this research is the difficulty related to its study. It is true that the scale at which facial actions occur is much smaller and the difference between facial actions is much finer. Eight parts (jaw, lips, eyebrow, etc.) divided between the upper face [Eye Action (EA)] and the lower face [Mouth Actions (MA)] are at play in facial actions. Most of these parts have a limited range of motion yet can communicate lots of information. Indeed, our brains have developed to recognize the most subtle changes in human faces (Ekman, 1984) as they carry crucial information for human interactions. If this has enabled us to implicitly perceive others' emotions, the conscious segmentation, organization, and analysis of such subtle movements remains a challenge.The existing research has mainly been focused on MA, as the lower part of the face plays a greater role in SL. Studies have brought to light the different linguistic values of MA, for example Mouthings, which are an articulation without voice of spoken words in the vocal language of the given country, and Mouth Gestures, which are mouth movements completely independent from vocal languages and can carry various types of values (adverbial, semantically empty, enacting, mouth activity in the context of the whole-face) (Fontana, 2008; Crasborn et al., 2008).These findings reveal the importance of MA in SL; yet, their study, difficult by nature, is made even more difficult by the lack of a complete and efficient transcription system. Several preliminary works (Bergman & Wallin, 2001; Sutton Spence & Day, 2001; Ajello et al., 2001) were led by a corpus driven approach resulting in different ways of categorizing mouth segments and positions. Some authors engineered a series of symbols to annotate a list of previously defined MA (HamNoSys of Prillwitz et al., 1989; Vogst-Svenden, 2001; Hohenberger & Happ, 2001). To date, the most developed tool available is SignWriting (Sutton, 1995), which offers 187 different symbols to transcribe MA. They can be compounded into an emoji like image. If the system offers an easy and intuitive way to symbolize facial expressions and emotions, it lacks granularity and fails to transcribe complex facial actions. Neither does it allow searchability as each composition results in an image from which no data can be extracted. Lastly, the symbols are based on the visible outer shapes of the face and not based on which facial parts were in action in order to create the facial expression. These features are setting limitations in the capacity to offer a deeper analysis of facial signs, of their values and of their interactions with other SL components.Typannot (Danet et al., 2020) is developing a complete typographic system to transcribe SL. For each pre defined SL parameter (handshape, initial location of upper limb, movement and facial actions), a dedicated type font, grouped in a complete type family, is currently being developed. All fonts within the Typannot system are based on the same structure. The fundamental principle upon which their graphematic formulas were designed is that the Typannot system transcribes which parts are in action in the making of a sign instead of transcribing the resulting image observed from an outer point of view.Each parameter font works using a two layer structure: the first displays all articulatory pieces of information required to make a sign [part(s) involved and value(s) of each part] in a string of individual characters; the second combines all pieces of information into a single morphological and intuitive to read glyph. This two layer system and the combination of multiple characters into glyphs is made possible through typographic engineering. To be fully efficient, these glyphs need to be legible and scriptable by all kinds of users, from beginners to experts. They also need to offer complete searchability and data mining by retaining every feature of every sign.In this presentation, Typannot graphematic formula using the XYZ axes as a frame of reference will be described. This spatial model allows the formula to be concise yet exhaustive in the description of any given mouth action. Then, the conception of the typographic system, from the initial sketches to the type design will be explained. Furthermore, the type engineering process enabling pieces of information from every part to be embedded into each of the Typannot glyphs will be demonstrated. Next, the digital interface currently being developed to allow complete accessibility and usability of the typographic system is presented. Lastly, questions about the possible evolution brought by the use of such a system in the analysis and understanding of MA, their roles and interactions within SL will be discussed.Bibliographical referencesAjello, R., Mazzoni, L., & Nicolai, F. (2001). Linguistic gestures: mouthing in Italian Sign Language. International studies on sign language and communication of the deaf, 39.Bergman, B., & Wallin, L. (2001). A preliminary analysis of visual mouth segments in Swedish Sign Language. International studies on sign language and communication of the deaf, 39.Crasborn, O., van der Kooij, E., Waters, D., Woll, B., & Mesch, J. (2008). Frequency and spreading behavior of different types of mouth actions in three sign languages. Sign Language & Linguistics, 11(1), 45-67.Danet, C., Boutet, D., Doan, P., Bianchini, C.S., & Contesse, A. (2020). Transcribing sign languages with TYPANNOT: the typographic system that retains and displays layers of information. Proceedings of Grapholinguistics in the 21st Century, 5, 1007-1035.Ekman, P. (1984). Expression and the nature of emotion. Approaches to Emotion, 3(19), 344.Fontana, S. (2008). Mouth actions as gesture in sign language. Gesture, 8(1), 104-123.Hohenberger, A., & Happ, D. (2001). The linguistic primacy of signs and mouth gestures over mouthings: evidence from language productions in German Sign Language. International studies on sign language and communication of the deaf, 39.Prillwitz, P., Leven, R., Zienert, H., Hamke, T., & Henning, J. (1989). HamNoSys version 2.0: Hamburg Notation System for Sign Languages: an introductory guide. International Studies on Sign Language and Communication of the Deaf, 5.Sutton, V. (1995). Lessons In SignWriting. The Deaf Action Committee for SignWriting.Sutton Spence, R., & Day, L. (2001). Mouthings and mouth gestures in British Sign Language. International studies on sign language and communication of the deaf, 39.Vogst Svenden, M. (2001). A comparison of mouth gestures and mouthings in Norwegian Sign Language. International studies on sign language and communication of the deaf, 39.

(poster); International audience; Two main issues are still at stake for the transcription of gestures (i.e., co speech gestures and SL signs): i/ the time used to transcribe; and ii/ a sole system for these two kinds of transcriptions (Slobin et al., 2001). The time needed to annotate gestural phenomena restricts drastically the size of the corpus and therefore the possible generalizations; on the other side, the lack of a common transcription system makes difficult any comparison across studies of sign or gesture forms (see the description of “pointing gestures with a functional point of view” [Kita, 2003] and of “buoys in SL” [Liddell, 2003]) and means of expression (for example, many co-speech gestures introduced in SL discourse). This fact increases the differences between a “genuine” linguistic production and the co-speech gestures (Schembri et al., 2005; Goldin-Meadow & Brentari, 2017) and sometimes even essentializing these differences (Singleton et al., 1995; McNeill, 2015).A formal transcription system designed for SL (developed by the Typannot project) - but available for co-speech gestures as well - depicts each segment (hand, forearm, arm and shoulder) of the upper limb, palm orientation, handshapes and non-manual parameters (including mouth actions, facial expressions, head and torso positions). Each articulator is associated to a different OpenType font (e.g., Typannot_HandShape font, Typannot_MouthAction font, etc.) belonging to a specific font family (Boutet et al., 2018). Every character (in Unicode sense) of a Typannot_font contains only one specific information (for example, fingers, shape, angle and closeness between fingers are provided for handshapes; Bianchini et al., 2018) necessary to describe all the features of an articulator. An advanced system of typographic ligature allows the user to see a single “holistic” glyph containing every feature, assuring searchability in a readable way.Because of their formal approach, using these Typannot_fonts allows: i/ a comparison between SL and gestures/mimics in multimodal discourses; and ii/ in SL, detailed sub-parameter queries, e.g., about the angle of a segment according to a peculiar degree of freedom (e.g., full extension of the hand) and up to the relations among segments - it is even possible to discriminate pointing forms, regardless of a finger. A corpus of multimodal French discourses and French Sign Language (LSF), of several minutes each, has been transcribed (with ELAN) using the Typannot_fonts in manual and semi-automatic ways. This last kind of transcription in Typannot_fonts is made filming the speaker wearing an IMU device, and processing the video with OpenFace (Baltrušaitis et al., 2018).In this talk, we will present this family fonts system, its structure, the annotated features, the possible queries made at two levels, the characters themselves and their ligatured glyphs. Some transcribed data and preliminary results will be exposed, focusing on the time used for transcription, by comparison of the ratio between manual and semi-automatic processing.We propose, as a side event to the conference, to set up a tutorial session (3 hours, small group) to learn how to use the Typannot transcription environment. BIBLIOGRAPHYBaltrušaitis T., Zadeh A., Chong Lim Y, Morency L-P. 2018. OpenFace 2.0: facial behavior analysis toolkit. 13th IEEE Intl Conf. Automatic Face & Gesture Recognition (FG 2018): 59-66. doi.org/10.1109/FG.2018.00019Bianchini C.S., Chèvrefils L., Danet C., Doan P., Rébulard M., Contesse A., Boutet D. 2018. Coding movement in sign languages: the Typannot approach. Proc. ACM 5th Intl Conf. Movement and Computing (MoCo'18), sect. 1(#9): 1-8.Boutet D., Doan P., Bianchini C.S., Danet C., Goguely T., Rébulard M. 2018. Systèmes graphématiques et écritures des langues signées. in “Signatures: (essais en) sémiotique de l’écriture” (J.M. Klinkenberg, S. Polis eds). Signata, 9: 391-426.Goldin-Meadow S., Brentari D. 2017. Gesture, sign and language: the coming of age of sign language and gesture studies. Behavioral and Brain Sciences, 40(e46): 1-82.Kita S. 2003. Pointing: a foundational building block of human communication. in "Pointing: where language, culture, and cognition meet" (S. Kita ed.). Erlbaum (Mahwah NJ): 1-8.Liddell S.K. 2003. Grammar, gesture, and meaning in American Sign Language. Cambridge University Press.McNeill D. (2015). Why we gesture. in: "Why we gesture: the surprising role of hand movements in communication". Cambridge University Press: 3-20. doi.org/10.1017/CBO9781316480526.002Schembri A., Jones C., Burnham D. 2005. Comparing action gestures and classifier verbs of motion: evidence from Australian Sign Language, Taiwan Sign Language, and nonsigners’ gestures without speech. Journal of Deaf Studies and Deaf Education, 10(3), 272-290. doi.org/10.1093/deafed/eni029Singleton J.L., Goldin-Meadow S., McNeill D. 1995. The cataclysmic break between gesticulation and sign: evidence against a unified continuum of gestural communication. in: "Language, gesture, and space" (K. Emmorey, J. Reilly eds). Hillsdale NJ: Lawrence Erlbaum Associates Publishers: 287-311.Slobin DI., Hoiting N., Anthony M., Biederman Y., Kuntze M., Lindert R., Pyers J., Thumann H., Weinberg A. 2001. Sign language transcription at the level of meaning components: the Berkeley Transcription System (BTS). Sign Language & Linguistics 4(1-2): 63-104. doi.org/10.1075/sll.4.12.07slo

see publication #024 - HAL‑02552935[conferenza annullata causa Covid]; International audience; The transcription of sign languages (SL) should take into account the gestural nature of these languages and, above all, why many typefont systems (e.g., HamNoSys, SignWriting) failed in establish themselves as a transcription tool (e.g., given the focus on the hands and the ego centered frame of reference). Gesturing sets all the segments of the upper limb in motion according to degrees of freedom, particular amplitudes and a series of intrinsic frames of reference centered on each segment. Typannot, the typefont system presented here, is based on these characteristics and integrates them into a structural hierarchy with an informational level corresponding to the features (characters), gathered in a level of glyphs composed. Typannot usability is ensured by respecting four design principles (genericity, modularity, legibility/readability and inscriptibility) and the use of a data entry interfaces managing these information levels. In order to illustrate the use of this font, analyses of the handshapes and proper location of the hand show the influence of practical gestures on symbolic gestures (i.e., SL signs). Considering the still cumbersome annotation time that such a transcription system imposes, the objective is to allow a “gestural dictation”, a direct transcription from motion capture data. This perspective should also facilitate the transcription of all the so called co verbal gestures of any vocal language.; Transcrire les langues des signes (LS) nécessite de prendre en considération leur nature gestuelle et de comprendre les raisons (parmi lesquelles centration sur les mains et utilisation d’un cadre de référence égocentré) pour lesquelles des systèmes typographiques (HamNoSys, SignWriting) ont échoué à s’imposer comme outil de transcription. La gestualité met en mouvement tous les segments du membre supérieur selon des degrés de liberté, en fonction d’amplitudes particulières et à travers une série de cadres de référence intrinsèque centrée sur chaque segment. Typannot, le système typographique présenté ici, repose sur ces caractéristiques et les intègre dans une hiérarchie structurelle ménageant un niveau informationnel correspondant aux traits (caractères), rassemblés dans un niveau de glyphes composés. La facilité d’utilisation de Typannot est assurée par le respect de quatre principes de conception (généricité, modularité, lisibilité et inscriptibilité) et l’utilisation d’une interface de saisie ménageant ces niveaux d’informations. Afin d’illustrer l’usage de Typannot, ont été menée des analyses portant sur les configurations et l’emplacement propre de la main, qui montrent l’influence de la gestualité praxique sur la gestualité symbolique (c.-à-d. les signes des LS). Vue la durée d’annotation encore très importante qu’un tel système de transcription impose, l’objectif est de permettre une "dictée gestuelle", une transcription directe à partir des données de capture de mouvement. Cette perspective devrait aussi faciliter la transcription de l’ensemble des gestes dits co-verbaux de n’importe quelle langue vocale.

see publication #026 - HAL‑xxxxxxx[presentazione "elettronica" a distanza]; International audience; Sign Languages (SL) are among the scarce languages which resist to be written. We will explain some of the reasons why it is so difficult to write them down and will expose the main solutions that have been developed to transcribe their key components, i.e., hand gestures. The two main transcription systems (HamNoSys [Prillwitz 1989] and SignWriting [Sutton 1995]) approach handshape representation with a fixed set of hand structures, in which orientation and location of the hands are defined in relation to the egocentric space (left, right, forward, backward, etc.). Other phonological approaches - e.g., Hand Tier Model (Sandler, 1989), Prosodic Model (Brentari, 1998), Hold-Movement Model (Johnson & Liddell, 2010) -, also deal with hand gestures using a relative perspective: orientation refers to the palm orientation according to different body planes (sagittal, etc.), location refers to the hand position in relation to the whole body, while movement refers to its course in space. This centralized perspective puts the focus on the hand and diminishes the role of other segments. This induces the idea that meaning is only carried by the hand and its trajectory/figure in space, while in fact it can also be carried by forearms, arms and shoulders.Although those approaches are very efficient at representing hand gestures from an external perspective, they disregard important informative dimensions by ignoring the internal perspective. We would like to promote another way to examine SL, in which the whole upper limb participates in expression of meaning. While the phonological description of handshape is relevant, its location and orientation are determined by the kinesiological system of the upper limb. This novel integration of the body parts configuration opens new possibilities to describe the active components of gestures and not just to show the final results (the hand behavior in space). This intrinsic point of view permits the study of gestures in new ways (i.e., investigating if and how meaning could originate from the structural various inner transformations [Boutet, 2015]) but presents new challenges on both conceptual and practical levels. How are the different levels of information organized? How can such complex phonological descriptions achieve readability? How can the linguist easily describe gestures from an intrinsic perspective that is experienced but not seen? How will the researcher transcribe several parameters at the same time?Typannot is a typographic system that structures phonological descriptions of SL signs into four distinct levels of information: the first level defines the parameters of the sign (i.e., handshape, initial localization, etc.); the second level describes the parts defining each parameter (i.e., fingers, thumb, etc.); the third level describes the features of the parts (i.e., form of the finger, angle of rotation, etc.); finally the fourth level shows the values of those features (i.e., curved, closed, etc.). Typannot allows both a linear and a multilinear organization of the information in transcription software like ELAN using a dedicated typographic system. The designs and functionalities of the system are built upon four principles: genericity, readability, modularity and inscriptibility. Genericity refers to the discretization of all four levels components (features levels in phonology) that structure SL. Each sign can be described by a sequence of generic characters (mapped in Unicode) using a predefined syntax to carry information consistently. This fundamental principle allows transcription to be analyzed and researched at various levels. Readability refers to the ease of reading conferred by a certain form of writing (McMonnies, 1999). While the generic form favours an analytical and fundamentally decomposed layout of components, the concept of readability requires to synthesize the string of information into a single, immediately recognizable, form. Visual analogy is used to design composed glyphs that can be viewed as a “logographic summary” of the parameter (i.e., handshape). The most important property of composed glyphs is that they retain all the pieces of information from their generic forms, preserving the analytic purpose of transcription a.k.a. data searchability. Users can switch between the two forms of reading using standard OpenType functionalities (i.e., ligature or contextual alternate). Modularity is a design principle which allows an automatic generation of any composed glyphs using glyphic modules and computer algorithms. Because of Unicode’s technical limit of 64 000 glyphs per font, Typannot’s repertoire of composed glyphs will start with a standard set, to be progressively extended by collecting “unmapped” combinations that users transcribe, making it a corpus-driven font system. Finally, the concept of inscriptibility refers to the writer’s ability to associate the transcription system (forms and functionalities) with the multiple performative dimensions of his/her language. The gestural-visual modality used by SL allows to consider a writing system based on corporal analogies as the mode of representation of components. It is worth reminding that, for SL, oral and written forms share the same gestural-visual modalities. The written word is not limited to conventional symbols disconnected from the dimensions of orality, as in the case of vocal language. To conclude, it is necessary to develop dedicated input systems in order to help users dealing with the multiple dimensions of SL signs, especially when they are seen from an intrinsic perspective. Two important ideas have to be kept in mind: first, the input systems must provide the necessary control over the complex parametric levels of information; second, they should visualize the effect of the parametric input to serve as an external visual feedback of the internal phonological encoding. Also, the current development of motion capture and recognition technologies has a deep impact on how transcription can be performed “directly” using the body. We will show the integration of such a possibility for inputting handshape phonological transcriptions and promoting the principle of inscriptibility. Bibliographic referencesBrentari, D. (1998). A prosodic model of sign language phonology. The MIT Press.Boutet, D. (2015). Conditions formelles d’une analyse de la négation gestuelle. Vestnik of Moscow State Linguistic University "Discourse as social practice: priorities and prospects", 6 (717): 116-129Johnson, R.E., & Liddell, S.K. (2010). Toward a phonetic representation of signs: sequentiality and contrast. Sign Language Studies, 11(2), 241-274. https://doi.org/10.1353/sls.2010.0008 Johnston, T. (2008). Corpus linguistics and signed languages: no lemmata, no corpus. 3rd Workshop on the Representation and Processing of Sign Languages, 82-88. O. Crasborn, E. Efthimiou, T. Hanke, E.D. Thoutenhoofd, I Zwitserlood eds.McMonnies, C.W. (1999). Chart construction and letter legibility/readability. Ophthalmic and Physiological Optics, 19(6), 498-506. https://doi.org/10.1046/j.1475-1313.1999.00460.x Prillwitz, S., Leven, R., Zienert, H., Hanke, T., & Henning, J. (1989). Hamburg notation system for sign languages: an introductory guide. Signum Press, Hamburg.Sandler, W. (1989). Phonological representation of the sign: linearity and nonlinearity in American Sign Language. vol. 32. Walter de Gruyter, Dordrecht.Sutton, V. (1995). Lessons in SignWriting. DAC. La Jolla CA

International audience

Scholare-ISBN 2261-2424; International audience; The transcription of sign languages (SL) should take into account the gestural nature of these languages and, above all, why many typefont systems (e.g., HamNoSys, SignWriting) failed in establish themselves as a transcription tool (e.g., given the focus on the hands and the ego-centered frame of reference). Gesturing sets all the segments of the upper limb in motion according to degrees of freedom, particular amplitudes and a series of intrinsic frames of reference centered on each segment. Typannot, the typefont system presented here, is based on these characteristics and integrates them into a structural hierarchy with an informational level corresponding to the features (characters), gathered in a level of glyphs composed. Typannot usability is ensured by respecting four design principles (genericity, modularity, legibility/readability and inscriptibility) and the use of a data entry interfaces managing these information levels. In order to illustrate the use of this font, analyses of the handshapes and proper location of the hand show the influence of practical gestures on symbolic gestures (i.e., SL signs). Considering the still cumbersome annotation time that such a transcription system imposes, the objective is to allow a “gestural dictation”, a direct transcription from motion capture data. This perspective should also facilitate the transcription of all the so-called co-verbal gestures of any vocal language.; Transcrire les langues des signes (LS) nécessite de prendre en considération leur nature gestuelle et de comprendre les raisons (parmi lesquelles centration sur les mains et utilisation d'un cadre de référence égocentré) pour lesquelles des systèmes typographiques (HamNoSys, SignWriting) ont échoué à s'imposer comme outil de transcription. La gestualité met en mouvement tous les segments du membre supérieur selon des degrés de liberté, en fonction d'amplitudes particulières et à travers une série de cadres de référence intrinsèque centrée sur chaque segment. Typannot, le système typographique présenté ici, repose sur ces caractéristiques et les intègre dans une hiérarchie structurelle ménageant un niveau informationnel correspondant aux traits (caractères), rassemblés dans un niveau de glyphes composés. La facilité d'utilisation de Typannot est assurée par le respect de quatre principes de conception (généricité, modularité, lisibilité et inscriptibilité) et l'utilisation d'une interface de saisie ménageant ces niveaux d'informations. Afin d'illustrer l'usage de Typannot, ont été menée des analyses portant sur les configurations et l'emplacement propre de la main, qui montrent l'influence de la gestualité praxique sur la gestualité symbolique (c.-à-d. les signes des LS). Vue la durée d'annotation encore très importante qu'un tel système de transcription impose, l'objectif est de permettre une "dictée gestuelle", une transcription directe à partir des données de capture de mouvement. Cette perspective devrait aussi faciliter la transcription de l'ensemble des gestes dits co-verbaux de n'importe quelle langue vocale.; La trascrizione delle lingue dei segni (LS) richiede di considerare sia la loro natura gestuale che di comprendere le ragioni (incluso il centraggio sulle mani e l'uso di un quadro di riferimento egocentrico) per cui i sistemi tipografici (HamNoSys, SignWriting) non sono riusciti ad affermarsi come strumento di trascrizione. I gesti mettono in movimento tutti i segmenti dell'arto superiore in base ai gradi di libertà e le loro ampiezze particolari, tramite una serie di strutture di riferimento intrinseche, centrate su ciascun segmento. Typannot, il sistema tipografico qui presentato, si basa su queste caratteristiche e le integra in una gerarchia strutturale fornendo un livello di informazione corrispondente ai tratti (caratteri), raccolti in un livello di glifi composti. La facilità d'uso di Typannot è assicurata dal rispetto di quattro principi di progettazione (generalità, modularità, leggibilità e scrivibilità) e dall'uso di un'interfaccia di input che gestisce questi livelli di informazione. Al fine di illustrare l'uso di Typannot, sono state condotte analisi sulle configurazioni e sulla posizione corretta della mano, che mostrano l'influenza dei gesti concreti sui gesti simbolici (cioè i segni della LS). Dato il tempo di annotazione ancora molto importante richiesto da un tale sistema di trascrizione, l'obiettivo ulteriore è permettere una "dettatura gestuale", una trascrizione diretta dai dati di motion capture. Questa prospettiva dovrebbe anche facilitare la trascrizione di tutti i cosiddetti gesti co-verbali di qualsiasi linguaggio vocale.

International audience; There are more than 140 sign languages (SLs) in the world and studying them is a relatively recent field of research (starting in the 1960s). Linguists have the need to represent the different levels of gestures that make up the signs in order to analyze the way SLs work. Such transcription requires the use of a dedicated graphic system (Slobin et al. 2001). TYPANNOT, the transcription system presented in this article, is a typographic system that allows the description of all formal features of SLs. Our contribution to the field of grapholinguistics is a phonological model and a transcription system for SLs that are rooted in the articulatory possibilities of the signer's body. Compared to existing graphematic systems, our approach of SLs description is both phonological, allowing descriptions of the different articulatory structures (low level) involved in SLs, and logographical, allowing users to read the transcriptions from a unified perspective (high level). We will detail the design principles that drive the development of such a typographic system, the graphemic model that derives from linguistic study, and the tools that allow researchers to use TYPANNOT to its fullest capacities. This article also outlines the kinesiological approach (Boutet, 2018), which TYPANNOT uses, noting radical changes in the way researchers should look at meaning through gesture. This approach opens new perspectives in researching movement itself as a central source of meaning in human communication via gesture.; Il existe plus de 140 langues des signes (LS) dans le monde et leur étude est un domaine de recherche relativement récent. Les linguistes ont besoin de représenter les différents niveaux de gestes qui composent les signes afin d'analyser le fonctionnement des LS. Une telle transcription nécessite l'utilisation d'un système graphique dédié. Typannot, le système de transcription présenté dans cet article, est un système typographique qui permet la description de toutes les fonctionnalités formelles des LS: la contribution au domaine de la grapholinguistique est un modèle phonologique et un système de transcription pour les LS, les deux enracinés dans les possibilités articulatoires du corps du signeur. Par rapport aux systèmes graphématiques existants, l'approche à la description des LS est à la fois phonologique - permettant des descriptions des différentes structures articulatoires (de bas niveau) impliquées dans les LS -, et logographique - permettant de lire les transcriptions dans une perspective unifiée (de haut niveau). On détaille les principes de conception qui mènent au développement d'un tel système typographique, le modèle graphémique issu de l'étude linguistique et les outils qui permettent aux chercheurs d'utiliser Typannot à son maximum. Cet article décrit également l'approche kinésiologique que Typannot utilise, notant des changements radicaux dans la façon dont les chercheurs devraient étudier le sens par le geste. Cette approche ouvre de nouvelles perspectives dans la recherche du mouvement, lui-même considéré comme source centrale de sens dans la communication gestuelle.; Nel mondo esistono più di 140 lingue dei segni (LS), e studiarle è un campo di ricerca relativamente recente. I linguisti hanno la necessità di rappresentare i diversi livelli di gestualità che compongono i segni al fine di analizzare il modo in cui funzionano le LS. Tale trascrizione richiede l'utilizzo di un sistema grafico dedicato. Typannot, il sistema di trascrizione presentato in questo articolo, è un sistema tipografico che consente di descrivere tutte le caratteristiche formali delle LS: il contributo al campo della grafolinguistica è un modello fonologico e un sistema di trascrizione per LS, ambedue radicati nelle possibilità articolatorie del corpo del segnante. Rispetto ai sistemi grafematici esistenti, l'approccio alla descrizione delle LS è sia fonologico – descrivendo le strutture articolatorie (basso livello) coinvolte negli SL -, sia logografico - consentendo di leggere le trascrizioni in una prospettiva unificata (alto livello). Vengono di seguito descritti i principi progettuali che guidano lo sviluppo di Typannot, il modello grafico derivato dallo studio linguistico ed infine gli strumenti che consentono ai ricercatori il pieno utilizzo di Typannot. Questo articolo delinea anche l'approccio kinesiologico utilizzato in Typannot, che palesa la necessità di cambiamenti radicali nello studio dei significati attraverso i gesti. Questo approccio apre nuove prospettive nella ricerca sulla comunicazione gestuale, ponendo il movimento stesso come fonte centrale di significato.

(exposition); International audience; Cette recherche développe une nouvelle approche graphique basée sur les possibilités graphiques du geste en langue des signes (LS). En focalisant la réflexion sur le mouvement de la main, l'objet de la recherche est de comprendre comment le signeur s'en sert pour tracer un signe dans l'espace et ensuite le tracer sur le papier.Dans le cadre d'une recherche sur la création scripturale des langues des signes (LS), la thèse expose son engagement à prendre en considération le point de vue des locuteurs sourds. Ceci afin de faire émerger des formes graphiques de la LS, et plus particulièrement sur sa gestuelle.La démonstration, proposée sous une forme ludique, permet de montrer comment fonctionne Typannot : les symboles créés graphiquement représentent la forme de la main de la personne en train de signer.gestualscript.fr

International audience; GestualScript est une équipe de recherche transdisciplinaire fondée en 2008 qui travaille autour de l’écriture des langues des signes. Elle est intégrée au laboratoire Design-e de l’Ésad d’Amiens. Le projet Typannot vise à l’élaboration d’un système typographique de transcription des langues signées. Claire, Adrien et Morgane témoigneront de cette aventure commune et de leur parcours respectif.

(poster); International audience; Non-manual actions, and more specifically facial actions (FA) can be found in all Sign Languages (SL). Those actions involve all the different facial parts and can have various and intricate linguistic relations with manual signs. Unlike in vocal languages, FA in SL provide more meaning than simple expressions of feelings and emotions. Yet non-manual parameters are among the most unknown formal features in SL studies. During the past 30 years, some studies have started questioning the meanings and linguistic values and the relations between manual and non-manual signs (Crashborn et al. 2008; Crashborn & Bank 2014); more recently, SL corpora have been analysed, segmented, and transcribed to help study FA (Vogst-Svenden 2008; Bergman et al. 2008; Sutton-Spence & Day 2008).Moreover, to fill the lack of an annotation system for FA, a few manual annotation systems have integrated facial glyphs, such as HamNoSys (Prillwitz et al. 1989) and SignWriting (Sutton 1995). On one hand, HamNoSys has been developed to describe all existing SLs at a phonetic level; it allows a formal, linear, highly detailed and searchable description of manual parameters. As for non-manual parameters, HamNoSys offers the replacement of hands by another articulators. Non-manual parameters can be written as “eyes” or “mouth” and described by the same symbols developed for hands (Hanke 2004). Unfortunately only a limited number of manual symbols can be translated into FA and the annotation system remains incomplete. On the other hand, SignWriting describes SL with iconic symbols placed in a 2D space representing the signer’s body. Facial expressions are divided into mouth, eyes, nose, eyebrows, etc., and are drawn in a circular “head” much like emoticons. SignWriting offers a detailed description of posture and actions of non-manual parameters, but does not ensure compatibility with the most common annotation software used by SL linguists (e.g., ELAN).Typannot, a interdisciplinary project led by linguists, designers, and developers, which aims to set up a complete transcription system for SL that includes every SL parameter (handshape, localisation, movement, FA), has developed a different methodology. As mentioned earlier, FA have various linguistic values (mouthings, adverbial mouth gestures, semantically empty, enacting, whole face) and also include prosody and emotional meanings. In this regard, they can be more variable and signer-related than manual parameters. To offer the best annotation tool, Typannot approach has been to define facial parameters and all their possible tangible configurations. The goal is to set up the most efficient, simple, yet complete and universal formula to describe all possible FA.This formula is based on a 3-dimensional grid. Indeed all the different configurations of FA can be described by its X, Y, Z axis position. As a result, all FA can be described and encoded using a restricted list of 39 qualifiers. Based on this model and to help reduce the annotation process, a set of generic glyphs has been developed. Each qualifier has its own symbolic “generic” glyph. This methodical decomposition of all facial components enables a precise and accurate transcription of a complex FA using only a few glyphs.This formula and its generic glyphs have gone through a series of tests and revisions. Recently, an 18m20s long FA corpus of two deaf signers has been recorded using two different cameras. The first one, RGB HQ, is used to capture a high quality image and the second, infrared Kinect, is used to captured the depth. The latter was linked with Brekel Proface 2 (Leong et al. 2015), a 3D animation software that enables an automatic recognition of FA. This corpus has been fully annotated using Typannot generic glyphs. These annotations have enabled the validation of the general structure of Typannot FA formula and to identify some minor corrections to be made. For instance, it has been shown that the description of the air used to puff out or suck in cheeks is too restrictive and the description of the opening and closure of the eyelids is too unnecessarily precise.When those changes are implemented, our next task will be to develop a morphological glyphic system that will combine the different generic glyphs used for each facial parameter into one unique morphological glyph. This means that for any given FA, all the information contained in Typannot descriptive formula will be contained within one legible glyph. Some early research and work has already begun on this topic, but needs further development before providing a statement on its typographic structure. When this system is completed, it will be released with its own virtual keyboard (Typannot Keyboard, currently in development for handshapes) to help transcription and improve annotation processes.Bibliography:-Chételat-Pelé, E. (2010). Les Gestes Non Manuels en Langue des Signes Française ; Annotation, analyse et formalisation : application aux mouvements des sourcils et aux clignements des yeux. Université de Provence – Aix-Marseille I.-Crasborn, O., van der Kooij, E., Waters, D., Woll, B., & Mesch, J. (2008). Frequency distribution and spreading behavior of different types of mouth actions in three sign languages. Sign Language & Linguistics, 11(1), 45-67.-Crasborn, O.A., & Bank, R. (2014). An annotation scheme for the linguistic study of mouth actions in sign languages. http://repository.ubn.ru.nl/handle/2066/132960-Fontana, S. (2008). Mouth actions as gesture in sign language. Gesture, 8(1), 104-123.-Hanke, T. (2004). HamNoSys - Representing sign language data in language resources and language processing contexts. In Workshop on the Representation and Processing of Sign Languages on the occasion of the Fourth International Conference on Language Resources and Evaluation (p. 1-6).-Leong, C.W., Chen, L., Feng, G., Lee, C.M., & Mulholland, M. (2015). Utilizing depth sensors for analyzing multimodal presentations: Hardware, software and toolkits (p. 547-556). Proceedings of the 2015 ACM International Conference on Multimodal Interaction, ACM.-Prillwitz, S., Leven, R., Zienert, H., Hanke, T., & Henning, J. (1989). Hamburg notation system for sign languages: an introductory guide. Signum Press, Hamburg.-Sandler, W. (2009). Symbiotic symbolization by hand and mouth in sign language. Semiotica, 2009(174), 241-275. http://doi.org/10.1515/semi.2009.035-Sutton, V. (1995). Lessons in Sign Writing: textbook. DAC, La Jolla (CA).-Sutton-Spence, R., & Boyes-Braem, P. (2001). The hands are the head of the mouth: the mouth as articulator in sign languages. Signum Press, Hamburg

rapporto R01; The sign language transcription system "Typannot" is developed by the GestualScript research team. Its purpose is to transcribe the form of all sign languages in the world, taking care for all parameters. The requirement of transcription leads to consider that one must therefore develop a font by parameter. The foundations and characteristics of Typannot fonts are presented first in a deliberately practical manner in order to meet the operational requirements, which are covered by an annotation guide; the insertion and the use of these fonts in the ELAN software, main place used for the transcription of LS, are shown subsequently. This annotation guide was created during a specific state of development of Typannot fonts, and it is obviously going to evolve according to advanced fonts and the virtual keyboard; therefore, the guide will be updated regularly.; Le système de transcription des langues des signes Typannot est développé par l’équipe de recherche GestualScript. Il a pour vocation de transcrire la forme de l’ensemble des langues des signes dans le monde pour tous ses paramètres. L’exigence de la transcription amène à considérer qu’on doit élaborer donc une police de caractère par paramètre. Dans un premier temps, les fondements et les caractéristiques des polices Typannot sont présentés de manière volontairement pratique afin de répondre aux exigences opératoires qui relève d’un guide d’annotation. Puis, dans un deuxième temps, on montre l’insertion et l’utilisation de ces polices de caractères dans le logiciel ELAN, principal lieu utilisés pour la transcription des LS. Ce guide d’annotation a été constitué lors d’un état particulier de développement des polices de caractères Typannot, et il est évidemment amené à évoluer en fonction des avancés des polices de caractère et du clavier virtuel; le guide sera donc mis à jour régulièrement.; Il sistema di trascrizione della lingua dei segni Typannot viene sviluppato dal team di ricerca GestualScript. Il suo scopo è trascrivere la forma di tutte le lingue dei segni nel mondo per tutti i parametri. Il requisito della trascrizione porta a considerare che bisogna quindi sviluppare una font di caratteri per parametro. Inizialmente, le basi e le caratteristiche dei caratteri Typannot sono presentate in modo deliberatamente pratico per soddisfare i requisiti operativi che rientrano in una guida all'annotazione. Quindi, in una seconda fase, viene mostrato l'inserimento e l'uso di questi caratteri nel software ELAN, principale sito utilizzato per la trascrizione di LS. Questa guida all'annotazione è stata creata durante un momento puntuale di sviluppo dei font Typannot e si evolverà ovviamente in base agli avanzamenti dei caratteri e della tastiera virtuale; la guida sarà quindi aggiornata regolarmente.

The digital revolution has evolved the act of writing; its forms have changed. From this phenomenon, new Graphomotor oriented research opportunities have emerged. The knowledge gathered by this research introduces a new angle for setting up a new writing system. Until today, no writing system has been able to transcribe the multidimensional nature of sign languages (SL). For that reason, sign languages offer the perfect opportunity for this kind of research. In this study we try to understand the link between gestures and meaning for the speaker and discover what features and how much of signing (gestures, body language) can be kept in the act of writing. Our objective is to maintain the integral meaning of gestures for the signer/writer. To do so, we offer the creation of a technologically advanced scriptural environment in which meaningful gestures can be put into perspective. First, this multidisciplinary research focuses on what can be transferred from the former gestural act (signing) to the latter (writing). Then, we consider the tool that will enable this transfer. To do so, we follow a phenomenological approach, or in other terms, a descriptive methodology from the firstperson point of view. This methodology is built upon signers’ feedback gathered of the experience lived during interviews. Shaping this method to fit the French SL offers precise gestural descriptions from signers themselves. This database is then compared with alinguistic and kinesiological analysis from the third-person point of view. These gestural meaning results enable us to reflect on how to create a guided experience tool enabling the assimilation of SL’s gestural matter and the creation of scriptural forms. To do that, we follow a UX design, an enaction design, and a tool based approach in order to offer immersion and interaction. This kind of device offers a new perspective to signers on their own language and more generally, offers the possibility for any user to form a new relationship with her or his own gestures.; L’avènement du numérique a déplacé et découplé l’acte d’écriture de sa forme graphique et ainsi ouvert des possibilités de recherche au sujet des performances graphomotrices. Ces dernières peuvent, en particulier, prendre une autre dimension dans le contexte de création d’une écriture. Les langues des signes (LS) n’ont pas trouvé à ce jour de système d’écriture à même de rendre compte de leur spatialité et multidimensionnalité. Elles offrent ainsi des circonstances propices à cette recherche. Notre étude cherche à savoir jusqu’à quel point une part de l’activité du langage oral (gestuelle, corporelle) peut être maintenue dans l’activité du langage écrit et vise à comprendre le lien, en première personne, entre gestes et sens. L’objectif est de préserver une signification profonde pour le locuteur/scripteur. Pour répondre à cette question, nous proposons de réinvestir une gestuelle porteuse de sens dans le cadre de la conception d’un environnement technique favorisant la création scripturale. Dans un premier temps, cette étude pluridisciplinaire explore ce qui peut être transférable d’une sphère gestuelle à l’autre. En d’autres termes, quels éléments de l’oral peuvent être rapportés à l’écriture. Dans un second temps, elle envisage l’instrument permettant ce transfert. Pour cela, nous employons une démarche phénoménologique entendue comme méthodologie descriptive du point de vue en première personne. Cette méthode s’appuie sur des techniques de verbalisation de l’expérience vécue lors d’entretiens. La construction de la méthode adaptée à la LS française permet d’accéder à des descriptions fines des locuteurs sourds sur leur gestuelle. Le corpus de données est ensuite mis en dialogue avec une analyse en troisième personne établie à l’aide d’études linguistiques et kinésiologiques. Les résultats sur les dimensions du geste sémiotique nous amènent à penser les conditions d’une expérience habilitée, dans une perspective d’appropriation et de création scripturale des LS. Nous suivons pour cela la démarche de conception du design d’expérience utilisateur, du design d’énaction et de l’approche instrumentale pour l’immersion et l’interaction. La conception d’un tel dispositif vient non seulement changer le regard des personnes sourdes sur leur langue, mais également de manière plus générale, changer la relation que tout utilisateur a avec sa production gestuelle.

International audience

(exposition); International audience

International audience; There are two main systems of transcription for sign languages (SL): SignWriting [1,2] and HamNoSys [3]. Each of them have been designed for different yet complementary purposes. The first system is oriented towards communication, focusing on readability and using a strong logographic approach. The second one is oriented towards analysis, focusing on searchability and using a strong phonographic approach. Both transcribe the form of signs with a symbolic glyphic system, instead of glosses [4] using vocal glyphic systems. Each of them can be appreciated in their own use and purpose but this separation reveals the incapacity so far to combine both qualities in one system: phonographic searchability and logographic readability.Our aim is to solve this semiotic dichotomy by designing a new highly modular system that can answer the needs of all existing 142 SL [5] around the world (and virtually any SL) to transcribe the handshapes. The development and the use of this system relies on a new graphematic formula and modern typographic technologies such as Opentype and Robofont glyphs automatic generation capacities. The transcription, employing various digital interfaces (virtual keyboards and gestural interfaces), use a modular glyphic structure that allows to stack several levels of description (phonologic [6] and phonetic [7,8]). This system can be actually displayed in two forms, a phonological decomposed sequence (generic form) or a logographical assembled symbol (composed form) in a totally transparent way for the annotator via a universally available opentype functionality (contextual ligatures). The entire list of possible glyphs is very large (tens of thousands), so the keyboard allows to choose the handshapes present in a specific SL, constraining the list to the repertoire of each SL. Once the typographic font is installed on a computer, it is available in any software including ELAN or ANVIL. We developed a graphic visual interface and also a real time gestural recognition (LEAP motion MoCap device [10]) tool to facilitate the selection of the glyphs. The user will be able to actually annotate the signs by gesturing. In this communication, we will present how the font and the devices (virtual keyboard and MoCap interface) work together. Finally, the ratio between annotating time and duration of corpus is one of the well-known problems in annotating SL. We will discuss this ratio on French SL corpora, at different level (generic vs. composed glyphs) and with the said devices. Results make the case for the use of our annotating system including the MoCap input.Bibliography:[1] Sutton, V. (1995). Lessons in SignWriting: Textbook. DAC, LaJolla (CA, USA)[2] Bianchini, C.S. (2012). Analyse métalinguistique de l’émergence d’un système d’écriture des Langues des Signes : SignWriting et son application à la Langue des Signes Italienne (LIS). PhD dissertation (Univ. Paris8 & Univ. Studi Perugia).[3] Prillwitz, S., Leven, R., Zienert, H., Hanke, T., & Henning, J. (1989). Hamburg Notation System for sign languages: an introductory guide. Signum Press, Hamburg.[4] Johnston, T. (2008). Corpus linguistics and signed languages: no lemmata, no corpus. Proceedings of the 3rd Workshop on the Representation and Processing of Sign Languages (Marrakesh, Morocco):82-88.[5] Lewis, M.P. (2017). Ethnologue: languages of the world, vers. 20. SIL International. http://www.ethnologue.com [online; accessed: 1/12/2017]. [6] Eccarius, P. & Brentari, D. (2008). Handshape coding made easier: a theoretically based notation for phonological transcription. Sign Language & Linguistics, 11(1): 69-101.[7] Johnson, R. E. & Liddell, S. K. (2011). Toward a Phonetic Representation of Hand Configuration: the fingers. Sign Language Studies, 12(1):5-45.[8] Johnson, R. E. & Liddell, S. K. (2011). Toward a Phonetic Representation of Hand Configuration: the thumb. Sign Language Studies, 12(2):316-333. [9] Wikipedia (2017). OpenType. https://en.wikipedia.org/wiki/OpenType [online; accessed: 1/12/2017].[10] Wikipedia (2017). Leap Motion.. https://en.wikipedia.org/wiki/Leap_Motion [online; accessed: 1/12/2017].

see paper #020 (HAL‑02551372); National audience; Les 140 Langues des signes (LS) dans le monde n’ont pas de système de transcription efficient. Or comprendre une langue suppose d’être au plus près de ses formes. Typannot, en tant que système de transcription, propose une approche centrée sur les formes gestuelles et non pas fondée sur la seule traduction en français. Nous développons une série de polices de caractères et de descripteurs formels. En accès libre, ces polices permettent de transcrire tous les corpus des LS et aussi de découvrir les signes de manière ludique à travers le geste, grâce au programme interactif Typannot Signs.

see paper #21 (HAL‑02367761); International audience; Historically, transcriptions in SL have largely focused on relaying the meaning of a sign rather than the sign's physical shape [1,2]. Two main graphical systems of annotation —SignWriting [3] and HamNoSys [4]— are able to annotate the form of the parameters: the first one is not designed to annotate [5]; the second, in most cases, is coupled to a database tool for integrating SL corpus, named iLex [6]. Therefore, except for few corpora [7,8], the parameters are only partially annotated. One of the reasons is the time needed to do so, even if this has not been documented in literature. Another challenge in the transcription of signs is the integration of generic information (features) into the specificities of each parameter of signs.At a phonological level [9,10], the features extracted for handshapes generate multi-million possibilities. Some rules to reduce these potential combinations have been established [11,12], and others might still be found.The aim of this communication is to present Typannot [13,14], a type font for handshapes created to transcribe all the existing SL (142 SL for Ethnologue [15]). This glyphic type font is readable, writable, searchable.It works upon a modular system; the separate features can be assembled into glyphs that enable the representation of every handshape.The features are all searchable through a set of generic (symbolic) glyphs. Those glyphs can be visualized by the annotator at any time thanks to the use of ligatures (Opentype format [16]), and are therefore transparent. At present, this font —freely downloadable on every operating system— can be used with ELAN [17] to transcribe directly by using a virtual keyboard (MacOS) or, for some SL, by using a dedicated template.Apart from the keyboard and templates for ELAN, a third annotation device has been developed. It works with a motion capture (MoCap) device (based on Leap Motion [18]) by which the annotator can transcribe a corpus by directly using his own hand (real-time controlled). We are in the process of measuring the annotation time per minute of corpus for the different levels and devices: with generic glyphs, with specific glyphs and with MoCap. The tendency seems to favour of the use of the MoCap device.Finally, we will present some preliminary results of the transcription made with the so-called Typannot Typefont, at several levels of annotation: from the selection of the fingers, the shapes and the bending angles of the fingers in the palm, their lateral distance to the final glyph of each handshape through the combination shape+angle. Bibliography:[1] Johnston, T. (2008). Corpus linguistics and signed languages: no lemmata, no corpus. Proceedings of 3rd Workshop on the Representation and Processing of Sign Languages (p. 82-88). O. Crasborn, E. Efthimiou, T.s Hanke, E. D. Thoutenhoofd, I. Zwitserlood eds.[2] Fenlon, J., Schembri, A., Johnston, T., & Cormier, K. (2015). Documentary and corpus approaches to sign language research. Research Methods in Sign Language Studies: a Practical Guide, 156-172.[3] Sutton, V. (1995). Lessons in SignWriting: Textbook. SignWriting.[4] Prillwitz, S., Leven, R., Zienert, H., Hanke, T., & Henning, J. (1989). Hamburg Notation System for sign languages: an introductory guide (Signum Press). Hamburg.[5] Bianchini, C. S. (2012, november 18th). Analyse métalinguistique de l’émergence d’un système d’écriture des Langues des Signes : SignWriting et son application à la Langue des Signes Italienne (LIS). PhD dissertation, Paris 8 & Università di Perugia, Paris.[6] Hanke, T. (2002). iLex - A tool for sign language lexicography and corpus analysis (p. 923-926). Proceedings of the 3rd International Conference on Language Resources and Evaluation, Las Palma de Gran Canaria, Spain.[7] Efthimiou, E., Fontinea, S.E., Vogler, C., Hanke, T., Glauert, J., Bowden, R., Braffort, A., Collet, C., Maragos, P., Ségouat, J. (2010). Dicta-sign–sign language recognition, generation and modelling: a research effort with applications in deaf communication (p. 80-83). Proceedings of the 4th Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies, La Valletta (Malta), 22-23/05/2010.[8] Hanke, T., König, S., Konrad, R., & Langer, G. (2012, May). Towards tagging of multi-sign lexemes and other multi-unit structures. LREC 2012, Istanbul, Turkey.[9] Liddell, S. K. (1990). Structures for representing handshape and local movement at the phonemic level. Theoretical Issues in Sign Language Research, 1, 37-65.[10] Brentari, D. (1998). A prosodic model of sign language phonology. The MIT Press. Cambridge.[11] Ann, J. (1996). On the relation between ease of articulation and frequency of occurrence of handshapes in two sign languages. Lingua, 98(1), 19–41.[12] Brentari, D. (2012). Handshape in Sign Language phonology: the word and sub-lexical structure. In Sign language: an international handbook (Vol. 37). Walter de Gruyter.[13] Boutet, D., Bianchini, C. S., Doan, P., Goguely, T., Rébulard, M., &Danet, C. (2016, juillet). Typannot: a glyphic system for the transcription of handshapes. Colloque international présenté à 7th Conference of the International Society of Gestures Studies, Paris.[14] Boutet, D., Doan, P., Danet, C., Bianchini C. S., Goguely, T., Contesse, A., Rébulard, M., (Forhtcoming). Systèmes graphématiques et écritures des langues signées. Signata, 8. Presses universitaires de Liège.[15] Lewis, M.P. (2017). Ethnologue: languages of the world, vers. 20. SIL International. http://www.ethnologue.com [online; accessed: 1/12/2017][16] Wikipedia (2017). OpenType. https://en.wikipedia.org/wiki/OpenType [online; accessed: 1/12/2017].[17] Crasborn, O., & Sloetjes, H. (2008). Enhanced ELAN functionality for sign language corpora. Proceedings of 6th International Conference on Language Resources and Evaluation (Marrakesh, Morocco).[18] Wikipedia (2017). Leap Motion. https://en.wikipedia.org/wiki/Leap_Motion [online; accessed: 1/12/2017].

Scopus; International audience; Typannot is an innovative transcription system (TranSys) for Sign Languages (SLs), based on robust graphematic and coherent typographic formulas. It is characterized by readability, writability, searchability, genericity and modularity. Typannot can be used to record handshapes, mouth actions, facial expressions, initial locations (LOCini) and movements of the upper limbs (MOV). For LOCini and MOV, Typannot uses intrinsic frames of reference (iFoR) to describe the position of each segment (arm, forearm, hand) in terms of degrees of freedom (DoF). It assumes that the motion is subdivided into a complex moment of initial preparation, leading to the stabilization of a LOCini, and a subsequent phase of MOV deployment based on simple motor patterns. The goal of Typannot is not only to create a new TranSys, but also to provide an instrument to advance the knowledge about SLs. The observation of the SLs makes it possible to formulate various hypotheses, among which: 1) MOV follows a simple motor scheme that aims at minimizing motor control during MOV; 2) proximal→distal flows of MOV are predominant in SLs. Only the use of a TranSys based on iFoR and the description of the DoF makes it possible to explore the data in order to test these hypotheses.; Typannot est un système de transcription (TranSys) des Langues des Signes (LS) innovant, basé sur une formule graphématique robuste et une formule typographique cohérente. Il est lisible, scriptible, requêtable, générique et modulaire. Typannot peut être utilisé pour enregistrer les formes de la main, les actions de la bouche, les expressions faciales, les localisations initiales (LOCini) et les mouvements des membres supérieurs (MOV). Pour LOCini et MOV, Typannot utilise des cadres de référence intrinsèques (iFoR) pour décrire la position de chaque segment (bras, avant-bras, main) en termes de degrés de liberté (DoF). Il part de l’hypothèse que le mouvement est subdivisé en une phase complexe de préparation initiale qui mène à la stabilisation d’une LOCini et d’une phase de déploiement du MOV basée sur des schémas moteurs simples. Le but de Typannot est non seulement de créer un nouveau TranSys, mais aussi de fournir un instrument pour faire progresser les connaissances sur les LS. L'observation des LS permet de formuler différentes hypothèses, parmi lesquelles: i) MOV suit un schéma moteur simple qui vise à minimiser le contrôle moteur au cours du MOV; ii) les flux proximaux→distaux du MOV sont prédominants dans les LS. Seul l'utilisation d'un TranSys basé sur un iFoR et la description des DoF permettent d'explorer les données pour tester ces hypothèses.; Typannot è un innovativo sistema di trascrizione (TranSys) per le lingue dei segni (LS), basato su formule grafematiche robuste e formule tipografiche coerenti. Typannot è caratterizzato da leggibilità, scrivibilità, ricercabilità, genericità e modularità. Typannot può essere utilizzato per registrare le forme manuali, le azioni della bocca, le espressioni facciali, le locazioni iniziali (LOCini) ed i movimenti degli arti superiori (MOV). Per LOCini e MOV, Typannot usa contesti di riferimento intrinseci (iFoR) per descrivere la posizione di ciascun segmento (braccio, avambraccio, mano) in termini di gradi di libertà (DoF). Typannot presuppone che il movimento sia suddiviso in un momento complesso di preparazione iniziale, che porta alla stabilizzazione di una LOCini, ed una successiva fase di dispiegamento del MOV basata su schemi motori semplici. L'obiettivo di Typannot non è solo quello di creare un nuovo TranSys, ma anche di fornire uno strumento per far progredire la conoscenza delle SL. L'osservazione delle SL rende possibile formulare varie ipotesi, tra cui: i) MOV segue uno schema motorio semplice, che mira a minimizzare il controllo motorio durante MOV; ii) i flussi prossimali→distali di MOV sono predominanti nelle SL. Solo l'uso di un TranSys basato su un iFoR e la descrizione dei DoF consentono di esplorare i dati per testare queste ipotesi.

The 140 sign languages (SL) of the world do not have an efficient transcription system; now, understanding a language supposes to be as close as possible to its forms. Typannot, as a transcription system, proposes an approach centered on gestural forms and not based solely on translation into French; for that, a series of ad hoc fonts and formal descriptors have been developed. These open-access fonts make it possible to transcribe all SL corpora.; Les 140 langues des signes (LS) dans le monde n’ont pas de système de transcription efficient; or, comprendre une langue suppose d’être au plus près de ses formes. Typannot, en tant que système de transcription, propose une approche centrée sur les formes gestuelles et non pas fondée sur la seule traduction en français; pour ça, on a développé une série de polices de caractères et de descripteurs formels. En accès libre, ces polices permettent de transcrire tous les corpus des LS.; Le 140 lingue dei segni (LS) esistenti al mondo non hanno un sistema di trascrizione efficiente; ora, capire una lingua presuppone di essere il più vicino possibile alle sue forme. Typannot, come sistema di trascrizione, propone un approccio centrato sulle forme gestuali, e non basato quindi esclusivamente sulla traduzione in francese; per questo, sono stati sviluppati una serie di caratteri e descrittori formali. In accesso libero, questi caratteri consentono di trascrivere tutti i corpora di LS.

see publication #018 (HAL-2342251); International audience; Typannot is an innovative transcription system (TranSys) for Sign Languages (SLs), based on robust graphematic and coherent typographic formulas. It is characterized by readability, writability, seachability, genericity and modularity. Typannot can be used to record handshapes, mouth actions, facial expressions, initial locations (LOCini) and movements of the upper limbs (MOV). For LOCine and MOV, Typannot uses intrinsic frames of reference (iFoR) to describe the position of each segment (arm, forearm, hand) in terms of degrees of freedom (DoF). It assumes that the motion is subdivided into a complex moment of initial preparation, leading to the stabilization of a LOCini, and a subsequent phase of MOV deployment based on simple motor patterns. The goal of Typannot is not only to create a new TranSys, but also to provide an instrument to advance the knowledge about SLs. The observation of the SLs makes it possible to formulate various hypotheses, among which: i) MOV follows a simple motor scheme that aims at minimizing motor control during MOV; ii) proximal→distal flows of MOV are predominant in SLs. Only the use of a TranSys based on an iFoR and the description of the DoF makes it possible to explore the data in order to test these hypotheses.

Scholar; International audience; In the last years, many ways have been explored to write Sign Languages (LS) through the invention of notation systems. The presentation of three of them allows explaining certain choices, which may open the question of their taxonomic hybridization. The LS and their writings recall in their production the same articulators (upper body) and in their reception the same visual mode: these circumstances of unprecedented sharing put writing and orality co-existence, where scripturality may arise from formal and gestural intrinsic dimensions of orality and expression of LS. This potential is variously implemented by these writing systems, which raises questions about the implications of updating the writing systems to each other. Some systems, like SignWriting, try a pictorial analogy between the glyph and what it represents; others, like Typannot, tend to exploit the sign weaving as dynamic. The multilinearity of LS remains a challenge, due to the complexity of the registers to be represented. From these constraints, an overview of the writings of LS is presented, with an interdisciplinary approach that combines linguistics, typography and calligraphy.; Depuis quelques décennies, les langues des signes (LS) connaissent des tentatives d’écritures à travers l’invention de systèmes de notation. La présentation de trois d’entre eux permet d’expliquer certains choix à même d’ouvrir la question de l’hybridation taxonomique des systèmes scripturaux. Les LS et leurs écritures convoquent dans leurs productions les mêmes articulateurs (membre supérieur) et dans leurs réceptions la même modalité visuelle: ces circonstances inédites de partage placent l’écriture et l’oralité dans un rapport de co-existence où la scripturalité peut se manifester comme un jaillissement à partir des dimensions formelles et gestuelles intrinsèques de l’oralité et de l’expression des LS. Ce potentiel est saisi ou non par ces systèmes d’écritures, ce qui pose des questions quant aux implications d’une forme d’actualisation de l’écriture par rapport à une autre. Certains, comme SignWriting, recherchent une analogie imagique entre l’unité glyphique et ce qu’elle figure de la scène; d’autres, comme Typannot, tendent à exploiter le tracé du signe en tant que dynamique. La multilinéarité des LS reste un défi à cause de la complexité des registres à représenter. À partir de ces contraintes, un tableau général des écritures en LS est présenté dans une approche interdisciplinaire mêlant linguistique, typographie et calligraphie.; Da qualche decina d’anni, si sta tentando di scrivere le Lingue dei Segni (LS) tramite l'invenzione di sistemi di notazione. La presentazione di tre di loro permette di spiegare certe scelte capaci di aprire la questione della loro ibridazione tassonomica. Le LS e le loro scritture richiamano nella loro produzione gli stessi articolatori (estremità superiore) e la loro recezione nella medesima modalità visiva: queste circostanze inedite di condivisione pongono scrittura ed oralità in un rapporto di co-esistenza, dove la scritturalità può scaturire dalle dimensioni formali e gestuali intrinseche dell’oralità e dell'espressione di LS. Questo potenziale è variamente recepito da questi sistemi di scrittura, il che solleva domande circa le implicazioni dell’aggiornamento di una scrittura rispetto all’altra. Alcuni sistemi, come SignWriting, cercano un'analogia di immagine tra il glifo e ciò che rappresenta; altri, come Typannot, tendono a sfruttare la trama del segno come dinamica. La multilinearità delle LS resta una sfida a causa della complessità dei registri da rappresentare. Da questi vincoli, viene presentata una panoramica delle scritture di LS con un approccio interdisciplinare che combina linguistica, tipografia e calligrafia.

The 140 sign languages (SL) of the world do not have an efficient transcription system; now, understanding a language supposes to be as close as possible to its forms. Typannot, as a transcription system, proposes an approach centered on gestural forms and not based solely on translation into French; for that, a series of ad hoc fonts and formal descriptors have been developed. These open-access fonts make it possible to transcribe all SL corpora.; Les 140 langues des signes (LS) dans le monde n’ont pas de système de transcription efficient; or, comprendre une langue suppose d’être au plus près de ses formes. Typannot, en tant que système de transcription, propose une approche centrée sur les formes gestuelles et non pas fondée sur la seule traduction en français; pour ça, on a développé une série de polices de caractères et de descripteurs formels. En accès libre, ces polices permettent de transcrire tous les corpus des LS.; Le 140 lingue dei segni (LS) esistenti al mondo non hanno un sistema di trascrizione efficiente; ora, capire una lingua presuppone di essere il più vicino possibile alle sue forme. Typannot, come sistema di trascrizione, propone un approccio centrato sulle forme gestuali, e non basato quindi esclusivamente sulla traduzione in francese; per questo, sono stati sviluppati una serie di caratteri e descrittori formali. In accesso libero, questi caratteri consentono di trascrivere tutti i corpora di LS.

(poster); International audience; Descriptions and analysis for co-speech gestures and for sign languages belong to different traditions: from phonology for sign languages (Stokoe 1960; Brentari 1998; Johnson & Liddell 2010; 2011a; 2011b) and from ethnology and pragmatic fields for co-speech gestures (De Jorio 1832; Mallery 1881; Mauss 1934; Kendon 1988). Thus, the descriptive types are parametric for SL and functional for gestures.The comparison between co-speech gestures and sign languages goes through a common analysis, either functional or formal. Because of language-specific status of LS, on one hand, and co-speech status for gestures, on the other, a functional approach seems to be asymmetric. However, a formal approach -through a common transcript- allows to place SL and co-speech gestures on the same footing.Several formal transcription systems exist already. Developed for SL, the most used -HamNoSys (Prillwitz et al. 1989) and SignWriting (Sutton 1995)- do not offer the span of granularity which exists nevertheless between LS and co-speech gestures. Handshapes, very diversified in the case of LS are fewer and fuzzier for the gestures. We have designed a glyphic system for transcription (Typannot) which corresponds to this range of granularity. Currently, the handshapes glyphs span more than 230 handshapes (based on Eccarius & Brentari’s inventory, 2008) and potentially all SL handshapes around the world. Each glyph has a graphematic formula which describes all the features of the considered handshape. Each feature is designed by a generic glyph. These generics make up together, offering several levels of depiction. From the base till a complete handshape including feature compositions, the notation system Typannot allows to transcribe and to demand several granularities.We will expose the capacity of Typannot to transcribe and to compare several corpora in three different SLs as well as several co-speech gestures. The transcript concerns all the manual parameters (handshapes, orientation, location and movement). The location is specified for each segment (arm, forearm and hand). The way we conceive the movement is based on the categories of Johnson & Liddell (2011a; 2011b). The transcripts are done with ELAN software (Sloetjes & Wittenburg 2008). We investigate the internal formal structure of co-speech gestures and signs in a contrastive manner. The results show structural links between the parameters location and movement for sign languages and a looser link for co-speech gestures. A principle of motion economy rules a part of the parametric structure for signs. It correlates the location and the direction of the movement of the signs. These two parameters are deeply involved in the general shape of sign. This principle does not apply to the same extent to gestures, whose forms are more dependant of the concurrent speech.BIBLIOGRAPHYBrentari D. 1998. A prosodic model of sign language phonology. The MIT Press (Cambridge).De Jorio A. 1832. La mimica degli antichi investigata nel gestire napoletano. Stamperia del Fibreno: 380 p. [available on Google books]Eccarius P., Brentari D. 2008. Handshape coding made easier: a theoretically based notation for phonological transcription. Sign Language & Linguistics 11(1): 69-101.Johnson R.E. & S.K. Liddell. 2010. Toward a phonetic representation of signs: sequentiality and contrast. Sign Language Studies, 11(2): 241-274. doi.org/10.1353/sls.2010.0008Johnson R.E., Liddell S.K. 2011a. Toward a phonetic representation of hand configuration: the fingers. Sign Language Studies, 12(1): 5-45. doi.org/10.1353/sls.2011.0013Johnson R.E., Liddell S.K. 2011b. Toward a phonetic representation of hand configuration: the thumb. Sign Language Studies, 12(2): 316-333. doi.org/10.1353/sls.2011.0020Kendon A. 1988. How gestures can become like words. in: "Cross-cultural perspectives in nonverbal communication" (F. Poyato ed.). C.J. Hogrefe (Toronto): 131-141.Mallery G. 1881. Sign language among North American Indians. reprinted: 2010, Dover Publications (Mineola NY)Mauss M. 1934. Les techniques du corps. presentée au Congres de la Societé de Psychologie le 17 mai, publiée dans le Journal de Psychologie Normale et Pathologique, 32(3-4), version numerique par Université du Quebec en 2002.Prillwitz S., Leven R., Zienert H., Hanke T., Henning J. 1989. Hamburg notation system for sign languages: an introductory guide. Signum Press (Hamburg): 46 p.Sloetjes H., Wittenburg P. 2008. Annotation by category - ELAN and ISO DCR. in: Proc. 6th Intl Conf. Language Resources and Evaluation (LREC 2008).Stokoe W.C. 1960. Sign language structure: an outline of the visual communication systems of the American deaf. Studies in Linguistics, 8 (occasional papers). reprinted in: 2005. Journal of Deaf Studies and Deaf Education, 10(1): 3-37.Sutton V. 1995. Lessons in SignWriting: textbook & workbook. Deaf Action Committee for Sign Writing (La Jolla CA).

(poster); International audience; Descriptions and analysis for co-speech gestures and for sign languages belong to different traditions: from phonology for sign languages (Stokoe 1960; Brentari 1998; Johnson & Liddell 2010; 2011a; 2011b) and from ethnology and pragmatic fields for co-speech gestures (De Jorio 1832; Mallery 1881; Mauss 1934; Kendon 1988). Thus, the descriptive types are parametric for SL and functional for gestures.The comparison between co-speech gestures and sign languages goes through a common analysis, either functional or formal. Because of language-specific status of LS, on one hand, and co-speech status for gestures, on the other, a functional approach seems to be asymmetric. However, a formal approach -through a common transcript- allows to place SL and co-speech gestures on the same footing.Several formal transcription systems exist already. Developed for SL, the most used -HamNoSys (Prillwitz et al. 1989) and SignWriting (Sutton 1995)- do not offer the span of granularity which exists nevertheless between LS and co-speech gestures. Handshapes, very diversified in the case of LS are fewer and fuzzier for the gestures. We have designed a glyphic system for transcription (Typannot) which corresponds to this range of granularity. Currently, the handshapes glyphs span more than 230 handshapes (based on Eccarius & Brentari’s inventory, 2008) and potentially all SL handshapes around the world. Each glyph has a graphematic formula which describes all the features of the considered handshape. Each feature is designed by a generic glyph. These generics make up together, offering several levels of depiction. From the base till a complete handshape including feature compositions, the notation system Typannot allows to transcribe and to demand several granularities.We will expose the capacity of Typannot to transcribe and to compare several corpora in three different SLs as well as several co-speech gestures. The transcript concerns all the manual parameters (handshapes, orientation, location and movement). The location is specified for each segment (arm, forearm and hand). The way we conceive the movement is based on the categories of Johnson & Liddell (2011a; 2011b). The transcripts are done with ELAN software (Sloetjes & Wittenburg 2008). We investigate the internal formal structure of co-speech gestures and signs in a contrastive manner. The results show structural links between the parameters location and movement for sign languages and a looser link for co-speech gestures. A principle of motion economy rules a part of the parametric structure for signs. It correlates the location and the direction of the movement of the signs. These two parameters are deeply involved in the general shape of sign. This principle does not apply to the same extent to gestures, whose forms are more dependant of the concurrent speech.BIBLIOGRAPHYBrentari D. 1998. A prosodic model of sign language phonology. The MIT Press (Cambridge).De Jorio A. 1832. La mimica degli antichi investigata nel gestire napoletano. Stamperia del Fibreno: 380 p. [available on Google books]Eccarius P., Brentari D. 2008. Handshape coding made easier: a theoretically based notation for phonological transcription. Sign Language & Linguistics 11(1): 69-101.Johnson R.E. & S.K. Liddell. 2010. Toward a phonetic representation of signs: sequentiality and contrast. Sign Language Studies, 11(2): 241-274. doi.org/10.1353/sls.2010.0008Johnson R.E., Liddell S.K. 2011a. Toward a phonetic representation of hand configuration: the fingers. Sign Language Studies, 12(1): 5-45. doi.org/10.1353/sls.2011.0013Johnson R.E., Liddell S.K. 2011b. Toward a phonetic representation of hand configuration: the thumb. Sign Language Studies, 12(2): 316-333. doi.org/10.1353/sls.2011.0020Kendon A. 1988. How gestures can become like words. in: "Cross-cultural perspectives in nonverbal communication" (F. Poyato ed.). C.J. Hogrefe (Toronto): 131-141.Mallery G. 1881. Sign language among North American Indians. reprinted: 2010, Dover Publications (Mineola NY)Mauss M. 1934. Les techniques du corps. presentée au Congres de la Societé de Psychologie le 17 mai, publiée dans le Journal de Psychologie Normale et Pathologique, 32(3-4), version numerique par Université du Quebec en 2002.Prillwitz S., Leven R., Zienert H., Hanke T., Henning J. 1989. Hamburg notation system for sign languages: an introductory guide. Signum Press (Hamburg): 46 p.Sloetjes H., Wittenburg P. 2008. Annotation by category - ELAN and ISO DCR. in: Proc. 6th Intl Conf. Language Resources and Evaluation (LREC 2008).Stokoe W.C. 1960. Sign language structure: an outline of the visual communication systems of the American deaf. Studies in Linguistics, 8 (occasional papers). reprinted in: 2005. Journal of Deaf Studies and Deaf Education, 10(1): 3-37.Sutton V. 1995. Lessons in SignWriting: textbook & workbook. Deaf Action Committee for Sign Writing (La Jolla CA).

(poster exposition) - two venues: 13/05-06/08 (Chaumont) and 17/10-30/11/2017 (Nancy); International audience; Typannot est un système graphématique et typographique qui propose des polices de caractères, des règles de composition et une interface de saisie pour transcrire la langue des signes. Son originalité réside dans l’articulation de deux modes de représentation : un mode phonologique permettant une analyse fine, et un mode logographique permettant une lecture globale aisée.L’étude des langues des signes suppose de pouvoir analyser des enregistrements vidéo grâce à des transcriptions. N’ayant pas d’écriture, la transcription des langues des signes s’avère être une tâche complexe et difficile. Le projet Typannot a pour but de concevoir une suite d’outils typographiques et numériques permettant de transcrire graphiquement les différents paramètres composants la langue des signes.Cette recherche a vu le jour à l’École Supérieure d’Art et de Design d’Amiens en 2006 et s’est développée au fil des ans pour devenir une unité structurée et autonome qui travaille et échange avec d’autres institutions culturelles et scientifiques. Les questions d’écriture des langues des signes sont explorées au travers des projets Photocalli et Typannot grâce à l’emploi de technologies novatrices de captation du geste et de conception typographique. Ce projet réunit Dominique Boutet, Patrick Doan, Claire Danet, Claudia Savina Bianchini, Timothée Goguely, Adrien Contesse et Morgane Rébulard. Cette recherche est soutenue par le conseil régional de Picardie, la Délégation Générale à la Langue Française et aux Langues de France, le Crédit Agricole, la société éditrice du logiciel RoboFont ainsi que par le Ministère de la Culture.

International audience

International audience