Search

International audience; Cet article présente une expérimentation de traduction automatique de texte vers la langue des signes (LS). Comme nous ne disposons pas de corpus aligné de grande taille, nous avons exploré une approche à base d'exemples, utilisant AZee, une représentation intermédiaire du discours en LS sous la forme d'expressions hiérarchisées

International audience

International audience; This paper is a contribution to sign language (SL) modeling. We focus on the hitherto imprecise notion of "Multiplicity", assumed to express plurality in French Sign Language (LSF), using AZee approach. AZee is a linguistic and formal approach to modeling LSF. It takes into account the linguistic properties and specificities of LSF while respecting constraints linked to a modeling process. We present the methodology to extract AZee production rules. Based on the analysis of strong form-meaning associations in SL data (elicited image descriptions and short news), we identified two production rules structuring the expression of multiplicity in LSF. We explain how these newly extracted production rules are different from existing ones. Our goal is to refine the AZee approach to allow the coverage of a growing part of LSF. This work could lead to an improvement in SL synthesis and SL automatic translation.

International audience; In natural language settings, many interactions include more than two speakers, and real-life interpretation is based on all types of information available in all modalities. This constitutes a challenge for corpus-based analyses because the information in the audio and visual channels must be included in the coding. The goal of the DINLANG project is to tackle that challenge and analyze spontaneous interactions in family dinner settings (two adults and two to three children). The families use either French, or LSF (French sign language). Our aim is to compare how participants share language across the range of modalities found in vocal and visual languaging in coordination with dining. In order to pinpoint similarities and differences, we had to find a common coding tool for all situations (variations from one family to another) and modalities. Our coding procedure incorporates the use of the ELAN software. We created a template organized around participants, situations, and modalities, rather than around language forms. Spoken language transcription can be integrated, when it exists, but it is not mandatory. Data that has been created with another software can be injected in ELAN files if it is linked using time stamps. Analyses performed with the coded files rely on ELAN's structured search functionalities, which allow to achieve fine-grained temporal analyses and which can be completed by using spreadsheets or R language.

This article presents an original method for Text-to-Sign Translation. It compensates data scarcity using a domain-specific parallel corpus of alignments between text and hierarchical formal descriptions of Sign Language videos in AZee. Based on the detection of similarities present in the source text, the proposed algorithm recursively exploits matches and substitutions of aligned segments to build multiple candidate translations for a novel statement. This helps preserving Sign Language structures as much as possible before falling back on literal translations too quickly, in a generative way. The resulting translations are in the form of AZee expressions, designed to be used as input to avatar synthesis systems. We present a test set tailored to showcase its potential for expressiveness and generation of idiomatic target language, and observed limitations. This work finally opens prospects on how to evaluate translation and linguistic aspects, such as accuracy and grammatical fluency.

To date, comparisons between sign languages (SL) and co-speech gestures (CSG) still face major challenges.One of them is the difficulty to choose, among all the existing transcription systems (TranSys), a single TranSys that could allow researchers to take into account the peculiar characteristics of SL and CSG. Even if the researchers of these two fields describe the same “gesturing body”, they have developed their own representation models (with their own form and scope) to code the features they are interested in. In fact, all these systems adopt a perceptual point of view , i.e. a visuo-spatial representation of the gesture, and are therefore subject to different insight (e.g., in the division of space {1}). This can lead to a fragmentation of the descriptive modalities, limiting the identification of formal and functional common characteristics between SL and CVG. Doing so, researchers currently ignore a rallying perspective allowing to describe both SL and CSG: the body structure. To fill this gap, we have developed Typannot {2}, a typographic TranSys that aims at representing gestures from an articulatory standpoint, using a descriptive approach rooted in the body: the kinesiological approach {3}.Designed for SL but suitable for CSG too, Typannot is a formal TranSys that reproduces the articulatory characteristics of each segment of the body (fingers, hand, forearm, arm; shoulder, torso and hip; neck, head, mouth and eyes). It describes their possibilities of organization from the body articulatory perspective: the positions and the transformations of the degrees of freedom. In Typannot, each articulator (upper-limb, bust, mouth and eyes segments) is associated with a different OpenType font; every character (in Unicode sense) of a Typannot font contains only one specific information (i.e., the degree of extension or abduction) necessary to describe all the features of a segment. An advanced system of typographic ligature allows the user to see a single “holistic” glyph containing every feature, thus assuring searchability in a still readable way.Another challenge faced by LS and CSG TranSys is the time required to transcribe gestural phenomena, which drastically restricts the size of multimodal corpus and therefore the possibility of generalizations. Here again, Typannot deals with this issue by facilitating and automating the work. On one hand, the input interface has been designed to help transcribers produce transcription easily; on the other hand, the articulatory model used in this TranSys can easily be associated with Motion Capture (MoCap) data. This correspondence makes it possible to envisage a (semi)automatic transcription of body structure, drastically minimizing the transcription time.Typannot has been used within the ANR-LexiKHuM project (2021-2025) {4} to transcribe the body postures present in the multimodal and parallel corpus DEGELS1 {5}, consisting of conversations in LSF (≃ 30 min) and in spoken French (≃ 30 min). One of the objectives of LexiKHuM is to identify the epistemic markers common to SL and to CSG, in particular those that are not brachial. Thanks to Typannot, an articulatory description of the upper trunk and of the facial expressions could be implemented out in a coherent manner between the data in SL and CSG. The corpus was transcribed using the ELAN software {6} and the annotation was carried out manually and with the sensor-free MoCap software AlphaPose {7}, allowing a semi-automatic annotation in Typannot characters.In this talk, we will present the kinesiological approach and the Typannot typographic font system; some transcribed data and preliminary results of LexiKHuM will be exposed, also focusing on the time used for manual and semi-automatic transcription.******This presentation is dedicated to the memory of Dominique Boutet (1966-2020), who was co-author of our proposal in 2020 and co-coordinator of the Typannot and the LexiKHuM projects.Bibliography{1} McNeill, D. (1992). Hands and mind: what gestures reveal about thought. Chicago: University of Chicago Press.{2} Danet C., Boutet D., Doan P., Bianchini C.S., Contesse A., Chevrefils L., Rébulard M., Thomas C. & Dauphin J.-F. (2021). Transcribing sign languages with Typannot: a typographic system which retains and displays layers of information. Grapholinguistics and its Applications, 5(2), 1009–1037. https://doi.org/10.36824/2020-graf-dane{3} Boutet D. (2018). Pour une approche kinésiologique de la gestualité: synthèse. Habilitation à diriger des recherches, Université́ de Rouen-Normandie.{4} Catteau F., Morgenstern A. & Bianchini C.S. (2022). From gestures to kinesthetic modality: how to express epistemicity in a haptic device for human-machine interaction. 9th Intl Conf. of Intl. Soc. for Gesture Studies [ISGS 2022].{5} Braffort A. & Boutora L. (2012), DEGELS1: a comparable corpus of French Sign Language and co-speech gestures, Proc. 8th Intl. Conf. on Language Resources and Evaluation [LREC 2012].{6} Wittenburg P., Brugman H., Russel A., Klassmann A. & Sloetjes H. (2006). ELAN: a professional framework for multimodality research. Proc. 5th Intl. Conf. on Language Resources and Evaluation [LREC 2006].{7} Fang H.-S., Xie S., Tai Y.-W. & Lu C. (2018). RMPE: Regional Multi-person Pose Estimation. arXiv:161200137 [cs].

Research on sign languages (SLs) requires dedicated, efficient and comprehensive transcription systems to analyze and compare the sign parameters; at present, many transcription systems focus on manual parameters, relegating the non-manual component to a lesser role. This article presents Typannot, a formal transcription system, and in particular its application to mouth gestures: 1) first, exposing its kinesiological approach, i.e. an intrinsic articulatory description anchored in the body; 2) then, showing its conception to integrate linguistic, graphic and technical aspects within a typeface; 3) finally, presenting its application to a corpus in French Sign Language (LSF) recorded with motion capture., La recherche sur les langues des signes (LS) nécessite des systèmes de transcription dédiés, efficaces et complets pour analyser et comparer les paramètres des signes; à l'heure actuelle, de nombreux systèmes de transcription se concentrent sur les paramètres manuels, reléguant la composante non manuelle à un rôle moindre. Cet article présente Typannot, un système de transcription formelle, et en particulier son application aux gestes de la bouche: 1) d'abord, il expose son approche kinésiologique, c'est-à-dire une description articulatoire intrinsèque ancrée dans le corps; 2) puis, il montre sa conception pour intégrer les aspects linguistiques, graphiques et techniques au sein d'une police de caractères; 3) enfin, il présente son application à un corpus en Langue des Signes Française (LSF) enregistré en motion capture., La ricerca sulle lingue dei segni (SL) richiede sistemi di trascrizione dedicati, efficienti e completi per analizzare e confrontare i parametri dei segni; attualmente molti sistemi di trascrizione si concentrano sui parametri manuali, relegando la componente non manuale ad un ruolo minore. Questo articolo presenta Typannot, un sistema di trascrizione formale, e in particolare la sua applicazione ai gesti della bocca: 1) in primo luogo, esponendo il suo approccio kinesiologico, cioè una descrizione articolatoria intrinseca ancorata al corpo; 2) poi, mostrando la sua concezione per integrare gli aspetti linguistici, grafici e tecnici all'interno di un carattere tipografico; 3) infine, presentando la propria applicazione ad un corpus in Lingua dei Segni Francese (LSF) registrato con motion capture.

Facial actions are among the least documented parameters in sign languages (SL). Yet facial signs have a key role in SL, operating either as inseparable parts of a given manual sign or carrying its own meaning independently. One could argue that a major obstacle to this research is the difficulty related to its study. It is true that the scale at which facial actions occur is much smaller and the difference between facial actions is much finer. Eight parts (jaw, lips, eyebrow, etc.) divided between the upper face [Eye Action (EA)] and the lower face [Mouth Actions (MA)] are at play in facial actions. Most of these parts have a limited range of motion yet can communicate lots of information. Indeed, our brains have developed to recognize the most subtle changes in human faces (Ekman, 1984) as they carry crucial information for human interactions. If this has enabled us to implicitly perceive others' emotions, the conscious segmentation, organization, and analysis of such subtle movements remains a challenge.The existing research has mainly been focused on MA, as the lower part of the face plays a greater role in SL. Studies have brought to light the different linguistic values of MA, for example Mouthings, which are an articulation without voice of spoken words in the vocal language of the given country, and Mouth Gestures, which are mouth movements completely independent from vocal languages and can carry various types of values (adverbial, semantically empty, enacting, mouth activity in the context of the whole-face) (Fontana, 2008; Crasborn et al., 2008).These findings reveal the importance of MA in SL; yet, their study, difficult by nature, is made even more difficult by the lack of a complete and efficient transcription system. Several preliminary works (Bergman & Wallin, 2001; Sutton Spence & Day, 2001; Ajello et al., 2001) were led by a corpus driven approach resulting in different ways of categorizing mouth segments and positions. Some authors engineered a series of symbols to annotate a list of previously defined MA (HamNoSys of Prillwitz et al., 1989; Vogst-Svenden, 2001; Hohenberger & Happ, 2001). To date, the most developed tool available is SignWriting (Sutton, 1995), which offers 187 different symbols to transcribe MA. They can be compounded into an emoji like image. If the system offers an easy and intuitive way to symbolize facial expressions and emotions, it lacks granularity and fails to transcribe complex facial actions. Neither does it allow searchability as each composition results in an image from which no data can be extracted. Lastly, the symbols are based on the visible outer shapes of the face and not based on which facial parts were in action in order to create the facial expression. These features are setting limitations in the capacity to offer a deeper analysis of facial signs, of their values and of their interactions with other SL components.Typannot (Danet et al., 2020) is developing a complete typographic system to transcribe SL. For each pre defined SL parameter (handshape, initial location of upper limb, movement and facial actions), a dedicated type font, grouped in a complete type family, is currently being developed. All fonts within the Typannot system are based on the same structure. The fundamental principle upon which their graphematic formulas were designed is that the Typannot system transcribes which parts are in action in the making of a sign instead of transcribing the resulting image observed from an outer point of view.Each parameter font works using a two layer structure: the first displays all articulatory pieces of information required to make a sign [part(s) involved and value(s) of each part] in a string of individual characters; the second combines all pieces of information into a single morphological and intuitive to read glyph. This two layer system and the combination of multiple characters into glyphs is made possible through typographic engineering. To be fully efficient, these glyphs need to be legible and scriptable by all kinds of users, from beginners to experts. They also need to offer complete searchability and data mining by retaining every feature of every sign.In this presentation, Typannot graphematic formula using the XYZ axes as a frame of reference will be described. This spatial model allows the formula to be concise yet exhaustive in the description of any given mouth action. Then, the conception of the typographic system, from the initial sketches to the type design will be explained. Furthermore, the type engineering process enabling pieces of information from every part to be embedded into each of the Typannot glyphs will be demonstrated. Next, the digital interface currently being developed to allow complete accessibility and usability of the typographic system is presented. Lastly, questions about the possible evolution brought by the use of such a system in the analysis and understanding of MA, their roles and interactions within SL will be discussed.Bibliographical referencesAjello, R., Mazzoni, L., & Nicolai, F. (2001). Linguistic gestures: mouthing in Italian Sign Language. International studies on sign language and communication of the deaf, 39.Bergman, B., & Wallin, L. (2001). A preliminary analysis of visual mouth segments in Swedish Sign Language. International studies on sign language and communication of the deaf, 39.Crasborn, O., van der Kooij, E., Waters, D., Woll, B., & Mesch, J. (2008). Frequency and spreading behavior of different types of mouth actions in three sign languages. Sign Language & Linguistics, 11(1), 45-67.Danet, C., Boutet, D., Doan, P., Bianchini, C.S., & Contesse, A. (2020). Transcribing sign languages with TYPANNOT: the typographic system that retains and displays layers of information. Proceedings of Grapholinguistics in the 21st Century, 5, 1007-1035.Ekman, P. (1984). Expression and the nature of emotion. Approaches to Emotion, 3(19), 344.Fontana, S. (2008). Mouth actions as gesture in sign language. Gesture, 8(1), 104-123.Hohenberger, A., & Happ, D. (2001). The linguistic primacy of signs and mouth gestures over mouthings: evidence from language productions in German Sign Language. International studies on sign language and communication of the deaf, 39.Prillwitz, P., Leven, R., Zienert, H., Hamke, T., & Henning, J. (1989). HamNoSys version 2.0: Hamburg Notation System for Sign Languages: an introductory guide. International Studies on Sign Language and Communication of the Deaf, 5.Sutton, V. (1995). Lessons In SignWriting. The Deaf Action Committee for SignWriting.Sutton Spence, R., & Day, L. (2001). Mouthings and mouth gestures in British Sign Language. International studies on sign language and communication of the deaf, 39.Vogst Svenden, M. (2001). A comparison of mouth gestures and mouthings in Norwegian Sign Language. International studies on sign language and communication of the deaf, 39.

(poster); International audience; Two main issues are still at stake for the transcription of gestures (i.e., co speech gestures and SL signs): i/ the time used to transcribe; and ii/ a sole system for these two kinds of transcriptions (Slobin et al., 2001). The time needed to annotate gestural phenomena restricts drastically the size of the corpus and therefore the possible generalizations; on the other side, the lack of a common transcription system makes difficult any comparison across studies of sign or gesture forms (see the description of “pointing gestures with a functional point of view” [Kita, 2003] and of “buoys in SL” [Liddell, 2003]) and means of expression (for example, many co-speech gestures introduced in SL discourse). This fact increases the differences between a “genuine” linguistic production and the co-speech gestures (Schembri et al., 2005; Goldin-Meadow & Brentari, 2017) and sometimes even essentializing these differences (Singleton et al., 1995; McNeill, 2015).A formal transcription system designed for SL (developed by the Typannot project) - but available for co-speech gestures as well - depicts each segment (hand, forearm, arm and shoulder) of the upper limb, palm orientation, handshapes and non-manual parameters (including mouth actions, facial expressions, head and torso positions). Each articulator is associated to a different OpenType font (e.g., Typannot_HandShape font, Typannot_MouthAction font, etc.) belonging to a specific font family (Boutet et al., 2018). Every character (in Unicode sense) of a Typannot_font contains only one specific information (for example, fingers, shape, angle and closeness between fingers are provided for handshapes; Bianchini et al., 2018) necessary to describe all the features of an articulator. An advanced system of typographic ligature allows the user to see a single “holistic” glyph containing every feature, assuring searchability in a readable way.Because of their formal approach, using these Typannot_fonts allows: i/ a comparison between SL and gestures/mimics in multimodal discourses; and ii/ in SL, detailed sub-parameter queries, e.g., about the angle of a segment according to a peculiar degree of freedom (e.g., full extension of the hand) and up to the relations among segments - it is even possible to discriminate pointing forms, regardless of a finger. A corpus of multimodal French discourses and French Sign Language (LSF), of several minutes each, has been transcribed (with ELAN) using the Typannot_fonts in manual and semi-automatic ways. This last kind of transcription in Typannot_fonts is made filming the speaker wearing an IMU device, and processing the video with OpenFace (Baltrušaitis et al., 2018).In this talk, we will present this family fonts system, its structure, the annotated features, the possible queries made at two levels, the characters themselves and their ligatured glyphs. Some transcribed data and preliminary results will be exposed, focusing on the time used for transcription, by comparison of the ratio between manual and semi-automatic processing.We propose, as a side event to the conference, to set up a tutorial session (3 hours, small group) to learn how to use the Typannot transcription environment. BIBLIOGRAPHYBaltrušaitis T., Zadeh A., Chong Lim Y, Morency L-P. 2018. OpenFace 2.0: facial behavior analysis toolkit. 13th IEEE Intl Conf. Automatic Face & Gesture Recognition (FG 2018): 59-66. doi.org/10.1109/FG.2018.00019Bianchini C.S., Chèvrefils L., Danet C., Doan P., Rébulard M., Contesse A., Boutet D. 2018. Coding movement in sign languages: the Typannot approach. Proc. ACM 5th Intl Conf. Movement and Computing (MoCo'18), sect. 1(#9): 1-8.Boutet D., Doan P., Bianchini C.S., Danet C., Goguely T., Rébulard M. 2018. Systèmes graphématiques et écritures des langues signées. in “Signatures: (essais en) sémiotique de l’écriture” (J.M. Klinkenberg, S. Polis eds). Signata, 9: 391-426.Goldin-Meadow S., Brentari D. 2017. Gesture, sign and language: the coming of age of sign language and gesture studies. Behavioral and Brain Sciences, 40(e46): 1-82.Kita S. 2003. Pointing: a foundational building block of human communication. in "Pointing: where language, culture, and cognition meet" (S. Kita ed.). Erlbaum (Mahwah NJ): 1-8.Liddell S.K. 2003. Grammar, gesture, and meaning in American Sign Language. Cambridge University Press.McNeill D. (2015). Why we gesture. in: "Why we gesture: the surprising role of hand movements in communication". Cambridge University Press: 3-20. doi.org/10.1017/CBO9781316480526.002Schembri A., Jones C., Burnham D. 2005. Comparing action gestures and classifier verbs of motion: evidence from Australian Sign Language, Taiwan Sign Language, and nonsigners’ gestures without speech. Journal of Deaf Studies and Deaf Education, 10(3), 272-290. doi.org/10.1093/deafed/eni029Singleton J.L., Goldin-Meadow S., McNeill D. 1995. The cataclysmic break between gesticulation and sign: evidence against a unified continuum of gestural communication. in: "Language, gesture, and space" (K. Emmorey, J. Reilly eds). Hillsdale NJ: Lawrence Erlbaum Associates Publishers: 287-311.Slobin DI., Hoiting N., Anthony M., Biederman Y., Kuntze M., Lindert R., Pyers J., Thumann H., Weinberg A. 2001. Sign language transcription at the level of meaning components: the Berkeley Transcription System (BTS). Sign Language & Linguistics 4(1-2): 63-104. doi.org/10.1075/sll.4.12.07slo

see publication #024 - HAL‑02552935[conferenza annullata causa Covid]; International audience; The transcription of sign languages (SL) should take into account the gestural nature of these languages and, above all, why many typefont systems (e.g., HamNoSys, SignWriting) failed in establish themselves as a transcription tool (e.g., given the focus on the hands and the ego centered frame of reference). Gesturing sets all the segments of the upper limb in motion according to degrees of freedom, particular amplitudes and a series of intrinsic frames of reference centered on each segment. Typannot, the typefont system presented here, is based on these characteristics and integrates them into a structural hierarchy with an informational level corresponding to the features (characters), gathered in a level of glyphs composed. Typannot usability is ensured by respecting four design principles (genericity, modularity, legibility/readability and inscriptibility) and the use of a data entry interfaces managing these information levels. In order to illustrate the use of this font, analyses of the handshapes and proper location of the hand show the influence of practical gestures on symbolic gestures (i.e., SL signs). Considering the still cumbersome annotation time that such a transcription system imposes, the objective is to allow a “gestural dictation”, a direct transcription from motion capture data. This perspective should also facilitate the transcription of all the so called co verbal gestures of any vocal language.; Transcrire les langues des signes (LS) nécessite de prendre en considération leur nature gestuelle et de comprendre les raisons (parmi lesquelles centration sur les mains et utilisation d’un cadre de référence égocentré) pour lesquelles des systèmes typographiques (HamNoSys, SignWriting) ont échoué à s’imposer comme outil de transcription. La gestualité met en mouvement tous les segments du membre supérieur selon des degrés de liberté, en fonction d’amplitudes particulières et à travers une série de cadres de référence intrinsèque centrée sur chaque segment. Typannot, le système typographique présenté ici, repose sur ces caractéristiques et les intègre dans une hiérarchie structurelle ménageant un niveau informationnel correspondant aux traits (caractères), rassemblés dans un niveau de glyphes composés. La facilité d’utilisation de Typannot est assurée par le respect de quatre principes de conception (généricité, modularité, lisibilité et inscriptibilité) et l’utilisation d’une interface de saisie ménageant ces niveaux d’informations. Afin d’illustrer l’usage de Typannot, ont été menée des analyses portant sur les configurations et l’emplacement propre de la main, qui montrent l’influence de la gestualité praxique sur la gestualité symbolique (c.-à-d. les signes des LS). Vue la durée d’annotation encore très importante qu’un tel système de transcription impose, l’objectif est de permettre une "dictée gestuelle", une transcription directe à partir des données de capture de mouvement. Cette perspective devrait aussi faciliter la transcription de l’ensemble des gestes dits co-verbaux de n’importe quelle langue vocale.

International audience

Scholare-ISBN 2261-2424; International audience; The transcription of sign languages (SL) should take into account the gestural nature of these languages and, above all, why many typefont systems (e.g., HamNoSys, SignWriting) failed in establish themselves as a transcription tool (e.g., given the focus on the hands and the ego-centered frame of reference). Gesturing sets all the segments of the upper limb in motion according to degrees of freedom, particular amplitudes and a series of intrinsic frames of reference centered on each segment. Typannot, the typefont system presented here, is based on these characteristics and integrates them into a structural hierarchy with an informational level corresponding to the features (characters), gathered in a level of glyphs composed. Typannot usability is ensured by respecting four design principles (genericity, modularity, legibility/readability and inscriptibility) and the use of a data entry interfaces managing these information levels. In order to illustrate the use of this font, analyses of the handshapes and proper location of the hand show the influence of practical gestures on symbolic gestures (i.e., SL signs). Considering the still cumbersome annotation time that such a transcription system imposes, the objective is to allow a “gestural dictation”, a direct transcription from motion capture data. This perspective should also facilitate the transcription of all the so-called co-verbal gestures of any vocal language.; Transcrire les langues des signes (LS) nécessite de prendre en considération leur nature gestuelle et de comprendre les raisons (parmi lesquelles centration sur les mains et utilisation d'un cadre de référence égocentré) pour lesquelles des systèmes typographiques (HamNoSys, SignWriting) ont échoué à s'imposer comme outil de transcription. La gestualité met en mouvement tous les segments du membre supérieur selon des degrés de liberté, en fonction d'amplitudes particulières et à travers une série de cadres de référence intrinsèque centrée sur chaque segment. Typannot, le système typographique présenté ici, repose sur ces caractéristiques et les intègre dans une hiérarchie structurelle ménageant un niveau informationnel correspondant aux traits (caractères), rassemblés dans un niveau de glyphes composés. La facilité d'utilisation de Typannot est assurée par le respect de quatre principes de conception (généricité, modularité, lisibilité et inscriptibilité) et l'utilisation d'une interface de saisie ménageant ces niveaux d'informations. Afin d'illustrer l'usage de Typannot, ont été menée des analyses portant sur les configurations et l'emplacement propre de la main, qui montrent l'influence de la gestualité praxique sur la gestualité symbolique (c.-à-d. les signes des LS). Vue la durée d'annotation encore très importante qu'un tel système de transcription impose, l'objectif est de permettre une "dictée gestuelle", une transcription directe à partir des données de capture de mouvement. Cette perspective devrait aussi faciliter la transcription de l'ensemble des gestes dits co-verbaux de n'importe quelle langue vocale.; La trascrizione delle lingue dei segni (LS) richiede di considerare sia la loro natura gestuale che di comprendere le ragioni (incluso il centraggio sulle mani e l'uso di un quadro di riferimento egocentrico) per cui i sistemi tipografici (HamNoSys, SignWriting) non sono riusciti ad affermarsi come strumento di trascrizione. I gesti mettono in movimento tutti i segmenti dell'arto superiore in base ai gradi di libertà e le loro ampiezze particolari, tramite una serie di strutture di riferimento intrinseche, centrate su ciascun segmento. Typannot, il sistema tipografico qui presentato, si basa su queste caratteristiche e le integra in una gerarchia strutturale fornendo un livello di informazione corrispondente ai tratti (caratteri), raccolti in un livello di glifi composti. La facilità d'uso di Typannot è assicurata dal rispetto di quattro principi di progettazione (generalità, modularità, leggibilità e scrivibilità) e dall'uso di un'interfaccia di input che gestisce questi livelli di informazione. Al fine di illustrare l'uso di Typannot, sono state condotte analisi sulle configurazioni e sulla posizione corretta della mano, che mostrano l'influenza dei gesti concreti sui gesti simbolici (cioè i segni della LS). Dato il tempo di annotazione ancora molto importante richiesto da un tale sistema di trascrizione, l'obiettivo ulteriore è permettere una "dettatura gestuale", una trascrizione diretta dai dati di motion capture. Questa prospettiva dovrebbe anche facilitare la trascrizione di tutti i cosiddetti gesti co-verbali di qualsiasi linguaggio vocale.

International audience; There are more than 140 sign languages (SLs) in the world and studying them is a relatively recent field of research (starting in the 1960s). Linguists have the need to represent the different levels of gestures that make up the signs in order to analyze the way SLs work. Such transcription requires the use of a dedicated graphic system (Slobin et al. 2001). TYPANNOT, the transcription system presented in this article, is a typographic system that allows the description of all formal features of SLs. Our contribution to the field of grapholinguistics is a phonological model and a transcription system for SLs that are rooted in the articulatory possibilities of the signer's body. Compared to existing graphematic systems, our approach of SLs description is both phonological, allowing descriptions of the different articulatory structures (low level) involved in SLs, and logographical, allowing users to read the transcriptions from a unified perspective (high level). We will detail the design principles that drive the development of such a typographic system, the graphemic model that derives from linguistic study, and the tools that allow researchers to use TYPANNOT to its fullest capacities. This article also outlines the kinesiological approach (Boutet, 2018), which TYPANNOT uses, noting radical changes in the way researchers should look at meaning through gesture. This approach opens new perspectives in researching movement itself as a central source of meaning in human communication via gesture.; Il existe plus de 140 langues des signes (LS) dans le monde et leur étude est un domaine de recherche relativement récent. Les linguistes ont besoin de représenter les différents niveaux de gestes qui composent les signes afin d'analyser le fonctionnement des LS. Une telle transcription nécessite l'utilisation d'un système graphique dédié. Typannot, le système de transcription présenté dans cet article, est un système typographique qui permet la description de toutes les fonctionnalités formelles des LS: la contribution au domaine de la grapholinguistique est un modèle phonologique et un système de transcription pour les LS, les deux enracinés dans les possibilités articulatoires du corps du signeur. Par rapport aux systèmes graphématiques existants, l'approche à la description des LS est à la fois phonologique - permettant des descriptions des différentes structures articulatoires (de bas niveau) impliquées dans les LS -, et logographique - permettant de lire les transcriptions dans une perspective unifiée (de haut niveau). On détaille les principes de conception qui mènent au développement d'un tel système typographique, le modèle graphémique issu de l'étude linguistique et les outils qui permettent aux chercheurs d'utiliser Typannot à son maximum. Cet article décrit également l'approche kinésiologique que Typannot utilise, notant des changements radicaux dans la façon dont les chercheurs devraient étudier le sens par le geste. Cette approche ouvre de nouvelles perspectives dans la recherche du mouvement, lui-même considéré comme source centrale de sens dans la communication gestuelle.; Nel mondo esistono più di 140 lingue dei segni (LS), e studiarle è un campo di ricerca relativamente recente. I linguisti hanno la necessità di rappresentare i diversi livelli di gestualità che compongono i segni al fine di analizzare il modo in cui funzionano le LS. Tale trascrizione richiede l'utilizzo di un sistema grafico dedicato. Typannot, il sistema di trascrizione presentato in questo articolo, è un sistema tipografico che consente di descrivere tutte le caratteristiche formali delle LS: il contributo al campo della grafolinguistica è un modello fonologico e un sistema di trascrizione per LS, ambedue radicati nelle possibilità articolatorie del corpo del segnante. Rispetto ai sistemi grafematici esistenti, l'approccio alla descrizione delle LS è sia fonologico – descrivendo le strutture articolatorie (basso livello) coinvolte negli SL -, sia logografico - consentendo di leggere le trascrizioni in una prospettiva unificata (alto livello). Vengono di seguito descritti i principi progettuali che guidano lo sviluppo di Typannot, il modello grafico derivato dallo studio linguistico ed infine gli strumenti che consentono ai ricercatori il pieno utilizzo di Typannot. Questo articolo delinea anche l'approccio kinesiologico utilizzato in Typannot, che palesa la necessità di cambiamenti radicali nello studio dei significati attraverso i gesti. Questo approccio apre nuove prospettive nella ricerca sulla comunicazione gestuale, ponendo il movimento stesso come fonte centrale di significato.

(exposition); International audience; Cette recherche développe une nouvelle approche graphique basée sur les possibilités graphiques du geste en langue des signes (LS). En focalisant la réflexion sur le mouvement de la main, l'objet de la recherche est de comprendre comment le signeur s'en sert pour tracer un signe dans l'espace et ensuite le tracer sur le papier.Dans le cadre d'une recherche sur la création scripturale des langues des signes (LS), la thèse expose son engagement à prendre en considération le point de vue des locuteurs sourds. Ceci afin de faire émerger des formes graphiques de la LS, et plus particulièrement sur sa gestuelle.La démonstration, proposée sous une forme ludique, permet de montrer comment fonctionne Typannot : les symboles créés graphiquement représentent la forme de la main de la personne en train de signer.gestualscript.fr

International audience; GestualScript est une équipe de recherche transdisciplinaire fondée en 2008 qui travaille autour de l’écriture des langues des signes. Elle est intégrée au laboratoire Design-e de l’Ésad d’Amiens. Le projet Typannot vise à l’élaboration d’un système typographique de transcription des langues signées. Claire, Adrien et Morgane témoigneront de cette aventure commune et de leur parcours respectif.

(poster); International audience; Non-manual actions, and more specifically facial actions (FA) can be found in all Sign Languages (SL). Those actions involve all the different facial parts and can have various and intricate linguistic relations with manual signs. Unlike in vocal languages, FA in SL provide more meaning than simple expressions of feelings and emotions. Yet non-manual parameters are among the most unknown formal features in SL studies. During the past 30 years, some studies have started questioning the meanings and linguistic values and the relations between manual and non-manual signs (Crashborn et al. 2008; Crashborn & Bank 2014); more recently, SL corpora have been analysed, segmented, and transcribed to help study FA (Vogst-Svenden 2008; Bergman et al. 2008; Sutton-Spence & Day 2008).Moreover, to fill the lack of an annotation system for FA, a few manual annotation systems have integrated facial glyphs, such as HamNoSys (Prillwitz et al. 1989) and SignWriting (Sutton 1995). On one hand, HamNoSys has been developed to describe all existing SLs at a phonetic level; it allows a formal, linear, highly detailed and searchable description of manual parameters. As for non-manual parameters, HamNoSys offers the replacement of hands by another articulators. Non-manual parameters can be written as “eyes” or “mouth” and described by the same symbols developed for hands (Hanke 2004). Unfortunately only a limited number of manual symbols can be translated into FA and the annotation system remains incomplete. On the other hand, SignWriting describes SL with iconic symbols placed in a 2D space representing the signer’s body. Facial expressions are divided into mouth, eyes, nose, eyebrows, etc., and are drawn in a circular “head” much like emoticons. SignWriting offers a detailed description of posture and actions of non-manual parameters, but does not ensure compatibility with the most common annotation software used by SL linguists (e.g., ELAN).Typannot, a interdisciplinary project led by linguists, designers, and developers, which aims to set up a complete transcription system for SL that includes every SL parameter (handshape, localisation, movement, FA), has developed a different methodology. As mentioned earlier, FA have various linguistic values (mouthings, adverbial mouth gestures, semantically empty, enacting, whole face) and also include prosody and emotional meanings. In this regard, they can be more variable and signer-related than manual parameters. To offer the best annotation tool, Typannot approach has been to define facial parameters and all their possible tangible configurations. The goal is to set up the most efficient, simple, yet complete and universal formula to describe all possible FA.This formula is based on a 3-dimensional grid. Indeed all the different configurations of FA can be described by its X, Y, Z axis position. As a result, all FA can be described and encoded using a restricted list of 39 qualifiers. Based on this model and to help reduce the annotation process, a set of generic glyphs has been developed. Each qualifier has its own symbolic “generic” glyph. This methodical decomposition of all facial components enables a precise and accurate transcription of a complex FA using only a few glyphs.This formula and its generic glyphs have gone through a series of tests and revisions. Recently, an 18m20s long FA corpus of two deaf signers has been recorded using two different cameras. The first one, RGB HQ, is used to capture a high quality image and the second, infrared Kinect, is used to captured the depth. The latter was linked with Brekel Proface 2 (Leong et al. 2015), a 3D animation software that enables an automatic recognition of FA. This corpus has been fully annotated using Typannot generic glyphs. These annotations have enabled the validation of the general structure of Typannot FA formula and to identify some minor corrections to be made. For instance, it has been shown that the description of the air used to puff out or suck in cheeks is too restrictive and the description of the opening and closure of the eyelids is too unnecessarily precise.When those changes are implemented, our next task will be to develop a morphological glyphic system that will combine the different generic glyphs used for each facial parameter into one unique morphological glyph. This means that for any given FA, all the information contained in Typannot descriptive formula will be contained within one legible glyph. Some early research and work has already begun on this topic, but needs further development before providing a statement on its typographic structure. When this system is completed, it will be released with its own virtual keyboard (Typannot Keyboard, currently in development for handshapes) to help transcription and improve annotation processes.Bibliography:-Chételat-Pelé, E. (2010). Les Gestes Non Manuels en Langue des Signes Française ; Annotation, analyse et formalisation : application aux mouvements des sourcils et aux clignements des yeux. Université de Provence – Aix-Marseille I.-Crasborn, O., van der Kooij, E., Waters, D., Woll, B., & Mesch, J. (2008). Frequency distribution and spreading behavior of different types of mouth actions in three sign languages. Sign Language & Linguistics, 11(1), 45-67.-Crasborn, O.A., & Bank, R. (2014). An annotation scheme for the linguistic study of mouth actions in sign languages. http://repository.ubn.ru.nl/handle/2066/132960-Fontana, S. (2008). Mouth actions as gesture in sign language. Gesture, 8(1), 104-123.-Hanke, T. (2004). HamNoSys - Representing sign language data in language resources and language processing contexts. In Workshop on the Representation and Processing of Sign Languages on the occasion of the Fourth International Conference on Language Resources and Evaluation (p. 1-6).-Leong, C.W., Chen, L., Feng, G., Lee, C.M., & Mulholland, M. (2015). Utilizing depth sensors for analyzing multimodal presentations: Hardware, software and toolkits (p. 547-556). Proceedings of the 2015 ACM International Conference on Multimodal Interaction, ACM.-Prillwitz, S., Leven, R., Zienert, H., Hanke, T., & Henning, J. (1989). Hamburg notation system for sign languages: an introductory guide. Signum Press, Hamburg.-Sandler, W. (2009). Symbiotic symbolization by hand and mouth in sign language. Semiotica, 2009(174), 241-275. http://doi.org/10.1515/semi.2009.035-Sutton, V. (1995). Lessons in Sign Writing: textbook. DAC, La Jolla (CA).-Sutton-Spence, R., & Boyes-Braem, P. (2001). The hands are the head of the mouth: the mouth as articulator in sign languages. Signum Press, Hamburg

The digital revolution has evolved the act of writing; its forms have changed. From this phenomenon, new Graphomotor oriented research opportunities have emerged. The knowledge gathered by this research introduces a new angle for setting up a new writing system. Until today, no writing system has been able to transcribe the multidimensional nature of sign languages (SL). For that reason, sign languages offer the perfect opportunity for this kind of research. In this study we try to understand the link between gestures and meaning for the speaker and discover what features and how much of signing (gestures, body language) can be kept in the act of writing. Our objective is to maintain the integral meaning of gestures for the signer/writer. To do so, we offer the creation of a technologically advanced scriptural environment in which meaningful gestures can be put into perspective. First, this multidisciplinary research focuses on what can be transferred from the former gestural act (signing) to the latter (writing). Then, we consider the tool that will enable this transfer. To do so, we follow a phenomenological approach, or in other terms, a descriptive methodology from the firstperson point of view. This methodology is built upon signers’ feedback gathered of the experience lived during interviews. Shaping this method to fit the French SL offers precise gestural descriptions from signers themselves. This database is then compared with alinguistic and kinesiological analysis from the third-person point of view. These gestural meaning results enable us to reflect on how to create a guided experience tool enabling the assimilation of SL’s gestural matter and the creation of scriptural forms. To do that, we follow a UX design, an enaction design, and a tool based approach in order to offer immersion and interaction. This kind of device offers a new perspective to signers on their own language and more generally, offers the possibility for any user to form a new relationship with her or his own gestures.; L’avènement du numérique a déplacé et découplé l’acte d’écriture de sa forme graphique et ainsi ouvert des possibilités de recherche au sujet des performances graphomotrices. Ces dernières peuvent, en particulier, prendre une autre dimension dans le contexte de création d’une écriture. Les langues des signes (LS) n’ont pas trouvé à ce jour de système d’écriture à même de rendre compte de leur spatialité et multidimensionnalité. Elles offrent ainsi des circonstances propices à cette recherche. Notre étude cherche à savoir jusqu’à quel point une part de l’activité du langage oral (gestuelle, corporelle) peut être maintenue dans l’activité du langage écrit et vise à comprendre le lien, en première personne, entre gestes et sens. L’objectif est de préserver une signification profonde pour le locuteur/scripteur. Pour répondre à cette question, nous proposons de réinvestir une gestuelle porteuse de sens dans le cadre de la conception d’un environnement technique favorisant la création scripturale. Dans un premier temps, cette étude pluridisciplinaire explore ce qui peut être transférable d’une sphère gestuelle à l’autre. En d’autres termes, quels éléments de l’oral peuvent être rapportés à l’écriture. Dans un second temps, elle envisage l’instrument permettant ce transfert. Pour cela, nous employons une démarche phénoménologique entendue comme méthodologie descriptive du point de vue en première personne. Cette méthode s’appuie sur des techniques de verbalisation de l’expérience vécue lors d’entretiens. La construction de la méthode adaptée à la LS française permet d’accéder à des descriptions fines des locuteurs sourds sur leur gestuelle. Le corpus de données est ensuite mis en dialogue avec une analyse en troisième personne établie à l’aide d’études linguistiques et kinésiologiques. Les résultats sur les dimensions du geste sémiotique nous amènent à penser les conditions d’une expérience habilitée, dans une perspective d’appropriation et de création scripturale des LS. Nous suivons pour cela la démarche de conception du design d’expérience utilisateur, du design d’énaction et de l’approche instrumentale pour l’immersion et l’interaction. La conception d’un tel dispositif vient non seulement changer le regard des personnes sourdes sur leur langue, mais également de manière plus générale, changer la relation que tout utilisateur a avec sa production gestuelle.

International audience

(exposition); International audience

International audience; There are two main systems of transcription for sign languages (SL): SignWriting [1,2] and HamNoSys [3]. Each of them have been designed for different yet complementary purposes. The first system is oriented towards communication, focusing on readability and using a strong logographic approach. The second one is oriented towards analysis, focusing on searchability and using a strong phonographic approach. Both transcribe the form of signs with a symbolic glyphic system, instead of glosses [4] using vocal glyphic systems. Each of them can be appreciated in their own use and purpose but this separation reveals the incapacity so far to combine both qualities in one system: phonographic searchability and logographic readability.Our aim is to solve this semiotic dichotomy by designing a new highly modular system that can answer the needs of all existing 142 SL [5] around the world (and virtually any SL) to transcribe the handshapes. The development and the use of this system relies on a new graphematic formula and modern typographic technologies such as Opentype and Robofont glyphs automatic generation capacities. The transcription, employing various digital interfaces (virtual keyboards and gestural interfaces), use a modular glyphic structure that allows to stack several levels of description (phonologic [6] and phonetic [7,8]). This system can be actually displayed in two forms, a phonological decomposed sequence (generic form) or a logographical assembled symbol (composed form) in a totally transparent way for the annotator via a universally available opentype functionality (contextual ligatures). The entire list of possible glyphs is very large (tens of thousands), so the keyboard allows to choose the handshapes present in a specific SL, constraining the list to the repertoire of each SL. Once the typographic font is installed on a computer, it is available in any software including ELAN or ANVIL. We developed a graphic visual interface and also a real time gestural recognition (LEAP motion MoCap device [10]) tool to facilitate the selection of the glyphs. The user will be able to actually annotate the signs by gesturing. In this communication, we will present how the font and the devices (virtual keyboard and MoCap interface) work together. Finally, the ratio between annotating time and duration of corpus is one of the well-known problems in annotating SL. We will discuss this ratio on French SL corpora, at different level (generic vs. composed glyphs) and with the said devices. Results make the case for the use of our annotating system including the MoCap input.Bibliography:[1] Sutton, V. (1995). Lessons in SignWriting: Textbook. DAC, LaJolla (CA, USA)[2] Bianchini, C.S. (2012). Analyse métalinguistique de l’émergence d’un système d’écriture des Langues des Signes : SignWriting et son application à la Langue des Signes Italienne (LIS). PhD dissertation (Univ. Paris8 & Univ. Studi Perugia).[3] Prillwitz, S., Leven, R., Zienert, H., Hanke, T., & Henning, J. (1989). Hamburg Notation System for sign languages: an introductory guide. Signum Press, Hamburg.[4] Johnston, T. (2008). Corpus linguistics and signed languages: no lemmata, no corpus. Proceedings of the 3rd Workshop on the Representation and Processing of Sign Languages (Marrakesh, Morocco):82-88.[5] Lewis, M.P. (2017). Ethnologue: languages of the world, vers. 20. SIL International. http://www.ethnologue.com [online; accessed: 1/12/2017]. [6] Eccarius, P. & Brentari, D. (2008). Handshape coding made easier: a theoretically based notation for phonological transcription. Sign Language & Linguistics, 11(1): 69-101.[7] Johnson, R. E. & Liddell, S. K. (2011). Toward a Phonetic Representation of Hand Configuration: the fingers. Sign Language Studies, 12(1):5-45.[8] Johnson, R. E. & Liddell, S. K. (2011). Toward a Phonetic Representation of Hand Configuration: the thumb. Sign Language Studies, 12(2):316-333. [9] Wikipedia (2017). OpenType. https://en.wikipedia.org/wiki/OpenType [online; accessed: 1/12/2017].[10] Wikipedia (2017). Leap Motion.. https://en.wikipedia.org/wiki/Leap_Motion [online; accessed: 1/12/2017].

see paper #020 (HAL‑02551372); National audience; Les 140 Langues des signes (LS) dans le monde n’ont pas de système de transcription efficient. Or comprendre une langue suppose d’être au plus près de ses formes. Typannot, en tant que système de transcription, propose une approche centrée sur les formes gestuelles et non pas fondée sur la seule traduction en français. Nous développons une série de polices de caractères et de descripteurs formels. En accès libre, ces polices permettent de transcrire tous les corpus des LS et aussi de découvrir les signes de manière ludique à travers le geste, grâce au programme interactif Typannot Signs.

see paper #21 (HAL‑02367761); International audience; Historically, transcriptions in SL have largely focused on relaying the meaning of a sign rather than the sign's physical shape [1,2]. Two main graphical systems of annotation —SignWriting [3] and HamNoSys [4]— are able to annotate the form of the parameters: the first one is not designed to annotate [5]; the second, in most cases, is coupled to a database tool for integrating SL corpus, named iLex [6]. Therefore, except for few corpora [7,8], the parameters are only partially annotated. One of the reasons is the time needed to do so, even if this has not been documented in literature. Another challenge in the transcription of signs is the integration of generic information (features) into the specificities of each parameter of signs.At a phonological level [9,10], the features extracted for handshapes generate multi-million possibilities. Some rules to reduce these potential combinations have been established [11,12], and others might still be found.The aim of this communication is to present Typannot [13,14], a type font for handshapes created to transcribe all the existing SL (142 SL for Ethnologue [15]). This glyphic type font is readable, writable, searchable.It works upon a modular system; the separate features can be assembled into glyphs that enable the representation of every handshape.The features are all searchable through a set of generic (symbolic) glyphs. Those glyphs can be visualized by the annotator at any time thanks to the use of ligatures (Opentype format [16]), and are therefore transparent. At present, this font —freely downloadable on every operating system— can be used with ELAN [17] to transcribe directly by using a virtual keyboard (MacOS) or, for some SL, by using a dedicated template.Apart from the keyboard and templates for ELAN, a third annotation device has been developed. It works with a motion capture (MoCap) device (based on Leap Motion [18]) by which the annotator can transcribe a corpus by directly using his own hand (real-time controlled). We are in the process of measuring the annotation time per minute of corpus for the different levels and devices: with generic glyphs, with specific glyphs and with MoCap. The tendency seems to favour of the use of the MoCap device.Finally, we will present some preliminary results of the transcription made with the so-called Typannot Typefont, at several levels of annotation: from the selection of the fingers, the shapes and the bending angles of the fingers in the palm, their lateral distance to the final glyph of each handshape through the combination shape+angle. Bibliography:[1] Johnston, T. (2008). Corpus linguistics and signed languages: no lemmata, no corpus. Proceedings of 3rd Workshop on the Representation and Processing of Sign Languages (p. 82-88). O. Crasborn, E. Efthimiou, T.s Hanke, E. D. Thoutenhoofd, I. Zwitserlood eds.[2] Fenlon, J., Schembri, A., Johnston, T., & Cormier, K. (2015). Documentary and corpus approaches to sign language research. Research Methods in Sign Language Studies: a Practical Guide, 156-172.[3] Sutton, V. (1995). Lessons in SignWriting: Textbook. SignWriting.[4] Prillwitz, S., Leven, R., Zienert, H., Hanke, T., & Henning, J. (1989). Hamburg Notation System for sign languages: an introductory guide (Signum Press). Hamburg.[5] Bianchini, C. S. (2012, november 18th). Analyse métalinguistique de l’émergence d’un système d’écriture des Langues des Signes : SignWriting et son application à la Langue des Signes Italienne (LIS). PhD dissertation, Paris 8 & Università di Perugia, Paris.[6] Hanke, T. (2002). iLex - A tool for sign language lexicography and corpus analysis (p. 923-926). Proceedings of the 3rd International Conference on Language Resources and Evaluation, Las Palma de Gran Canaria, Spain.[7] Efthimiou, E., Fontinea, S.E., Vogler, C., Hanke, T., Glauert, J., Bowden, R., Braffort, A., Collet, C., Maragos, P., Ségouat, J. (2010). Dicta-sign–sign language recognition, generation and modelling: a research effort with applications in deaf communication (p. 80-83). Proceedings of the 4th Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies, La Valletta (Malta), 22-23/05/2010.[8] Hanke, T., König, S., Konrad, R., & Langer, G. (2012, May). Towards tagging of multi-sign lexemes and other multi-unit structures. LREC 2012, Istanbul, Turkey.[9] Liddell, S. K. (1990). Structures for representing handshape and local movement at the phonemic level. Theoretical Issues in Sign Language Research, 1, 37-65.[10] Brentari, D. (1998). A prosodic model of sign language phonology. The MIT Press. Cambridge.[11] Ann, J. (1996). On the relation between ease of articulation and frequency of occurrence of handshapes in two sign languages. Lingua, 98(1), 19–41.[12] Brentari, D. (2012). Handshape in Sign Language phonology: the word and sub-lexical structure. In Sign language: an international handbook (Vol. 37). Walter de Gruyter.[13] Boutet, D., Bianchini, C. S., Doan, P., Goguely, T., Rébulard, M., &Danet, C. (2016, juillet). Typannot: a glyphic system for the transcription of handshapes. Colloque international présenté à 7th Conference of the International Society of Gestures Studies, Paris.[14] Boutet, D., Doan, P., Danet, C., Bianchini C. S., Goguely, T., Contesse, A., Rébulard, M., (Forhtcoming). Systèmes graphématiques et écritures des langues signées. Signata, 8. Presses universitaires de Liège.[15] Lewis, M.P. (2017). Ethnologue: languages of the world, vers. 20. SIL International. http://www.ethnologue.com [online; accessed: 1/12/2017][16] Wikipedia (2017). OpenType. https://en.wikipedia.org/wiki/OpenType [online; accessed: 1/12/2017].[17] Crasborn, O., & Sloetjes, H. (2008). Enhanced ELAN functionality for sign language corpora. Proceedings of 6th International Conference on Language Resources and Evaluation (Marrakesh, Morocco).[18] Wikipedia (2017). Leap Motion. https://en.wikipedia.org/wiki/Leap_Motion [online; accessed: 1/12/2017].

Scopus; International audience; Typannot is an innovative transcription system (TranSys) for Sign Languages (SLs), based on robust graphematic and coherent typographic formulas. It is characterized by readability, writability, searchability, genericity and modularity. Typannot can be used to record handshapes, mouth actions, facial expressions, initial locations (LOCini) and movements of the upper limbs (MOV). For LOCini and MOV, Typannot uses intrinsic frames of reference (iFoR) to describe the position of each segment (arm, forearm, hand) in terms of degrees of freedom (DoF). It assumes that the motion is subdivided into a complex moment of initial preparation, leading to the stabilization of a LOCini, and a subsequent phase of MOV deployment based on simple motor patterns. The goal of Typannot is not only to create a new TranSys, but also to provide an instrument to advance the knowledge about SLs. The observation of the SLs makes it possible to formulate various hypotheses, among which: 1) MOV follows a simple motor scheme that aims at minimizing motor control during MOV; 2) proximal→distal flows of MOV are predominant in SLs. Only the use of a TranSys based on iFoR and the description of the DoF makes it possible to explore the data in order to test these hypotheses.; Typannot est un système de transcription (TranSys) des Langues des Signes (LS) innovant, basé sur une formule graphématique robuste et une formule typographique cohérente. Il est lisible, scriptible, requêtable, générique et modulaire. Typannot peut être utilisé pour enregistrer les formes de la main, les actions de la bouche, les expressions faciales, les localisations initiales (LOCini) et les mouvements des membres supérieurs (MOV). Pour LOCini et MOV, Typannot utilise des cadres de référence intrinsèques (iFoR) pour décrire la position de chaque segment (bras, avant-bras, main) en termes de degrés de liberté (DoF). Il part de l’hypothèse que le mouvement est subdivisé en une phase complexe de préparation initiale qui mène à la stabilisation d’une LOCini et d’une phase de déploiement du MOV basée sur des schémas moteurs simples. Le but de Typannot est non seulement de créer un nouveau TranSys, mais aussi de fournir un instrument pour faire progresser les connaissances sur les LS. L'observation des LS permet de formuler différentes hypothèses, parmi lesquelles: i) MOV suit un schéma moteur simple qui vise à minimiser le contrôle moteur au cours du MOV; ii) les flux proximaux→distaux du MOV sont prédominants dans les LS. Seul l'utilisation d'un TranSys basé sur un iFoR et la description des DoF permettent d'explorer les données pour tester ces hypothèses.; Typannot è un innovativo sistema di trascrizione (TranSys) per le lingue dei segni (LS), basato su formule grafematiche robuste e formule tipografiche coerenti. Typannot è caratterizzato da leggibilità, scrivibilità, ricercabilità, genericità e modularità. Typannot può essere utilizzato per registrare le forme manuali, le azioni della bocca, le espressioni facciali, le locazioni iniziali (LOCini) ed i movimenti degli arti superiori (MOV). Per LOCini e MOV, Typannot usa contesti di riferimento intrinseci (iFoR) per descrivere la posizione di ciascun segmento (braccio, avambraccio, mano) in termini di gradi di libertà (DoF). Typannot presuppone che il movimento sia suddiviso in un momento complesso di preparazione iniziale, che porta alla stabilizzazione di una LOCini, ed una successiva fase di dispiegamento del MOV basata su schemi motori semplici. L'obiettivo di Typannot non è solo quello di creare un nuovo TranSys, ma anche di fornire uno strumento per far progredire la conoscenza delle SL. L'osservazione delle SL rende possibile formulare varie ipotesi, tra cui: i) MOV segue uno schema motorio semplice, che mira a minimizzare il controllo motorio durante MOV; ii) i flussi prossimali→distali di MOV sono predominanti nelle SL. Solo l'uso di un TranSys basato su un iFoR e la descrizione dei DoF consentono di esplorare i dati per testare queste ipotesi.

The 140 sign languages (SL) of the world do not have an efficient transcription system; now, understanding a language supposes to be as close as possible to its forms. Typannot, as a transcription system, proposes an approach centered on gestural forms and not based solely on translation into French; for that, a series of ad hoc fonts and formal descriptors have been developed. These open-access fonts make it possible to transcribe all SL corpora.; Les 140 langues des signes (LS) dans le monde n’ont pas de système de transcription efficient; or, comprendre une langue suppose d’être au plus près de ses formes. Typannot, en tant que système de transcription, propose une approche centrée sur les formes gestuelles et non pas fondée sur la seule traduction en français; pour ça, on a développé une série de polices de caractères et de descripteurs formels. En accès libre, ces polices permettent de transcrire tous les corpus des LS.; Le 140 lingue dei segni (LS) esistenti al mondo non hanno un sistema di trascrizione efficiente; ora, capire una lingua presuppone di essere il più vicino possibile alle sue forme. Typannot, come sistema di trascrizione, propone un approccio centrato sulle forme gestuali, e non basato quindi esclusivamente sulla traduzione in francese; per questo, sono stati sviluppati una serie di caratteri e descrittori formali. In accesso libero, questi caratteri consentono di trascrivere tutti i corpora di LS.

see publication #018 (HAL-2342251); International audience; Typannot is an innovative transcription system (TranSys) for Sign Languages (SLs), based on robust graphematic and coherent typographic formulas. It is characterized by readability, writability, seachability, genericity and modularity. Typannot can be used to record handshapes, mouth actions, facial expressions, initial locations (LOCini) and movements of the upper limbs (MOV). For LOCine and MOV, Typannot uses intrinsic frames of reference (iFoR) to describe the position of each segment (arm, forearm, hand) in terms of degrees of freedom (DoF). It assumes that the motion is subdivided into a complex moment of initial preparation, leading to the stabilization of a LOCini, and a subsequent phase of MOV deployment based on simple motor patterns. The goal of Typannot is not only to create a new TranSys, but also to provide an instrument to advance the knowledge about SLs. The observation of the SLs makes it possible to formulate various hypotheses, among which: i) MOV follows a simple motor scheme that aims at minimizing motor control during MOV; ii) proximal→distal flows of MOV are predominant in SLs. Only the use of a TranSys based on an iFoR and the description of the DoF makes it possible to explore the data in order to test these hypotheses.

Scholar; International audience; In the last years, many ways have been explored to write Sign Languages (LS) through the invention of notation systems. The presentation of three of them allows explaining certain choices, which may open the question of their taxonomic hybridization. The LS and their writings recall in their production the same articulators (upper body) and in their reception the same visual mode: these circumstances of unprecedented sharing put writing and orality co-existence, where scripturality may arise from formal and gestural intrinsic dimensions of orality and expression of LS. This potential is variously implemented by these writing systems, which raises questions about the implications of updating the writing systems to each other. Some systems, like SignWriting, try a pictorial analogy between the glyph and what it represents; others, like Typannot, tend to exploit the sign weaving as dynamic. The multilinearity of LS remains a challenge, due to the complexity of the registers to be represented. From these constraints, an overview of the writings of LS is presented, with an interdisciplinary approach that combines linguistics, typography and calligraphy.; Depuis quelques décennies, les langues des signes (LS) connaissent des tentatives d’écritures à travers l’invention de systèmes de notation. La présentation de trois d’entre eux permet d’expliquer certains choix à même d’ouvrir la question de l’hybridation taxonomique des systèmes scripturaux. Les LS et leurs écritures convoquent dans leurs productions les mêmes articulateurs (membre supérieur) et dans leurs réceptions la même modalité visuelle: ces circonstances inédites de partage placent l’écriture et l’oralité dans un rapport de co-existence où la scripturalité peut se manifester comme un jaillissement à partir des dimensions formelles et gestuelles intrinsèques de l’oralité et de l’expression des LS. Ce potentiel est saisi ou non par ces systèmes d’écritures, ce qui pose des questions quant aux implications d’une forme d’actualisation de l’écriture par rapport à une autre. Certains, comme SignWriting, recherchent une analogie imagique entre l’unité glyphique et ce qu’elle figure de la scène; d’autres, comme Typannot, tendent à exploiter le tracé du signe en tant que dynamique. La multilinéarité des LS reste un défi à cause de la complexité des registres à représenter. À partir de ces contraintes, un tableau général des écritures en LS est présenté dans une approche interdisciplinaire mêlant linguistique, typographie et calligraphie.; Da qualche decina d’anni, si sta tentando di scrivere le Lingue dei Segni (LS) tramite l'invenzione di sistemi di notazione. La presentazione di tre di loro permette di spiegare certe scelte capaci di aprire la questione della loro ibridazione tassonomica. Le LS e le loro scritture richiamano nella loro produzione gli stessi articolatori (estremità superiore) e la loro recezione nella medesima modalità visiva: queste circostanze inedite di condivisione pongono scrittura ed oralità in un rapporto di co-esistenza, dove la scritturalità può scaturire dalle dimensioni formali e gestuali intrinseche dell’oralità e dell'espressione di LS. Questo potenziale è variamente recepito da questi sistemi di scrittura, il che solleva domande circa le implicazioni dell’aggiornamento di una scrittura rispetto all’altra. Alcuni sistemi, come SignWriting, cercano un'analogia di immagine tra il glifo e ciò che rappresenta; altri, come Typannot, tendono a sfruttare la trama del segno come dinamica. La multilinearità delle LS resta una sfida a causa della complessità dei registri da rappresentare. Da questi vincoli, viene presentata una panoramica delle scritture di LS con un approccio interdisciplinare che combina linguistica, tipografia e calligrafia.

The 140 sign languages (SL) of the world do not have an efficient transcription system; now, understanding a language supposes to be as close as possible to its forms. Typannot, as a transcription system, proposes an approach centered on gestural forms and not based solely on translation into French; for that, a series of ad hoc fonts and formal descriptors have been developed. These open-access fonts make it possible to transcribe all SL corpora.; Les 140 langues des signes (LS) dans le monde n’ont pas de système de transcription efficient; or, comprendre une langue suppose d’être au plus près de ses formes. Typannot, en tant que système de transcription, propose une approche centrée sur les formes gestuelles et non pas fondée sur la seule traduction en français; pour ça, on a développé une série de polices de caractères et de descripteurs formels. En accès libre, ces polices permettent de transcrire tous les corpus des LS.; Le 140 lingue dei segni (LS) esistenti al mondo non hanno un sistema di trascrizione efficiente; ora, capire una lingua presuppone di essere il più vicino possibile alle sue forme. Typannot, come sistema di trascrizione, propone un approccio centrato sulle forme gestuali, e non basato quindi esclusivamente sulla traduzione in francese; per questo, sono stati sviluppati una serie di caratteri e descrittori formali. In accesso libero, questi caratteri consentono di trascrivere tutti i corpora di LS.

(poster); International audience; Descriptions and analysis for co-speech gestures and for sign languages belong to different traditions: from phonology for sign languages (Stokoe 1960; Brentari 1998; Johnson & Liddell 2010; 2011a; 2011b) and from ethnology and pragmatic fields for co-speech gestures (De Jorio 1832; Mallery 1881; Mauss 1934; Kendon 1988). Thus, the descriptive types are parametric for SL and functional for gestures.The comparison between co-speech gestures and sign languages goes through a common analysis, either functional or formal. Because of language-specific status of LS, on one hand, and co-speech status for gestures, on the other, a functional approach seems to be asymmetric. However, a formal approach -through a common transcript- allows to place SL and co-speech gestures on the same footing.Several formal transcription systems exist already. Developed for SL, the most used -HamNoSys (Prillwitz et al. 1989) and SignWriting (Sutton 1995)- do not offer the span of granularity which exists nevertheless between LS and co-speech gestures. Handshapes, very diversified in the case of LS are fewer and fuzzier for the gestures. We have designed a glyphic system for transcription (Typannot) which corresponds to this range of granularity. Currently, the handshapes glyphs span more than 230 handshapes (based on Eccarius & Brentari’s inventory, 2008) and potentially all SL handshapes around the world. Each glyph has a graphematic formula which describes all the features of the considered handshape. Each feature is designed by a generic glyph. These generics make up together, offering several levels of depiction. From the base till a complete handshape including feature compositions, the notation system Typannot allows to transcribe and to demand several granularities.We will expose the capacity of Typannot to transcribe and to compare several corpora in three different SLs as well as several co-speech gestures. The transcript concerns all the manual parameters (handshapes, orientation, location and movement). The location is specified for each segment (arm, forearm and hand). The way we conceive the movement is based on the categories of Johnson & Liddell (2011a; 2011b). The transcripts are done with ELAN software (Sloetjes & Wittenburg 2008). We investigate the internal formal structure of co-speech gestures and signs in a contrastive manner. The results show structural links between the parameters location and movement for sign languages and a looser link for co-speech gestures. A principle of motion economy rules a part of the parametric structure for signs. It correlates the location and the direction of the movement of the signs. These two parameters are deeply involved in the general shape of sign. This principle does not apply to the same extent to gestures, whose forms are more dependant of the concurrent speech.BIBLIOGRAPHYBrentari D. 1998. A prosodic model of sign language phonology. The MIT Press (Cambridge).De Jorio A. 1832. La mimica degli antichi investigata nel gestire napoletano. Stamperia del Fibreno: 380 p. [available on Google books]Eccarius P., Brentari D. 2008. Handshape coding made easier: a theoretically based notation for phonological transcription. Sign Language & Linguistics 11(1): 69-101.Johnson R.E. & S.K. Liddell. 2010. Toward a phonetic representation of signs: sequentiality and contrast. Sign Language Studies, 11(2): 241-274. doi.org/10.1353/sls.2010.0008Johnson R.E., Liddell S.K. 2011a. Toward a phonetic representation of hand configuration: the fingers. Sign Language Studies, 12(1): 5-45. doi.org/10.1353/sls.2011.0013Johnson R.E., Liddell S.K. 2011b. Toward a phonetic representation of hand configuration: the thumb. Sign Language Studies, 12(2): 316-333. doi.org/10.1353/sls.2011.0020Kendon A. 1988. How gestures can become like words. in: "Cross-cultural perspectives in nonverbal communication" (F. Poyato ed.). C.J. Hogrefe (Toronto): 131-141.Mallery G. 1881. Sign language among North American Indians. reprinted: 2010, Dover Publications (Mineola NY)Mauss M. 1934. Les techniques du corps. presentée au Congres de la Societé de Psychologie le 17 mai, publiée dans le Journal de Psychologie Normale et Pathologique, 32(3-4), version numerique par Université du Quebec en 2002.Prillwitz S., Leven R., Zienert H., Hanke T., Henning J. 1989. Hamburg notation system for sign languages: an introductory guide. Signum Press (Hamburg): 46 p.Sloetjes H., Wittenburg P. 2008. Annotation by category - ELAN and ISO DCR. in: Proc. 6th Intl Conf. Language Resources and Evaluation (LREC 2008).Stokoe W.C. 1960. Sign language structure: an outline of the visual communication systems of the American deaf. Studies in Linguistics, 8 (occasional papers). reprinted in: 2005. Journal of Deaf Studies and Deaf Education, 10(1): 3-37.Sutton V. 1995. Lessons in SignWriting: textbook & workbook. Deaf Action Committee for Sign Writing (La Jolla CA).

(poster); International audience; Descriptions and analysis for co-speech gestures and for sign languages belong to different traditions: from phonology for sign languages (Stokoe 1960; Brentari 1998; Johnson & Liddell 2010; 2011a; 2011b) and from ethnology and pragmatic fields for co-speech gestures (De Jorio 1832; Mallery 1881; Mauss 1934; Kendon 1988). Thus, the descriptive types are parametric for SL and functional for gestures.The comparison between co-speech gestures and sign languages goes through a common analysis, either functional or formal. Because of language-specific status of LS, on one hand, and co-speech status for gestures, on the other, a functional approach seems to be asymmetric. However, a formal approach -through a common transcript- allows to place SL and co-speech gestures on the same footing.Several formal transcription systems exist already. Developed for SL, the most used -HamNoSys (Prillwitz et al. 1989) and SignWriting (Sutton 1995)- do not offer the span of granularity which exists nevertheless between LS and co-speech gestures. Handshapes, very diversified in the case of LS are fewer and fuzzier for the gestures. We have designed a glyphic system for transcription (Typannot) which corresponds to this range of granularity. Currently, the handshapes glyphs span more than 230 handshapes (based on Eccarius & Brentari’s inventory, 2008) and potentially all SL handshapes around the world. Each glyph has a graphematic formula which describes all the features of the considered handshape. Each feature is designed by a generic glyph. These generics make up together, offering several levels of depiction. From the base till a complete handshape including feature compositions, the notation system Typannot allows to transcribe and to demand several granularities.We will expose the capacity of Typannot to transcribe and to compare several corpora in three different SLs as well as several co-speech gestures. The transcript concerns all the manual parameters (handshapes, orientation, location and movement). The location is specified for each segment (arm, forearm and hand). The way we conceive the movement is based on the categories of Johnson & Liddell (2011a; 2011b). The transcripts are done with ELAN software (Sloetjes & Wittenburg 2008). We investigate the internal formal structure of co-speech gestures and signs in a contrastive manner. The results show structural links between the parameters location and movement for sign languages and a looser link for co-speech gestures. A principle of motion economy rules a part of the parametric structure for signs. It correlates the location and the direction of the movement of the signs. These two parameters are deeply involved in the general shape of sign. This principle does not apply to the same extent to gestures, whose forms are more dependant of the concurrent speech.BIBLIOGRAPHYBrentari D. 1998. A prosodic model of sign language phonology. The MIT Press (Cambridge).De Jorio A. 1832. La mimica degli antichi investigata nel gestire napoletano. Stamperia del Fibreno: 380 p. [available on Google books]Eccarius P., Brentari D. 2008. Handshape coding made easier: a theoretically based notation for phonological transcription. Sign Language & Linguistics 11(1): 69-101.Johnson R.E. & S.K. Liddell. 2010. Toward a phonetic representation of signs: sequentiality and contrast. Sign Language Studies, 11(2): 241-274. doi.org/10.1353/sls.2010.0008Johnson R.E., Liddell S.K. 2011a. Toward a phonetic representation of hand configuration: the fingers. Sign Language Studies, 12(1): 5-45. doi.org/10.1353/sls.2011.0013Johnson R.E., Liddell S.K. 2011b. Toward a phonetic representation of hand configuration: the thumb. Sign Language Studies, 12(2): 316-333. doi.org/10.1353/sls.2011.0020Kendon A. 1988. How gestures can become like words. in: "Cross-cultural perspectives in nonverbal communication" (F. Poyato ed.). C.J. Hogrefe (Toronto): 131-141.Mallery G. 1881. Sign language among North American Indians. reprinted: 2010, Dover Publications (Mineola NY)Mauss M. 1934. Les techniques du corps. presentée au Congres de la Societé de Psychologie le 17 mai, publiée dans le Journal de Psychologie Normale et Pathologique, 32(3-4), version numerique par Université du Quebec en 2002.Prillwitz S., Leven R., Zienert H., Hanke T., Henning J. 1989. Hamburg notation system for sign languages: an introductory guide. Signum Press (Hamburg): 46 p.Sloetjes H., Wittenburg P. 2008. Annotation by category - ELAN and ISO DCR. in: Proc. 6th Intl Conf. Language Resources and Evaluation (LREC 2008).Stokoe W.C. 1960. Sign language structure: an outline of the visual communication systems of the American deaf. Studies in Linguistics, 8 (occasional papers). reprinted in: 2005. Journal of Deaf Studies and Deaf Education, 10(1): 3-37.Sutton V. 1995. Lessons in SignWriting: textbook & workbook. Deaf Action Committee for Sign Writing (La Jolla CA).

(poster exposition) - two venues: 13/05-06/08 (Chaumont) and 17/10-30/11/2017 (Nancy); International audience; Typannot est un système graphématique et typographique qui propose des polices de caractères, des règles de composition et une interface de saisie pour transcrire la langue des signes. Son originalité réside dans l’articulation de deux modes de représentation : un mode phonologique permettant une analyse fine, et un mode logographique permettant une lecture globale aisée.L’étude des langues des signes suppose de pouvoir analyser des enregistrements vidéo grâce à des transcriptions. N’ayant pas d’écriture, la transcription des langues des signes s’avère être une tâche complexe et difficile. Le projet Typannot a pour but de concevoir une suite d’outils typographiques et numériques permettant de transcrire graphiquement les différents paramètres composants la langue des signes.Cette recherche a vu le jour à l’École Supérieure d’Art et de Design d’Amiens en 2006 et s’est développée au fil des ans pour devenir une unité structurée et autonome qui travaille et échange avec d’autres institutions culturelles et scientifiques. Les questions d’écriture des langues des signes sont explorées au travers des projets Photocalli et Typannot grâce à l’emploi de technologies novatrices de captation du geste et de conception typographique. Ce projet réunit Dominique Boutet, Patrick Doan, Claire Danet, Claudia Savina Bianchini, Timothée Goguely, Adrien Contesse et Morgane Rébulard. Cette recherche est soutenue par le conseil régional de Picardie, la Délégation Générale à la Langue Française et aux Langues de France, le Crédit Agricole, la société éditrice du logiciel RoboFont ainsi que par le Ministère de la Culture.

International audience

International audience

(similar to C036); International audience; Existing writing systems for Sign Language (SL) (SignWriting [Sutton,2015]; HamNoSys [Prillwitz et al. 1989]) have seldom taken into account the results of the phonology of SL. Yet, the diversity of phonological approaches and the scope of the results, especially for handshapes, are very rich and may cover many languages [Johnson & Liddell 2011a, 2011b; Ann 2005; Eccarius & Brentari 2008]. In the context of SL study, three distinct functions can mainly define a writing system: the readibility, the writability and the searchability. None of the existing systems offers a good compromise. In this work, we present the construction of a graphematic system able to cover all SL handshapes existing in the world, consisting of features based on a phonological description [Eccarius & Brentari 2008]. This multidisciplinary project, summoning linguists and type designers, aims at producing a readable and stable, unambiguous and fully searchable glyphic system that provides a relevant solution for the transcription of SL and for a forthcoming writing. The graphematic and the glyphic systems (237 glyphs) are modular and adaptable to new configurations as needed, potentially including all the 120 SLs found in the world. The conception of the glyphs was equally guided by the highlighted phonological/phonetical features (graphematic system) and the graphical and scriptural rules (glyph formula) [Bringshurst 2004; Billeter 2010].We consider that the phonetics for handshapes is praxis: beside a linguistic use, phonetic doesn’t exist for signs independently from manual activities. The influence of the gestural praxis on the symbolic gestures is investigated here (connection between phonology and phonetics). What roles do the handshapes of prehension play [Napier 1956] on SL handshapes [Marzke & Shackley 1986]? The distribution of the selection of the fingers, the independence of each finger, and the behavior of each phalanx have been studied. The results challenge the ideas developped by [Ann 2005] in favour of Napier’s hypothesis of power and precise grips. This analysis suggests an influence of phonetics on the phonology of the handshapes. These features helped designing the shapes and the graphematic system we use and on which we build the glyph formula.Following this work, and according to the phonological graphematic description of handshapes by [Eccarius & Brentari 2008], a modular graphic system based on phonological key components was devised. This approach allowed an economical and visual, yet rigorous, design process. After this first phase we are organizing and encoding the glyphs in order for them to be fully searchable through the Unicode standard.Bibliography:[1] Sutton, Valery. “SignWriting for sign languages”. checked online 13/11/2015. http://www.signwriting.org/[2] Prillwitz, Siegmund, Regina Leven, Heiko Zienert, Thomas Hanke & Jan Henning. “Hamburg notation system for sign languages: an introductory guide”. Signum Press. Hamburg, 1989.[3] Johnson, Robert E. & Scott K. Liddell. “Toward a phonetic representation of hand configuration: the fingers”. Sign Language Studies 12, no° 1 (2011a): 5-45. doi: 10.1353/sls.2011.0013[4] Johnson, Robert E. & Scott K. Liddell. “Toward a phonetic representation of hand configuration: the thumb”. Sign Language Studies 12, no° 2 (2011b): 316-333. doi: 10.1353/sls.2011.0020[5] Ann, Jean. “A functional explanation of Taiwan Sign Language handshape frequency”. Language and Linguistics (Taipei) 6, no° 2 (2005): 217.[6] Eccarius, Petra & Brentari, Diane. “Handshape coding made easier: a theoretically based notation for phonological transcription”. Sign Language & Linguistics 11, no° 1 (2008): 69-101.[7] Bringshurst, Robert. The solid form of language: an essay on writing and meaning. 2004, Gaspereau.[8] Billeter, Jean-François. Essai sur l’art chinois de l’écriture et ses fondements, Paris, Allia,‎ 2010, 416 p.[9] Napier, John R. “The prehensile movements of the human hand”. Journal of Bone and Joint Surgery 38, n° 4 (1956): 902-913.[10] Marzke, Mary W. & M. Steven Shackley. “Hominid hand use in the Pliocene and Pleistocene: evidence from experimental archaeology and comparative morphology”. Journal of Human Evolution 15, no 6 (septembre 1986): 439-460. doi: 10.1016/S0047-2484(86)80027-6.

(similar to C037); International audience; Existing writing systems for SL (SignWriting [Sutton, 1974]; HamNoSys [Prillwitz et al., 1989]) have seldom taken into account the results of the phonology of SL. Yet, the diversity of phonological approaches and the scope of the results, especially for handshapes, are very rich and may cover many languages (Johnson & Liddell, 2011; Ann, 2005; Eccarius & Brentari, 2008). In the context of SL study, three distinct functions can mainly define a writing system: the readibility, the writability and the searchability. None of the existing systems offers a good compromise. In this work, we present the construction of a graphematic system able to cover all SL handshapes existing in the world, consisting of features based on a phonological description (Eccarius & Brentari, 2008). This multidisciplinary project, summoning linguists and type designers, aims at producing a readable and stable, unambiguous and fully searchable glyphic system that provides a relevant solution for the transcription of SL and for a forthcoming writing. The graphematic and the glyphic systems (237 glyphs) are modular and adaptable to new configurations as needed, potentially including all the 120 SLs found in the world. The conception of the glyphs was equally guided by the highlighted phonological/phonetical features (graphematic system) and the graphical and scriptural rules (glyph formula) (Bringhurst, 2004; Billeter, 2010).We consider that the phonetics for handshapes is praxis: beside a linguistic use, phonetic doesn’t exist for signs apart from the manual activities. The influence of the gestural praxis on the symbolic gestures is investigated here (connection between phonology and phonetics). What roles do the handshapes of prehension play (Napier, 1956) on SL handshapes (Marzke, 1986)? The distribution of the selection of the fingers, the independence of each finger, and the behavior of each phalanx have been studied. The results challenge the ideas developped by Ann (2005) in favour of Napier’s hypothesis of power and precise grips. This analysis suggests an influence of phonetics on the phonology of the handshapes. These features helped designing the shapes and the graphematic system we use, and on which we build the glyph formula.Following this work, and according to the phonological graphematic description of handshapes by Eccarius & Brentari (2008), a modular graphic system based on phonological key components was devised. This approach allowed an economical and visual, yet rigorous, design process. These glyphs have been tested during a protocol leading to a recognition task (52 subjects) and a compositional task (6 subjects). The results underlined the need to follow logical construction rules using limited numbers of compounds (Version 2), as opposed to more customized shapes for each configuration (V1). After this first phase we are organizing and encoding the glyphs in order for them to be fully searchable through the Unicode standard.Bibliography:−Ann, Jean. “A functional explanation of Taiwan Sign Language handshape frequency”. Language and Linguistics (Taipei) 6, no 2 (2005): 217.−Billeter, Jean François. Essai sur l’art chinois de l’écriture et ses fondements, Paris, Allia,‎ 2010, 416 p.−Bringshurst, Robert. The solid form of language: an essay on writing and meaning. 2004, Gaspereau.−Eccarius, Petra, and Brentari, Diane. “Handshape coding made easier: a theoretically based notation for phonological transcription”. Sign Language & Linguistics 11, no 1 (2008): 69-101.−Johnson, Robert E., and Scott K. Liddell. “Toward a phonetic representation of hand configuration: the fingers”. Sign Language Studies 12, no 1 (2011a): 545. doi: 10.1353/sls.2011.0013−Johnson, Robert E., and Scott K. Liddell. “Toward a phonetic representation of hand configuration: the thumb”. Sign Language Studies 12, no 2 (2011b): 316-333. doi: 10.1353/sls.2011.0020−Marzke, Mary W., and M. Steven Shackley. “Hominid hand use in the Pliocene and Pleistocene: Evidence from experimental archaeology and comparative morphology”. Journal of Human Evolution 15, no 6 (septembre 1986): 439-460. doi:10.1016/S0047-2484(86)80027-6.−Napier, John R. “The prehensile movements of the human hand”. Journal of Bone and Joint Surgery 38, no 4 (1956): 902-913.−Prillwitz, Siegmund, Regina Leven, Heiko Zienert, Thomas Hanke, and Jan Henning. “Hamburg notation system for sign languages: an introductory guide”. Signum Press. Hamburg, 1989.−Sutton, Valery. “SignWriting for sign languages”. checked online 26/10/2012. http://www.signwriting.org/

(poster)

International audience

(poster); International audience; Few systems are able to annotate sign language (SL): Stokoe Notation System, HamNoSys, SignWriting, BTS. Either they are based on symbols representing SL specific parameters - difficult to write, or they are based on linear alpha-numerical arbitrary encodings. The question of annotation is widely debated and remains an essential requirement for linguistic analysis whether in the form of ID-gloss or shape description. Typannot aims at giving better typographic tools by creating a dedicated glyph system that allows users to annotate the SL parameters in an iconic way thanks to the expertise of linguists, type designers and a roboticist.We will present the concepts and the rules behind our glyphic system and show how we are able to design 237 handshapes glyphs allowing their annotation in 10 SL (based on Eccarius & Brentari 2008). The components and the rules are limited, organized in order to maximize the system economy and learning curve, both in writing and reading (Noordzij 2006).This glyphic system can annotate the other manual parameters allowing combination of handshapes, movements and location. The orientation can be deduced from information inscribed in each glyphic unit. The design of this typeface is based at least on a graphematic description of a panel of 240 signs (Bickford 2005) for 4 SL. Those three graphematic levels once unified in a common writing space should open new perspectives in researching a writing system aimed at the deaf community. This typographic system will reach a phonological level of representation in order to qualify as a viable transcription of SL.A low-cost motion capture system (Weichert et al. 2013) will be used to enrich the glyphs adding more information than just configuration, as speed, acceleration and orientation. We are also considering the latest font format (OpenType) to allow dynamic font features like contextual glyph replacement.Bibliography[1] Eccarius, Petra, & Diane Brentari. «Handshape coding made easier: a theoretically based notation for phonological transcription». Sign Language & Linguistics 11, no 1 (2008): 69-101.[2] Bickford, J. Albert. The signed languages of Eastern Europe. Citeseer, 2005. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.103.5039&rep=rep1&type=pdf.[3] Weichert, Frank, Daniel Bachmann, Bartholomäus Rudak, & Denis Fisseler. «Analysis of the Accuracy and Robustness of the Leap Motion Controller». Sensors 13, no 5 (14 May 2013): 6380-93. doi:10.3390/s130506380.

International audience; Cet article expose la perspective d’engager une recherche phénoménologique en première personne afin d’approfondir nosconnaissances sur la langue des signes (LS) et plus particulièrement sa gestuelle. Cette approche s’inscrit dans le cadre d’une recherchesur la création scripturale basée sur le geste.Nous commencerons par étudier ce qu’implique une écriture pour la LS, puis par une approche complémentaire de celles existantes enlinguistique nous exposerons la méthode employée, ses ajustements nécessaires dus à la LS ainsi que les premières observations.

International audience

DOPPIONE con HAL-01297193; Les systèmes d’écriture existants pour les Langues des Signes (LS) (SignWriting [Sutton 1974], Hamnosys [Prillwitz et al. 1989], SIS5) n’ont que très peu pris en compte les résultats de la phonologie des LS. Pourtant la diversité des approches en phonologie et la portée des résultats notamment pour les configurations (conformation stabilisée de la main et des doigts) sont très riches et couvrent parfois de nombreuses langues (Sandler 2012, Johnson & Liddell 2011, Ann 2005, Eccarius & Brentari 2008). Nous exposons ici les règles de construction d’un système graphématique capable de couvrir l’ensemble des configurations des LS du monde, composé de tracés modulaires, basée sur la description phonologique de Brentari (modèle prosodique), offrant une lecture aisée (legibility, McMonnies 1999), une écriture simple, non ambiguë et requêtable, réalisée dans le cadre d’un projet pluridisciplinaire convoquant linguistes et designers typographes.237 handshapes sont représentées sur la base de la description phonologique d’Eccarius & Brentari faite pour 9 LS (2008). Le système de description phonologique initial est réduit à 9 règles ; ceci permet de diminuer le nombre d’informations nécessaire à la graphie/lecture. Le système graphématique et glyphique, modulable, est extensible à de nouvelles configurations si besoin, incluant potentiellement les 120 LS du monde (recensement Ethnologue.org). Notons que la description adoptée -Eccarius & Brentari- assure une couverture assez complète des configurations existantes dans les LS (237 ici contre 250 pour SignWriting basé sur une combinatoire théorique).Certaines relations entre phonologie et substrat phonétique sont interrogées dans cette présentation ; notamment, l’appréciation de l’influence qu’exerce la praxis gestuelle sur la gestuelle symbolique. Quels rôles les configurations de préhension (Napier 1956) jouent-elles sur les configurations dans les LS ? Pour cela nous nous intéressons dans un premier temps aux flexions des doigts (prises digito-palmaires), puis au verrouillage avec le pouce (pouce opposé). La distribution de la sélection des doigts et l’autonomie de chacun seront abordées, ainsi que leur comportement (flexion, extension, écartement) jusqu’à savoir sur quelle(s) phalange(s) la flexion opère.Parmi les 4 doigts, hormis le pouce donc, un gradient croissant de sélection allant de l’auriculaire à l’index se dessine, marquant une sélection préférentielle pour l’index et le majeur. Ces quatre doigts ne présentent pas la même autonomie. Sans réelle surprise, l’index est le doigt qui présente le plus grand nombre d’occurrences lorsqu’il apparaît seul, même si son association avec d’autres doigts est assez équilibrée. A l’inverse, l’annulaire n’est absolument pas autonome. De plus il n’apparaît que très peu lié à un autre doigt adjacent -auriculaire ou majeur-, mais est plutôt sélectionné avec deux autres doigts adjacents et très majoritairement avec les trois autres doigts. L’autonomie des doigts est donc très contrastée comme on le voit d’ailleurs dans les tâches de saisie d’objets. Elle n’est pas en rapport avec le nombre de sélection des doigts et laisse présager que le gradient de sélection intervient hiérarchiquement avant l'autonomie des doigts.Les doigts sont sélectionnés de manière ordonnée et croissante, ils ne présentent pas la même autonomie les uns par rapport aux autres. Ont-ils malgré cela, un comportement similaire (mêmes traits) ? Cette question revient à savoir si l’autonomie a un impact sur le comportement. Ce n’est pas le cas ; les doigts présentent le même comportement. Cette similarité ne dépend pas d’un type de configuration en particulier (‘’B’’ [sélection des 4 doigts] ou ‘’U’’ [sélection du majeur et de l’index] par exemple). Par contre le comportement diffère selon qu’on s’intéresse aux premières phalanges ou aux deuxièmes et troisièmes. En résumé, la sélection des doigts répond à un gradient croissant de l’auriculaire à l’index correspondant à ce qu’on remarque dans le recrutement des doigts pour la préhension. Le comportement des doigts est similaire en termes de flexion et d’extension montrant ainsi une absence de spécialisation comportementale d’un des quatre doigts, contrairement à ce qu’on pourrait attendre de l’index par exemple (configuration ‘’1’’), souvent présenté dans les études sur les langues signées comme spécialisé dans le pointage (extension). Les lieux de changements concernent les phalanges quels que soient les doigts. De ce point de vue l’approche phonologique plus précise de Johnson & Liddell (2011) notant jusqu’aux phalanges est pertinente. Toutefois, par delà la dimension phonologique, il semble que le niveau phonétique, c’est-à-dire ici la préhension, part praxique de la gestuelle, joue un rôle prépondérant dans l’inventaire des configurations pour les langues signées.L’étude sur le pouce qui sera également présentée ici corrobore l’incorporation de la préhension dans les configurations symboliques.Certains éléments saillants -souvent liés à cette préhension sous-jacente- ont guidé les choix graphiques des glyphes du système d’annotation Typannot. Le comportement identique des quatre doigts nous a poussé à ne pas les différencier graphiquement (contrairement à SignWriting). La flexion/extension de certaines phalanges oriente la représentation glyphique des configurations (de face versus de trois-quarts face). La flexion maximale de l’ensemble des phalanges lorsque les doigts ne sont pas sélectionnés n’est pas marquée, par contre elle l’est graphiquement lorsqu’il s’agit de l’extension. Eccarius & Brentari quant à elles, marquent les deux. L’accolement des doigts (versus leur écartement latéral) est traité graphiquement comme un tout par un contour enveloppant l’ensemble des doigts accolés renvoyant à la massivité vs la dispersion de la prise ; en référence aux deux catégories de préhension de Napier : saisie de force avec des doigts accolés versus de précision (doigts écartés).Ainsi on peut dire que la phonologie a structuré le système glyphique Typannot dans son ensemble y compris dans ses allers-retours vers la phonétique.RéférencesAnn, Jean. « A functional explanation of Taiwan Sign Language handshape frequency ». Language and Linguistics - Taipei 6, n° 2 (2005): 217.Eccarius, Petra, et Diane Brentari. « Handshape coding made easier; a theoretically based notation for phonological transcription ». Sign Language & Linguistics 11, n° 1 (2008): 69-101.Johnson, Robert E., et Scott K. Liddell. « Toward a phonetic representation of hand configuration: the fingers ». Sign Language Studies 12, n° 1 (2011): 5-45. doi: 10.1353/sls.2011.0013.McMonnies, Charles W. « Chart construction and letter legibility/readability ». Ophthalmic and Physiological Optics 19, no 6 (1999): 498-506. doi: 10.1046/j.1475-1313.1999.00460.x.Napier, John R. « The prehensile movements of the human hand ». Journal of Bone and Joint Surgery 38, n° 4 (1956): 902-913.Prillwitz S., R. Leven, H. Zienert, T. Hanke, et J. Henning. Hamburg notation system for sign languages: an introductory guide. Signum Press, Hamburg, 1989.Sandler, Wendy. « The phonological organization of Sign Languages: Sign Language phonology ». Language and Linguistics Compass 6, n° 3 (mars 2012): 162-82. doi: 10.1002/lnc3.326.Sutton, Valery. « SignWriting for Sign Languages ». Consulté le 26 octobre 2013. http://www.signwriting.org/

poster; International audience; Few systems are able to annotate sign language (SL): HamNoSys (Prillwitz et al. 1989), SignWriting (Sutton 1995, Bianchini 2012), BTS (Slobin et al. 2001). They are either based on symbols representing SL specific parameters that are often difficult to write, or they are based on linear alpha-numerical arbitrary encodings. The question of annotation is widely debated (Johnston 2008, Hoiting & Slobin 2002, Hanke 2004, Konrad et al. 2012) and remains an essential requirement for linguistic analysis whether in the form of ID-gloss or shape description. Typannot aims at giving better typographic tools by creating a dedicated glyph system that allows users to annotate the different SL parameters in an iconic way thanks to the expertise of linguists, type designers and a robotics engineer (Miletitch et al. 2013).We will present the concepts and rules behind our glyphic system and show how we are able to design 237 hand shapes glyphs allowing their annotation in 10 SL (based on the organization made by Eccarius & Brentari 2008). The components and the rules are limited and organized in order to maximize the system economy and learning curve, both in writing and reading (McMonnies 1999; Noordzij 2006).We will also present a glyphic system for annotating the orientation of the palm allowing combination with hand shapes and hand movements representations. Those three graphematic levels once unified in a common writing space should open new perspectives in researching a writing system aimed at the deaf community. We will present a typographic system that will reach a phonological level of representation in order to qualify as a viable transcription of SL.A low-cost motion capture system will be used to enrich the glyphs adding more information than just configuration, i.e. speed, acceleration and orientation. We are also considering the latest font format (opentype) to allow dynamic font features like contextual glyph replacement.Bibliography[1] Prillwitz, S., R. Leven, H. Zienert, T. Hanke, J. Henning. Hamburg notation system for sign languages: an introductory guide. Signum Press. Hamburg, 1989.[2] Sutton V. 1995. Lessons in SignWriting: textbook & workbook. Deaf Action Committee for Sign Writing, La Jolla (CA)[3] Bianchini, Claudia S., Christian Cuxac, Dominique Boutet, Margherita Castelli, Elena Antinoro Pizzuto. Analyse métalinguistique de l’émergence d’un système d’écriture des Langues des Signes: SignWriting et son application à la Langue des Signes Italienne (LIS). Paris 8 et CNR-ISTC-LLS, 2012.[4] Slobin, Dan I., Nini Hoiting, Michelle Anthony, Yael Biederman, Marlon Kuntze, Reyna Lindert, Jennie Pyers, Helen Thumann, Amy Weinberg. Signlanguage transcription at the level of meaning components: the Berkeley Transcription System (BTS). Sign Language & Linguistics 4, no 1-2 (2001): 63-104. doi:10.1075/sll.4.12.07slo[5] Johnston, Trevor. Corpus linguistics and signed languages: no lemmata, no corpus. In 3rd Workshop on the Representation and Processing of Sign Languages, 82-88. Onno Crasborn, Eleni Efthimiou, Thomas Hanke, Ernst D. Thoutenhoofd, Inge Zwitserlood, 2008.[6] Hoiting, N., D.I. Slobin. Transcription as a tool for understanding: the Berkeley Transcription System for sign language research (BTS). Directions in sign language acquisition, 2002, 55-75.[7] Hanke, Thomas. HamNoSys — Representing sign language data in language resources and language processing contexts. In Workshop on the Representation and Processing of Sign Languages on the occasion of the 4th International Conference on Language Resources and Evaluation, 1-6. Lisbon: ELDA, 2004.[8] Konrad, R., T. Hanke, S. König, G. Langer, S. Matthes, R. Nishio, A. Regen. From form to function: a database approach to handle lexicon building and spotting token forms in sign languages. In LREC, 2012.[9] Miletitch, Roman, Claire Danet, Morgane Rébulard, Raphël de Courville, Patrick Doan, Dominique Boutet. Photocalligraphy: writing sign language. In Electronic Visualisation in Arts and Culture, Jonathan P. Bowen, Suzanne Keene, KiaNg eds, 167-180. Springer Series on Cultural Computing. Springer London, 2013. http://link.springer.com/chapter/10.1007/978-1-4471-5406-8_12[10] Eccarius, Petra, Diane Brentari. Hand shape coding made easier: a theoretically based notation for phonological transcription. Sign Language & Linguistics 11, no 1 (2008): 69-101.[11] McMonnies, Charles W. Chart construction and letter legibility/readability. Ophthalmic and Physiological Optics 19, no 6 (1999): 498-506. doi:10.1046/j.1475-1313.1999.00460.x[12] Noordzij, G. The stroke, theory of writing. Hyphenpress 2006.

International audience; This work constitutes a contribution to the emergence of a common writing for French Sign Language in a graphical or even a typographical framework. Our hypothesis is as follows : in its execution, the gestural sign contains a readable graphic trace that can be visualized with a photographic device. In order to evaluate this hypothesis, we gather a photographic corpus made of isolated elicited signs.; Le projet GestuelScript se présente comme une contribution à l'émergence d'une écriture courante pour la Langue des Signes Française dans un cadre de travail essentiellement graphique, voire typographique. Notre hypothèse : le signe gestuel décrirait, dans sa réalisation, une trace graphique lisible qu'un dispositif photographique permet de visualiser. Pour la tester, nous constituons un corpus de travail photographique fait de signes isolés élicités.

International audience; This work constitutes a contribution to the emergence of a common writing for French Sign Language. Our approach, while in line with previously conducted research (the LSscript project made a review of past works in the field of research of a graphical formalism for LSF), brings the reflexion into a multidisciplinary context. Our assumption is that writing and sign language share a same visual-gestural modality. Our hypothesis is as follows : in its execution, the gestural sign contains a readable graphic trace. In order to question our assumption, we use multi-angle photography to constitute a corpus of signs in which all three dimensions of these traces are visible. This article describes the principles and hypothesis, the techniques for building the corpus, and the validation protocol of our hypothesis.; Ce travail constitue une contribution à l'émergence d'une écriture courante pour la Langue des Signes Française. Complémentaire des recherches déjà menées (le projet LSscript a fait la recension des travaux sur la recherche d'un formalisme graphique pour la LSF), notre projet s'inscrit dans un espace de réflexion pluridisciplinaire. Nous suivons un postulat selon lequel l'écriture et les LS partagent une même modalité visuo-gestuelle. Notre hypothèse : le signe gestuel contiendrait, dans sa réalisation, une trace graphique lisible. Nous proposons de capter des traces graphiques par une technique de photographie multi- angles, afin de constituer un corpus de signes rendant visible ce tracé dans les trois dimensions de l'espace. Cet article décrit nos principes et hypothèses, les techniques de constitution du corpus et notre protocole de validation des hypothèses.