Search

International audience; This paper uses the recent glottal flow model for iterative adaptive inverse filtering to analyze recordings from dysfunctional speakers, namely those with larynx-related impairment such as laryngectomy. The analytical model allows extraction of the voice source spectrum, described by a compact set of parameters. This single model is used to visualize and better understand speech production characteristics across impaired and nonimpaired voices. The analysis reveals some discriminative aspects of the source model which map to a physiological class description of those impairments. Furthermore, being based on analysis of source parameters only, it is complementary to any existing techniques of vocal-tract or phonetic analysis. The results indicate the potential for future automated speech reconstruction systems that adapt to the method of reconstruction required, as well as being useful for mainstream speech systems, such as ASR, in which front-end analysis can direct back-end models to suit characteristics of impaired speech.

Speech glottal flow has been predominantly described in the time-domain in past decades, the Liljencrants-Fant (LF) model being the most widely used in speech analysis and synthesis, despite its computational complexity. The causal/anti-causal linear model (LF

International audience; Whisper-to-speech conversion is motivated by laryngeal disorders, in which malfunction of the vocal folds leads to loss of voicing. Many patients with laryngeal disorders can still produce functional whispers, since these are characterised by the absence of vocal fold vibration. Whispers therefore constitute a common ground for speech rehabilitation across many kinds of laryngeal disorder. Whisper-to-speech conversion involves recreating natural-sounding speech from recorded whispers, and is a non-invasive and non-surgical rehabilitation that can maintain a natural method of speaking, unlike the existing methods of rehabilitation. This paper proposes a new rule-based method for whisper-to-speech conversion that replaces the noisy whisper sound source with a synthesised speech-like harmonic source, while maintaining the vocal tract component unaltered. In particular, a novel glottal source generator is developed in which whisper information is used to parameterise the excitation through a high-quality glottis model. Evaluation of the system against the standard pulse train excitation method reveals significantly improved performance. Since our method is glottis-based, it is potentially compatible with the many existing vocal tract component adaptation systems.

International audience; Neural vocoders are systematically evaluated on homogeneous train and test databases. This kind of evaluation is efficient to compare neural vocoders in their "comfort zone", yet it hardly reveals their limits towards unseen data during training. To compare their extrapolation capabilities, we introduce a methodology that aims at quantifying the robustness of neural vocoders in synthesising unseen data, by precisely controlling the ranges of seen/unseen data in the training database. By focusing in this study on the pitch (F0) parameter, our methodology involves a careful splitting of a dataset to control which F0 values are seen/unseen during training, followed by both global (utterance) and local (frame) evaluation of vocoders. Comparison of four types of vocoders (autoregressive, sourcefilter, flows, GAN) displays a wide range of behaviour towards unseen input pitch values, including excellent extrapolation (WaveGlow); widely-spread F0 errors (WaveRNN); and systematic generation of the training set median F0 (LPCNet, Parallel WaveGAN). In contrast, fewer differences between vocoders were observed when using homogeneous train and test sets, thus demonstrating the potential and need for such evaluation to better discriminate the neural vocoders abilities to generate out-of-training-range data.

International audience; Audio books are commonly used to train text-to-speech models (TTS), as they offer large phonetic content with rather expressive pronunciation, but number and sizes of publicly available audio books corpora differ between languages. Moreover, the quality and accuracy of the available utterance segmentations are debatable. Yet, the impact of segmentation on the output synthesis is not well established. Additionally, utterances are generally used individually, without taking advantage of text level structuring information, even though they influence speaker reading. In this paper, we conduct a multidimensional evaluation of Tacotron2 trained on different segmentations and text level annotations of the same French corpus. We show that both spectrum accuracy and expressiveness depend on the segmentation used. In particular, a shorter segmentation, in addition with the annotation of paragraphs, benefits to spectrum reconstruction at the detriment of phrasing. Multidimensional analysis of mean opinion scores obtained with a MUSHRA-experiment revealed that phrasing was relatively more important than spectrum accuracy in perceptual judgement. This work serves as evidence that particular attention must be given to models evaluation, as well as how to use the training corpus to maximize synthesis characteristics of interest.

Digital plays a key role in the transformation of medicine. Beyond the simple computerisation of healthcare systems, many non-drug treatments are now possible thanks to digital technology. Thus, interactive stimulation exercises can be offered to people suffering from cognitive disorders, such as developmental disorders, neurodegenerative diseases, stroke or traumas. The efficiency of these new treatments, which are still primarily offered face-to-face by therapists, can be greatly improved if patients can pursue them at home. However, patients are left to their own devices which can be problematic. We introduce THERADIA, a 5-year project that aims to develop an empathic virtual agent that accompanies patients while receiving digital therapies at home, and that provides feedback to therapists and caregivers. We detail the architecture of our agent as well as the framework of our Wizard-of-Oz protocol, designed to collect a large corpus of interactions between people and our virtual assistant in order to train our models and improve our dialogues.

International audience

International audience; Cantor Digitalis is a performative singing synthesizer that is composed of two main parts: a chironomic control interface and a parametric voice synthesizer. The control interface is based on a pen/touch graphic tablet equipped with a template representing vocalic and melodic spaces. Hand and pen positions, pen pressure, and a graphical user interface are assigned to specific vocal controls. This interface allows for real-time accurate control over high-level singing synthesis parameters. The sound generation system is based on a parametric synthesizer that features a spectral voice source model, a vocal tract model consisting of parallel filters for vocalic formants and cascaded with anti-resonance for the spectral effect of hypo-pharynx cavities, and rules for parameter settings and source/filter dependencies between fundamental frequency, vocal effort, and formants. Because Cantor Digitalis is a parametric system, every aspect of voice quality can be controlled (e.g., vocal tract size, aperiodicities in the voice source, vowels, and so forth). It offers several presets for different voice types. Cantor Digitalis has been played on stage in several public concerts, and it has also been proven to be useful as a tool for voice pedagogy. The aim of this article is to provide a comprehensive technical overview of Cantor Digitalis.

International audience; Audio, visual, and proprioceptive actions are involved when manipulating a graphic tablet musical interface. Previous works suggested a possible dominance of the visual over the auditory modality in this situation. The main goal of the present study is to examine the interferences between these modalities in visual, audio, and audiovisual target acquisition tasks. Experiments are based on a movement replication paradigm, where a subject controls a cursor on a screen or the pitch of a synthesized sound by changing the stylus position on a covered graphic tablet. The experiments consisted of the following tasks: (1) a target acquisition task that was aimed at a visual target (reaching a cue with the cursor displayed on a screen), an audio target (reaching a reference note by changing the pitch of the sound played in headsets), or an audiovisual target, and (2) the replication of the target acquisition movement in the opposite direction. In the return phase, visual and audio feedback were suppressed. Different gain factors perturbed the relationships among the stylus movements, visual cursor movements, and audio pitch movements. The deviations between acquisition and return movements were analyzed. The results showed that hand amplitudes varied in accordance with visual, audio, and audiovisual perturbed gains, showing a larger effect for the visual modality. This indicates that visual, audio, and audiovisual actions interfered with the motor modality and confirms the spatial representation of pitch reported in previous studies. In the audiovisual situation, vision dominated over audition, as the latter had no significant influence on motor movement. Consequently, visual feedback is helpful for musical targeting of pitch on a graphic tablet, at least during the learning phase of the instrument. This result is linked to the underlying spatial organization of pitch perception. Finally, this work brings a complementary approach to previous studies showing that audition may dominate over vision for other aspects of musical sound (e.g., timing, rhythm, and timbre).

International audience; The proposed performance features six musicians coming from two ensembles: Chorus Digitalis (Paris) and Vox Tactum (Vancouver, Mons). These two ensembles have grown separately from a common seed research on expressive ges- tural control of speech and singing synthesis, started in 2005. Several musical instruments have emerged from this research, such as Cantor Digitalis, HandSketch and ChoirMob. As six of us are planning to attend NIME’13 with scientific papers, we would like to take this unique chance to gather on stage in Daejeon. By this mean, we hope to perform as one choir, just made of synthetic voices and fully operated by control surfaces. The performance will contain ”scenes” that will reveal our various usages of tablets for the expressive production of artificial speech and singing.

International audience

International audience; Cantor Digitalis, a real-time formant synthesizer controlled by a graphic tablet and a stylus, is used for assessment of melodic precision and accuracy in singing synthesis. Melodic accuracy and precision are measured in three experiments for groups of 20 and 28 subjects. The task of the subjects is to sing musical intervals and short melodies, at various tempi, using chironomy (hand-controlled singing), mute chironomy (without audio feedback), and their own voices. The results show the high accuracy and precision obtained by all the subjects for chironomic control of singing synthesis. Some subjects performed significantly better in chironomic singing compared to natural singing, although other subjects showed comparable proficiency. For the chironomic condition, mean note accuracy is less than 12 cents and mean interval accuracy is less than 25 cents for all the subjects. Comparing chironomy and mute chironomy shows that the skills used for writing and drawing are used for chironomic singing, but that the audio feedback helps in interval accuracy. Analysis of blind chironomy (without visual reference) indicates that a visual feedback helps greatly in both note and interval accuracy and precision. This study demonstrates the capabilities of chironomy as a precise and accurate mean for controlling singing synthesis

International audience; Deux méthodes d’ajustement de justesse mélodique sont présentées et comparées en terme de justesse et d’expressivité : l’ajustement notes étendues et l’ajustement élastique. Le but de ces ajustements est d’améliorer la justesse du musicien, tout en minimisant les contraintes nuisant à son expressivité. Les méthodes diffèrent par le type de transformation appliquée à la hauteur jouée. De plus, leurs dynamiques d’application sont paramétrables. La comparaison des deux montre des performances similaires en termes de justesse. En revanche la limitation de l’expressivité dépend grandement de la dynamique d’application choisie. Finalement, l’ajustement élastique permet la plus grande expressivité.

National audience; Les instruments chanteurs a synthèse ou l'analyse-synthèse de la voix chantée est un sujet de recherche déjà ancien. La voix chan-tée a été synthétisée dès les premières tentatives de synthèse numérique de la parole dans les années 1960. Le propos de cet article est de présenter le principe de fonctionnement des instruments chanteurs que nous avons développés au cours des dernières années, et de discuter des applications et de l'ave-nir de cette approche. Comme la place manque pour détailler tous ces travaux, une liste de publications en fin d'article permettra au lecteur curieux d'appro-fondir le sujet. Les instruments ont été utilisés dans divers concerts et des exemples sonores se trouvent facilement sur la toile. Résumé Les instruments chanteurs sont nés de la rencontre entre synthèse vocale et nouvelles interfaces pour l'interaction humain-machine. La voix n'est pas un instrument de musique, car il n'y a pas d'objet externe mis en jeu par les membres ou par le souffle. La synthèse numérique permet pour la première fois une coupure entre le sujet et sa voix, en construisant des instruments chanteurs manipulés par les mains, les pieds, ou par toutes sortes d'interfaces humain-machine. Cependant, les possibilités de contrôle des instruments chanteurs sont encore limitées à certains aspects, car une transposition des gestes internes du chant effectués par l'appareil vocal en gestes externes des membres ne va pas de soi. Certains gestes sont analogues, d'autres sont médiatisés par un espace perceptif. Les travaux menés sur trois instruments chanteurs sont présentés : le contrôle de l'intonation par un stylet sur une tablette graphique et des gestes d'écriture ; le contrôle des voyelles et de la qualité vocale sur une surface ; le contrôle bimanuel de l'articulation consonantique ; le contrôle syllabique du rythme. Les modèles de synthèse sous-jacents utilisent soit la simulation du modèle source-filtre, soit la modification d'échantillons préenregistrés et étiquetés. Le contrôle des instruments chanteurs est multimodal, impliquant l'ouïe ainsi que la vue, le toucher, et la kinesthésie. Cette combinaison sensorielle et motrice permet dans certains cas de rendre les instruments chanteurs plus justes et précis que la voix, la vue privilégiant les aspects mélodiques et l'audition les aspects rythmiques. Miroir de la voix, l'instrument chanteur autorise toutes sortes de spéculations : musicales bien sûr, avec le Chorus Digitalis, choeur de voix de synthèse, mais aussi pour l'analyse des pratiques vocales, pour l'éducation ou la rééducation, en renforçant l'apprentissage de gestes vocaux par des traces visuelles, des gestes manuels ou corporels. Le statut symbolique de la voix est également affecté par la possibilité de contrôler et de produire le son vocal en dehors du corps : corps augmenté, mise en scène de l'expression vocale, double de la voix, jeu de la voix d'un ou d'une autre. Abstract Singing instruments are the result of the encounter between voice synthesis and new interfaces for human-computer interaction. Voice is not a musical instrument, since it does not involve an external object stimulated by limbs or breathe. In contrast, digital synthesis allows for the first time to separate the subject from its voice, by building singing instruments manipulated by hands, feet, or any human-computer interface. However, possibilities for singing instruments control are still limited to some aspects, as the transposition from internal singing gestures realised by the vocal apparatus to external gestures is not trivial. Some gestures are analogous, while others are transposed in perceptive spaces. Related work realised on three singing instruments is introduced: the intonation control by a stylus on a graphic tablet and writing gestures; the vocalic and voice quality controls on a surface; the bi-manual control of consonantal articulation; the rhythmic control of syllables. The underlying voice production models use either the simulation of a source-filter model, or the modification of pre-recorded and labelled samples. The control of singing instruments is multi-modal, involving hearing, sight, touch, and kinaesthesia. In some extent, this sensorimotor combination allows the singing instrument to be more accurate and precise than natural voice: the sight favouring melodic aspects while hearing being more related to rhythmic aspects. Mirror of voice, the singing instrument allows any kind of speculation: indubitably musical with Chorus Digitalis, a choir of synthesised voices, but also for the analysis of vocal practices, for education or re-education by strengthening the learning of vocal gestures through uses of visual traces, and manual and corporal gestures. Finally, the symbolic status of voice is also affected by the possibility to produce a vocal sound from outside the body: augmented body, staging of vocal expression, voice double, play of someone else's voice.