Search

Your search keyword '"Karyna Isaieva"' showing total 4 results
Start Over Author "Karyna Isaieva" Topic 0305 other medical science
4 results on '"Karyna Isaieva"'

1. Towards the prediction of the vocal tract shape from the sequence of phonemes to be articulated

Author

Karyna Isaieva, Pierre-André Vuissoz, Yves Laprie, Justine Leclere, Vinicius Ribeiro, Speech Modeling for Facilitating Oral-Based Communication (MULTISPEECH), Inria Nancy - Grand Est, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Department of Natural Language Processing & Knowledge Discovery (LORIA - NLPKD), Laboratoire Lorrain de Recherche en Informatique et ses Applications (LORIA), Centre National de la Recherche Scientifique (CNRS)-Université de Lorraine (UL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-Université de Lorraine (UL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Laboratoire Lorrain de Recherche en Informatique et ses Applications (LORIA), Centre National de la Recherche Scientifique (CNRS)-Université de Lorraine (UL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-Université de Lorraine (UL), Imagerie Adaptative Diagnostique et Interventionnelle (IADI), Institut National de la Santé et de la Recherche Médicale (INSERM)-Université de Lorraine (UL), Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Laboratoire Lorrain de Recherche en Informatique et ses Applications (LORIA), Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS), and Souza Ribeiro, Vinicius de Paulo

Subjects
Speech production, Generalization, Computer science, speech production, Speech recognition, Articulator, 020206 networking & telecommunications, 02 engineering and technology, [INFO] Computer Science [cs], neural networks, Euclidean distance, 030507 speech-language pathology & audiology, 03 medical and health sciences, Position (vector), Data efficiency, 0202 electrical engineering, electronic engineering, information engineering, [INFO]Computer Science [cs], 0305 other medical science, Set (psychology), phoneme-to-articulatory, Vocal tract
Abstract

International audience; In this work, we address the prediction of speech articulators' temporal geometric position from the sequence of phonemes to be articulated. We start from a set of real-time MRI sequences uttered by a female French speaker. The contours of five articulators were tracked automatically in each of the frames in the MRI video. Then, we explore the capacity of a bidirectional GRU to correctly predict each articulator's shape and position given the sequence of phonemes and their duration. We propose a 5-fold cross-validation experiment to evaluate the generalization capacity of the model. In a second experiment, we evaluate our model's data efficiency by reducing training data. We evaluate the point-to-point Euclidean distance and the Pearson's correlations along time between the predicted and the target shapes. We also evaluate produced shapes of the critical articulators of specific phonemes. We show that our model can achieve good results with minimal data, producing very realistic vocal tract shapes.

Published
2021
Full Text
View/download PDF

2. Using Silence MR Image to Synthesise Dynamic MRI Vocal Tract Data of CV

Author

Chrysanthi Dourou, Pierre-André Vuissoz, Karyna Isaieva, Yu Xie, Jacques Felblinger, Ioannis Douros, Ajinkya Kulkarni, Yves Laprie, Laprie, Yves, Speech Modeling for Facilitating Oral-Based Communication (MULTISPEECH), Inria Nancy - Grand Est, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Department of Natural Language Processing & Knowledge Discovery (LORIA - NLPKD), Laboratoire Lorrain de Recherche en Informatique et ses Applications (LORIA), Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Laboratoire Lorrain de Recherche en Informatique et ses Applications (LORIA), Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS), Imagerie Adaptative Diagnostique et Interventionnelle (IADI), Institut National de la Santé et de la Recherche Médicale (INSERM)-Université de Lorraine (UL), School of of Electrical and Computer Engineering [Athens] (School of E.C.E), National Technical University of Athens [Athens] (NTUA), Department of Neurology, Wuhan University [China], Centre d'investigation clinique [Nancy] (CIC), Centre Hospitalier Régional Universitaire de Nancy (CHRU Nancy)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Université de Lorraine (UL), Université de Lorraine (UL)-Institut National de la Santé et de la Recherche Médicale (INSERM), and Université de Lorraine (UL)-Centre Hospitalier Régional Universitaire de Nancy (CHRU Nancy)-Institut National de la Santé et de la Recherche Médicale (INSERM)

Subjects
Computer science, [INFO.INFO-TS] Computer Science [cs]/Signal and Image Processing, Speech recognition, Frame (networking), 020206 networking & telecommunications, 02 engineering and technology, [INFO.INFO-CL]Computer Science [cs]/Computation and Language [cs.CL], Image (mathematics), Set (abstract data type), Silence, 030507 speech-language pathology & audiology, 03 medical and health sciences, image transformation, [INFO.INFO-TS]Computer Science [cs]/Signal and Image Processing, [INFO.INFO-CL] Computer Science [cs]/Computation and Language [cs.CL], vocal tract, Dynamic contrast-enhanced MRI, 0202 electrical engineering, electronic engineering, information engineering, Mr images, 0305 other medical science, Speech resources enrichment, Vocal tract, rtMRI data, pseudo rtMRI synthesis
Abstract

International audience; In this work we present an algorithm for synthesising pseudo rtMRI data of the vocal tract. rtMRI data on the midsagittal plane were used to synthesise target consonant-vowel (CV) using only a silence frame of the target speaker. For this purpose, several single speaker models were created. The input of the algorithm is a silence frame of both train and target speaker and the rtMRI data of the target CV. An image transformation is computed from each CV frame to the next one, creating a set of transformations that describe the dynamics of the CV production. Another image transformation is computed from the silence frame of train speaker to the silence frame of the target speaker and is used to adapt the set of transformations computed previously to the target speaker. The adapted set of transformations is applied to the silence of the target speaker tosynthesise his/her CV pseudo rtMRI data. Synthesised images from multiple single speaker models are frame aligned and then averaged to create the final version of synthesised images. Synthesised images are compared with the original ones using image cross-correlation. Results show good agreement between the synthesised and the original images.

Published
2020

3. Towards a Method of Dynamic Vocal Tract Shapes Generation by Combining Static 3D and Dynamic 2D MRI Speech Data

Author

Ioannis Douros, Anastasiia Tsukanova, Karyna Isaieva, Pierre-André Vuissoz, Yves Laprie, Speech Modeling for Facilitating Oral-Based Communication (MULTISPEECH), Inria Nancy - Grand Est, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Department of Natural Language Processing & Knowledge Discovery (LORIA - NLPKD), Laboratoire Lorrain de Recherche en Informatique et ses Applications (LORIA), Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Laboratoire Lorrain de Recherche en Informatique et ses Applications (LORIA), Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS), Imagerie Adaptative Diagnostique et Interventionnelle (IADI), Université de Lorraine (UL)-Institut National de la Santé et de la Recherche Médicale (INSERM), Douros, Ioannis, and Institut National de la Santé et de la Recherche Médicale (INSERM)-Université de Lorraine (UL)

Subjects
speech resources enrichment, Computer science, Image quality, 02 engineering and technology, [INFO] Computer Science [cs], Set (abstract data type), 030507 speech-language pathology & audiology, 03 medical and health sciences, Dimension (vector space), vocal tract, 0202 electrical engineering, electronic engineering, information engineering, [INFO]Computer Science [cs], Computer vision, Spatial analysis, MRI data, business.industry, Frame (networking), 020206 networking & telecommunications, image transformation, Transformation (function), [INFO.INFO-TI] Computer Science [cs]/Image Processing [eess.IV], [INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV], modality transformation, Artificial intelligence, 0305 other medical science, business, Vocal tract
Abstract

International audience; We present an algorithm for augmenting the shape of the vocal tract using 3D static and 2D dynamic speech MRI data. While static 3D images have better resolution and provide spatial information, 2D dynamic images capture the transitions. The aim of this work is to combine strong points of these two types of data to obtain better image quality of 2D dynamic images and extend the 2D dynamic images to the 3D domain. To produce a 3D dynamic consonant-vowel (CV) sequence, our algorithm takes as input the 2D CV transition and the static 3D targets for C and V. To obtain the enhanced sequence of images , the first step is to find a transformation between the 2D images and the mid-sagittal slice of the acoustically corresponding 3D image stack, and then find a transformation between neighbouring sagittal slices in the 3D static image stack. Combination of these transformations allows producing the final set of images. In the present study we first examined the transformation from the 3D mid-sagittal frame to the 2D video in order to improve image quality and then we examined the extension of the 2D video to the 3rd dimension with the aim to enrich spatial information.

Published
2019
Full Text
View/download PDF

4. A Multimodal Real-Time MRI Articulatory Corpus of French for Speech Research

Author

Arun A. Joseph, Jens Frahm, Dirk Voit, Yves Laprie, Karyna Isaieva, Freddy Odille, Ioannis Douros, Anastasiia Tsukanova, Jacques Felblinger, Pierre-André Vuissoz, Laprie, Yves, Speech Modeling for Facilitating Oral-Based Communication (MULTISPEECH), Inria Nancy - Grand Est, Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)-Department of Natural Language Processing & Knowledge Discovery (LORIA - NLPKD), Laboratoire Lorrain de Recherche en Informatique et ses Applications (LORIA), Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Laboratoire Lorrain de Recherche en Informatique et ses Applications (LORIA), Institut National de Recherche en Informatique et en Automatique (Inria)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS)-Université de Lorraine (UL)-Centre National de la Recherche Scientifique (CNRS), Imagerie Adaptative Diagnostique et Interventionnelle (IADI), Institut National de la Santé et de la Recherche Médicale (INSERM)-Université de Lorraine (UL), Biomedizinische NMR Forschungs GmbH [Göttingen], Max-Planck-Institut für Biophysikalische Chemie - Max Planck Institute for Biophysical Chemistry [Göttingen], Max-Planck-Gesellschaft-Max-Planck-Gesellschaft, and Université de Lorraine (UL)-Institut National de la Santé et de la Recherche Médicale (INSERM)

Subjects
Larynx, [INFO.INFO-AI] Computer Science [cs]/Artificial Intelligence [cs.AI], Speech production, Epiglottis, Computer science, speech production, Speech recognition, 02 engineering and technology, speech syn- thesis, [INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI], 030507 speech-language pathology & audiology, 03 medical and health sciences, Tongue, 0202 electrical engineering, electronic engineering, information engineering, medicine, Index Terms: speech corpus, Spontaneous speech, multi-modal database, 020206 networking & telecommunications, Real-time MRI, medicine.anatomical_structure, Duration (music), French language, real-time MRI data, 0305 other medical science, Vocal tract, 3D MRI data
Abstract

International audience; In this work we describe the creation of ArtSpeechMRIfr: a real-time as well as static magnetic resonance imaging (rtMRI, 3D MRI) database of the vocal tract. The database contains also processed data: denoised audio, its phonetically aligned annotation, articulatory contours, and vocal tract volume information , which provides a rich resource for speech research. The database is built on data from two male speakers of French. It covers a number of phonetic contexts in the controlled part, as well as spontaneous speech, 3D MRI scans of sustained vocalic articulations, and of the dental casts of the subjects. The corpus for rtMRI consists of 79 synthetic sentences constructed from a phonetized dictionary that makes possible to shorten the duration of acquisitions while keeping a very good coverage of the phonetic contexts which exist in French. The 3D MRI includes acquisitions for 12 French vowels and 10 consonants, each of which was pronounced in several vocalic contexts. Ar-ticulatory contours (tongue, jaw, epiglottis, larynx, velum, lips) as well as 3D volumes were manually drawn for a part of the images.

Published
2019

Catalog

Books, media, physical & digital resources
See catalog results