Search

cote interne IRCAM: Caramiaux12a; None / None; National audience; This article presents a segmentation model applied to musician movements, taking into account different time structures. In particular we report on ancillary gestures that are not directly linked to sound production, whilst still being entirely part of the global instrumental gesture. Precisely, we study movements of the clarinet captured with an optical 3D motion capture system, analysing ancillary movements assuming that they can be considered as a sequence of primitive actions regarded as base shapes. A stochastic model called the Segmental Hidden Markov Model is used. It allows for the representation of a continuous trajectory as a sequence of primitive temporal profiles taken from a given dictionary. We evaluate the model using two criteria: the Euclidean norm and the log-likelihood, then show that the size of the dictionary does not influence the fitting accuracy and propose a method for building a dictionary based on the log-likelihood criterion. Finally, we show that the sequence of primitive shapes can also be considered as a sequence of symbols enabling us to interpret the data as symbolic patterns and motifs. Based on this representation, we show that circular patterns occur in all players' performances. This symbolic step produces a different layer of interpretation, linked to a larger time scale, which might not be obvious from a direct signal representation. more...

cote interne IRCAM: Bevilacqua12a; None / None; National audience; We present interdisciplinary research undertaken for the development of an ‘augmented’ string quartet. Hardware and software components were especially designed to enable mixed acoustic/electronic music where bow gestures drive digital sound processes. Precisely, inertial motion sensors and a bow force sensor were added to each musician's bow, and dedicated modules allowed forthe wireless data transmission to an on-line gesture analysis system. Prior to the performance, a research phase was performed to evaluate qualitatively the variability of the gesture data. Recording sessions of both gesture and audio data were carried out with a professional string quartet. The music material included a set of prototypical musical phrases containing various bowing styles and playing techniques as well as a complete music composition. The analysis of the recorded sessions allowed us to compare the consistency within and between players. While a given player was found to be generally consistent, the comparison between players revealed significant gesture idiosyncrasies. These results helped us to adapt a real-time gesture analysis system called the gesture follower. This tool was successful to automatically synchronize the live performance with electronic sound transformation in two concerts. A quantitative assessment is reported on a specific section of the piece, illustrating the accuracy and the types of errors encountered. more...

cote interne IRCAM: Einbond14a; None / None; National audience; The electronic musician’s toolkit is increasingly characterized by fluidity between software, techniques, and genres. By combining two of the most exciting recent packages for MAX, CATART corpus-based concatenative synthesis (CBCS) and BACH: AUTOMATED COMPOSER’S HELPER, we propose a rich tool for real-time creation, storage, editing, re-synthesis, and transcription of concatenative sound. The modular structures of both packages can be advantageously recombined to exploit the best of their real-time and computer-assisted composition (CAC) capabilities. After loading a sample corpus in CATART, each grain, or unit, played from CATART is stored as a notehead in the bach.roll object along with its descriptor data and granular synthesis parameters including envelope and spatialization. The data is attached to the note itself (pitch, velocity, duration) or stored in user-defined slots than can be adjusted by hand or batch-edited using lambda-loops. Once stored, the contents of bach.roll can be dynamically edited and auditioned using CATART for playback. The results can be output as a sequence for synthesis, or used for CAC score-generation through a process termed Corpus-Based Transcription: rhythms are output with bach.quantize and further edited in bach.roll before export as a MUSICXML file to a notation program to produce a performer-readable score. Together these techniques look toward a concatenative DAW with promising capabilities for composers, improvisers, installation artists, and performers. more...

cote interne IRCAM: Schwarz14b; None / None; National audience; We propose ways of enriching the timbral potential of gestural sonic material captured via piezo or contact microphones, through latency-free convolution of the microphone signal with grains from a sound corpus. This creates a new way to combine the sonic richness of large sound corpora, easily accessible via navigation through a timbral descriptor space, with the intuitive gestural interaction with a surface, captured by any contact microphone. We use convolution to excite the grains from the corpus via the microphone input, capturing the contact interaction sounds, which allows articulation of the corpus by hitting, scratching, or strumming a surface with various parts of the hands or objects. We also show how changes of grains have to be carefully handled, how one can smoothly interpolate between neighbouring grains, and finally evaluate the system against previous attempts. more...

cote interne IRCAM: Caramiaux14a; National audience; We investigated gesture description of sound stimuli performed during a listening task. Our hypothesis is that the strategies in gestural responses depend on the level of identification of the sound source and specifically on the identification of the action causing the sound. To validate our hypothesis, we conducted two experiments. In the first experiment, we built two corpora of sounds. The first corpus contains sounds with identifiable causal actions. The second contains sounds for which no causal actions could be identified. These corpora properties were validated through a listening test. In the second experiment, participants performed arm and hand gestures synchronously while listening to sounds taken from these corpora. Afterward, we conducted interviews asking participants to verbalize their experience while watching their own video recordings. They were questioned on their perception of the listened sounds and on their gestural strategies. We showed that for the sounds where causal action can be identified, participants mainly mimic the action that has produced the sound. In the other case, when no action can be associated with the sound, participants trace contours related to sound acoustic features. We also found that the interparticipants’ gesture variability is higher for causal sounds compared to noncausal sounds. Variability demonstrates that, in the first case, participants have several ways of producing the same action, whereas in the second case, the sound features tend to make the gesture responses consistent. more...

cote interne IRCAM: Caramiaux14c; None / None; National audience; Gesture-to-sound mapping is generally defined as the association between gestural and sound parameters. This article describes an approach that brings forward the perception–action loop as a fundamental design principle for gesture–sound mapping in digital music instrument. Our approach considers the processes of listening as the foundation—and the first step—in the design of action–sound relationships. In this design process, the relationship between action and sound is derived from actions that can be perceived in the sound. Building on previous work on listening modes and gestural descriptions, we propose to distinguish between three mapping strategies: instantaneous, temporal, and metaphorical. Our approach makes use of machine-learning techniques for building prototypes, from digital music instruments to interactive installations. Four different examples of scenarios and prototypes are described and discussed. more...

cote interne IRCAM: Sanlaville14a; None / None; International audience; Adaptation in an Interactive Model designed for Human Conversation and Music Improvisation: a preparatory outline

cote interne IRCAM: Cacciatore13a; None / None; National audience; Tamonontamo (2012) is a piece for amplified vocal quartet, choir of 24 singers and live electronics. The focus of this article is about the work made in collaboration with the Real-Time Music Interaction Team of Ircam. Augustin Muller, computer music programmer working with me at Ircam for this project, upgraded the CataRT software adding a Spat~ module to relate the corpus-based synthesis with the spatialization. In most of works the spatialization of sounds depends from the aesthetic choose of the composer. The Spat~ software allows the drawing in the space of linear movements of a source among a pre-selected number of loudspeakers with a user-friendly graphical interface. In this patch, we used Spat to spatialize the sounds in a non-linear way. The logic of spatialization depends on the pair of audio descriptors chosen in the set-up of the CataRT graphical display. In this sense, the aesthetic ideas of the composer refer not locally to each sound but on the choosing of the audio descriptors used for the x, y axis. A further Spat-related display has been implemented to split the space of analysis/playing from the space of diffusion of sounds. The implementation of the Unispring algorithm in the mnm.distribute external Max/MSP object allows the distribution of grains rescaling their position inside a pre-drawn sub-space. The interpolation between different shapes or the change of audio descriptors on the x, y axis for both displays can be programmed and made in real time. The storage of the synthesis as database (in a text file), allows the possibility to recall the analysis and recover the shapes drawn previously. Hiding and permanent deletion of some selected corpora in the database is also possible. The use of colour to work on the z axis is possible and is among the next step of this work. The upgrade of the use of Spat~ to an ambisonics system for diffusion of sounds represents the possibility to make—through the concatenative synthesis—a genuine work of 3D-sound-sculpting. more...

cote interne IRCAM: Schwarz13b; None / None; National audience; Corpus-based concatenative synthesis (CBCS) is a recent sound synthesis method, based on descriptor analysis of any number of existing or live-recorded sounds, and synthesis by selection of sound segments from the database matching sound characteristics given by the user. While the previous phase of research focused mainly on the analysis and synthesis methods, and handling of the corpus, current research now turns more and more towards how expert musicians, designers, the general public, or children can interact efficiently and creatively with a corpus of sounds. We'll look at gestural control of the navigation through the sound space, how it can be controlled by audio input to transform sound in surprising ways, or to transcribe and re-orchestrate environmental sound, and at its use for live performance, especially in an improvisation setting between a instrumental and a CBCS performer, where recording the corpus live from the instrument creates a stronger and more direct coupling between them, and lastly sound installations that open up discovery and interaction with rich sound corpora to the general public and children. more...

cote interne IRCAM: Bevilacqua13c; None / None; National audience; Sonification of the arm movement might be beneficial for rehabilitation of stroke patients. This implies to provide patients with auditory feedback relative to the coordination between the shoulder and elbow movements, and relative to the motion smoothness. To this purpose, we are exploring different types of sonification and musical metaphors, including source-filter, concatenative-granular and physical models sound synthesis. more...

cote interne IRCAM: Boyer14a; National audience; This study introduces an experiment designed to analyze the sensorimotor adaptation to a motion-based sound synthesis system. We investigated a sound-oriented learning task, namely to reproduce a tar- geted sound. The motion of a small handheld object was used to control a sound synthesizer. The object angular velocity was measured by a gy- roscope and transmitted in real time wirelessly to the sound system. The targeted sound was reached when the motion matched a given refer- ence angular velocity profile with a given accuracy. An incorrect velocity profile produced either a noisier sound or a sound with a louder high har- monic, depending on the sign of the velocity error. The results showed that the participants were generally able to learn to reproduce sounds very close to the targeted sound. A corresponding motor adaptation was also found to occur, at various degrees, in most of the participants when the profile is altered. more...

cote interne IRCAM: Houix13a; None / None; National audience; The Legos project aims at studying sound gesture relationship. During the project, two participatory workshops related to sonic interaction design have been organized. The aim of the workshops was to stimulate ideas related to the control of everyday objects using sound, and then, to create and experiment with new sonic augmented objects. The present article provides an overview of the two workshops. The methodology we used is based on industrial design process of new product creation and theoretical issues were discussed. The rst workshop focused on the analyze and the possible soni cation of everyday objects. New scenarios of use were obtained and tested. The second workshop focused on sound metaphor, questioning the relationship between sound and gesture using the concept of basic actions and the work on the sound synthesis engines. During the two workshops, experiments using sensors and real-time synthesis were carried on a selection of case studies. more...

cote interne IRCAM: Boyer13e; None / None; National audience; Auditory feedback can be beneficial to motor control, however the precise mechanisms that enable sound features to influence movements remain poorly understood. We propose an experiment to evaluate whether continuous auditory feedback can improve tracking performance. Different types of sonification were assessed. more...

cote interne IRCAM: Boyer13b; None / None; National audience; We report on an experiment designed to evaluate sensorimo- tor adaptation in a motion-based sound synthesis system. We propose to investigate a sound-oriented task, namely to reproduce a targeted sound. The system is a small handheld object whose motion drives a sound syn- thesizer. In particular, the angular velocity is captured in real-time by a gyroscope and transmitted wirelessly to the sound engine. The targeted sound is obtained when the motion matches a given reference velocity profile. If the velocity profile is performed with sufficient accuracy, a specific sound is heard, while a incorrect velocity profile produces either noisier sound or sound with a louder high harmonic (depending on lower or higher velocity values). The results show that subjects are generally able to learn to reproduce the target sound. Motor adaptation is also found to occur, at various degrees, in most subjects when the profile is altered. more...

This SIG intends to investigate the ongoing dialogue between music technology and the field of human-computer interaction. Our specific aims are to consider major findings of musical interface research over recent years and discuss how these might best be conveyed to CHI researchers interested but not yet active in this area, as well as to consider how to stimulate future collaborations between music technology and CHI research communities. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in CHI EA '13 Extended Abstracts in Human Factors in Computing Systems. more...

Chiseling Bodies is an interactive augmented dance performance, where a dancer interacts with abstract visuals. They are massive mass-spring systems whose dynamical behaviors are echoing the dancer's movement qualities. more...

cote interne IRCAM: Savary13a; National audience; Dirty Tangible Interfaces (DIRTI) are a new concept in interface design that forgoes the dogma of repeatability in favor of a richer and more complex experience, constantly evolving, never reversible, and infinitely modifiable. We built a prototype interface realizing the DIRTI principles based on low-cost commodity hardware and kitchenware: A video camera tracks a granular or liquid interaction material placed in a glass dish. The 3D relief estimated from the images, and the dynamic changes applied to it by the user(s), are used to control two applications: For 3D scene authoring, the relief is directly translated into a terrain, allowing fast and intuitive map editing. For expressive audio-graphic music performance, both the relief and real-time changes are interpreted as activation profiles to drive corpus-based concatenative sound synthesis, allowing one or more musicians to mold sonic landscapes and to plow through them in an inherently collaborative, expressive, and dynamic experience. more...

cote interne IRCAM: Furfaro13a; None / None; National audience; Enrico Furfaro, Nadia Berthouze, Frédéric Bevilacqua and Ana Tajadura-Jiménez

cote interne IRCAM: Schwarz13a; None / None; National audience; We present a way to make environmental recordings controllable again by the use of continuous annotations of the high-level semantic parameter one wishes to control, e.g. "wind strength" or "crowd excitation level". The annotations serve as a descriptor in corpus-based concatenative synthesis. The workflow has been evaluated by a preliminary subject test and first results on the canonical correlation analysis show high consistency between annotations and a small set of audio descriptors being well correlated with them. more...

cote interne IRCAM: Boyer13d; National audience; We present a motion capture device to measure in real-time table tennis strokes. A six degree-of-freedom sensing device, inserted into the racket handle, measures 3D acceleration and 3-axis angular velocity values at a high sampling rate. Data are wirelessly transmitted to a computer in real-time. This flexible system allows for recording and analyzing kinematics information on the motion of the racket, along with synchronized video and sound recordings. Recorded gesture data are analyzed using several algorithms we developed to segment and extract movement features, and to build a reference motion database. more...

Gesture-based interaction is widespread in touch screen interfaces. The goal of this paper is to tap the richness of expressive variation in gesture to facilitate continuous interaction. We achieve this through novel techniques of adaptation and estimation of gesture characteristics. We describe two experiments. The first aims at understanding whether users can control certain gestural characteristics and if that control depends on gesture vocabulary. The second study uses a machine learning technique based on particle filtering to simultaneously recognize and measure variation in a gesture. With this technology, we create a gestural interface for a playful photo processing application. From these two studies, we show that 1) multiple characteristics can be varied independently in slower gestures (Study 1), and 2) users find gesture-only interaction less pragmatic but more stimulating than traditional menu-based systems (Study 2). more...

cote interne IRCAM: Schwarz12b; None / None; National audience; We present a method, applicable to corpus-based concatenative synthesis and specifically to audio mosaicing, that assists the composer in exploring the relationship between the parameterization of a concatenative algorithm and the resulting similarity between the output sound and the original target soundfile. Rather than focus solely on straightforward imitation, our work is predicated upon the notion that similarity can be manifest in a variety of perceptually meaningful ways and that both semblance and dissemblance have compositional utility. Our method consists of visualizing a collection of concatenated outputs, each of which is a unique solution to the problem of matching the same target soundfile with the same sound database but using a different combination of descriptor weights. We create a solution space where the location of each output is modeled by its similarity to the target as well as its similarity to each other solution. Visualization and navigation of this space is made possible through a multi-dimensional scaling algorithm, permitting 2D browsing, aural feedback, and the composition of paths through the solution space. This meta-control framework helps to give the composer a more comprehensive understanding of concatenative potential. By arranging concatenated outputs into different regions of similarity and dissimilarity, the solution space provides a rich and expansive terrain for compositional exploration and discovery. more...

cote interne IRCAM: Einbond12a; None / None; National audience; The need for fine-tuned microtonal pitch combined with the timbral richness of corpus-based concatenative synthesis has led to the development of a new tool for corpus-based pitch and loudness control in real time with CataRT. Drawing on recent research in feature modulation synthesis (FMS) as well as the bach library for MAX/MSP, we have implemented a set of new modules for CataRT that permit the user to define microtonal har- monies graphically and combine them with other audio descriptors to trigger concatenative synthesis in real or deferred time. Pitch information is generated from a pitch analysis or extracted from soundfile meta-data, and loudness may be controlled independently for different sound sets. Musical implementations already suggest promising results as well as future goals to generalize this approach to further timbral features for corpus-based FMS. more...

cote interne IRCAM: Lallemand12a; None / None; National audience; Query by example retrieval of environmental sound recordings is a research area with applications to sound design, music composition and automatic suggestion of metadata for the labeling of sound databases. Retrieval problems are usually composed of successive feature extraction (FE) and similarity measurement (SM) steps, in which a set of extracted features encoding important properties of the sound recordings are used to compute the distance between elements in the database. Previous research has pointed out that successful features in the domains of speech and music, like MFCCs, might fail at describing environmental sounds, which have intrinsic variability and noisy characteristics. We present a set of novel multiresolution features obtained by modeling the distribution of wavelet subband coefficients with generalized Gaussian densities (GGDs). We define the similarity measure in terms of the Kullback-Leibler divergence between GGDs. Experimental results on a database of 1020 environmental sound recordings show that our approach always outperforms a method based on traditional MFCC features and Euclidean distance, improving retrieval rates from 51% to 62%. more...

cote interne IRCAM: FdiliAlaoui12b; None / None; National audience; In this paper, we explore the use of movement qualities as interaction modality. The notion of movement qualities is widely used in dance practice and can be understood as how the movement is performed, independently of its specific trajectory in space. We implemented our approach in the context of an artistic installation called A light touch. This installation invites the participant to interact with a moving light spot reacting to the hand movement qualities. We conducted a user experiment that showed that such an interaction based on movement qualities tends to enhance the user experience favouring explorative and expressive usage. more...

cote interne IRCAM: Kimura12a; National audience; As a 2010 Artist in Residence in Musical Research at IRCAM, Mari Kimura used the Augmented Violin to develop new compositional approaches, and new ways of creating interactive performances. She contributed her empirical and historical knowledge of violin bowing technique, working with the RealTime Musical Interactions Team at IRCAM. Thanks to this residency, her ongoing long-distance collaboration with the team since 2007 dramatically accelerated, and led to solving several compositional and calibration issues of the GestureFollower (GF). Kimura was also the first artist to develop projects between the two teams at IRCAM, using OMAX (Musical Representation Team) with GF. In the past year, the performance with Augmented Violin has been expanded in larger scale interactive audio/visual projects as well. In this paper, we report on the various techniques developed for the Augmented Violin and compositions by Kimura using them, offering specific examples and scores more...

cote interne IRCAM: Savary12a; National audience; Dirty Tangible Interfaces (DIRTI) are a new concept in interface design that forgoes the dogma of repeatability in favor of a richer and more complex experience, constantly evolving, never reversible, and infinitely modifiable. We built a prototype based on granular or liquid interaction material placed in a glass dish, that is analyzed by video tracking for its 3D relief. This relief, and the dynamic changes applied to it by the user, are interpreted as activation profiles to drive corpus-based concatenative sound synthesis, allowing one or more players to mold sonic landscapes and to plow through them in an inherently collaborative, expressive, and dynamic experience. more...

cote interne IRCAM: Schwarz12a; None / None; National audience; Corpus-based concatenative synthesis is a fairly recent sound synthesis method, based on descriptor analysis of any number of existing or live-recorded sounds, and synthesis by selection of sound segments from the database matching given sound characteristics. It is well described in the literature, but has been rarely examined for its capacity as a new interface for musical expression. The interesting outcome of such an examination is that the actual instrument is the space of sound characteristics, through which the performer navigates with gestures captured by various input devices. We will take a look at different types of interaction modes and controllers (positional, inertial, audio analysis) and the gestures they afford, and provide a critical assessment of their musical and expressive capabilities, based on several years of musical experience, performing with the CataRT system for real-time CBCS. more...

cote interne IRCAM: Rasamimanana12a; None / None; National audience; We present Urban Musical Game, an installation using augmented sports balls to manipulate and transform an interactive music environment. The interaction is based on playing techniques, a concept borrowed from traditional music instruments and applied here to non musical objects. more...

cote interne IRCAM: Bianco12b; None / None; National audience; Trumpet performance builds on the synergy of multiple factors. From a physiological point of view, the production of a simple musical tone relies on the control of the pressure at the lips, the contact force on the mouthpiece, and the oral articulatory gestures and configurations. Whereas previous studies described the magnitude of these control parameters in quasi-static regime of oscillation, the present investigation focuses on their temporal evolution and interrelation. Intra-oral pressure, mouthpiece force, facial muscular activity of two group of muscles, and the radiated sound were recorded from three professional players performing isolated and articulated tones with three different dynamics. Results showed that the experimental conditions affected the amount and location of variability in the performance, and the behavior of intraoral pressure at note initiation. All together, this suggests that different dynamics and sequencing may cause a reorganization of the mechanisms underlying the production of a tone. more...

cote interne IRCAM: Schwarz11d; None / None; National audience; Live corpus-based concatenative synthesis permits a new approach to improvisation, where sound from an instrument is recontextualised by interactive, gesture-controlled software, creating a symbiosis between the two performers. more...

cote interne IRCAM: Schwarz11c; National audience; The synthesis of sound textures, such as rain, wind, or crowds, is an important application for cinema, multimedia creation, games and installations. However, despite the clearly defined requirements of naturalness and flexibility, no automatic method has yet found widespread use. After clarifying the definition, terminology, and usages of sound texture synthesis, we will give an overview of the many existing methods and approaches, and the few available software implementations, and classify them by the synthesis model they are based on, such as subtractive or additive synthesis, granular synthesis, corpus-based concatenative synthesis, wavelets, or physical modeling. Additionally, an overview is given over analysis methods used for sound texture synthesis, such as segmentation, statistical modeling, timbral analysis, and modeling of transitions. more...

cote interne IRCAM: Lallemand11a; None / None; National audience; Interactive navigation within geometric, feature-based database representations allows expressive musical performances and installations. Once mapped to the feature space, the user’s position in a physical interaction setup (e.g. a multitouch tablet) can be used to select elements or trigger audio events. Hence physical displacements are directly connected to the evolution of sonic characteristics—a property we call analytic sound–control correspondence. However, automatically computed representations have a complex geometry which is unlikely to fit the interaction setup optimally. After a review of related work, we present a physical model-based algorithm that redistributes the representation within a user-defined region according to a user-defined density. The algorithm is designed to preserve the analytic sound-control correspondence property as much as possible, and uses a physical analogy between the triangulated database representation and a truss structure. After preliminary pre-uniformisation steps, internal repulsive forces help to spread points across the whole region until a target density is reached. We measure the algorithm performance relative to its ability to produce representations corresponding to user-specified features and to preserve analytic sound–control correspondence during a standard density-uniformisation task. Quantitative measures and visual evaluation outline the excellent performances of the algorithm, as well as the interest of the pre-uniformisation steps. more...

cote interne IRCAM: Dessein11c; None / None; National audience; Applications of information geometry to audio signal processing

cote interne IRCAM: Bonardi11a; None / None; National audience; Ce chapitre rend compte des recherches autour d'un assistant virtuel de metteur en scène et performeur dans le cadre du spectacle théâtral "Les Petites Absences" créé à la Comédie de Caen en 2008. more...

cote interne IRCAM: Schwarz11a; None / None; National audience; In the most common approach to corpus-based concatenative synthesis, the unit selection takes places as a content-based similarity match based on a weighted Euclidean distance between the audio descriptors of the database units, and the synthesis target. While the simplicity of this method explains the relative success of CBCS for interactive descriptor-based granular synthesis — especially when combined with a graphical interface — and audio mosaicing, and still allows to express categorical matches, certain desirable constraints can not be formulated, such as disallowing repetition of units, matching a disjunction of descriptor ranges, or asymmetric distances. We therefore propose a new method of mapping the individual signed descriptor distances by a warping function that can express these criteria, while still being amenable to efficient multi-dimensional search indices like the kD-tree, for which we define the preconditions and cases of applicability. more...

cote interne IRCAM: Schwarz11b; None / None; National audience; This paper shows methods for defining and rendering Sound Level of Detail (SLOD) for audiographic scenes using corpus-based granular synthesis. We introduce three levels of detail for sound (individual events, statistical texture, background din), to define a proximity profile around the listener. The smooth transition between the levels is assured alternatively by statistical modeling or audio impostors. The activation of the three levels is controlled by invisible and editable profile objects mapped to presets of audio process parameters. These also serve to balance the CPU load between the different audio processes. We have tested this method on various virtual scenes such as crowds, rain, foliage, or traffic. more...

cote interne IRCAM: Caramiaux11a; None / None; National audience; In this paper we present an experimental study concerning gestural embodiment of environmental sounds in a listening context. The presented work is part of a project aiming at modeling movement-sound relationships, with the end goal of proposing novel approaches for designing musical instruments and sounding objects. The experiment is based on sound stimuli corresponding to ``causal'' and ``non-causal'' sounds. It is divided into a performance phase and an interview. The experiment is designed to investigate possible correlation between the perception of the ``causality'' of environmental sounds and different gesture strategies for the sound embodiment. In analogy with the perception of the sounds' causality, we propose to distinguish gestures that ``mimic'' a sound's cause and gestures that ``trace'' a sound's morphology following temporal sound characteristics. Results from the interviews show that, first, our causal sounds database lead to consistent descriptions of the action at the origin of the sound and participants mimic this action. Second, non-causal sounds lead to inconsistent metaphoric descriptions of the sound and participants make gestures following sound ``contours''. Quantitatively, the results show that gesture variability is higher for causal sounds that non-causal sounds. more...

cote interne IRCAM: Ding11a; None / None; National audience; In audio–graphic scenes, visual and audio modalities are synchronized in time and space, and their behaviour is determined by a common process. We present here a novel way of modeling audio–graphic content for interactive 3D scenes with the concept of sound processes and their activation through 2D or 3D profiles. Many 3D applications today support both graphical and audio effects to provide a more realistic user experience; however a common model and interchange format for interactive audio–graphic scenes is still lacking. X3D is one of the most promising formats for 3D scene representation. It is extensible and supports simple spatial audio representation and almost all basic and advanced 3D computer graphics techniques. We therefore propose an extension of the X3D standard to represent the sound process and activation profile model for providing a rich audio–graphic description in X3D. more...

cote interne IRCAM: Flety11a; None / None; National audience; We present a strategy for the improvement of wireless sensor data transmission latency, implemented in two current projects involving gesture/control sound interaction. Our platform was designed to be capable of accepting accessories using a digital bus. The receiver features a IEEE 802.15.4 microcontroller associated to a TCP/IP stack integrated circuit that transmits the received wireless data to a host computer using the Open Sound Control protocol. This paper details how we improved the latency and sample rate of the said technology while keeping the device small and scalable. more...

cote interne IRCAM: Caramiaux11b; None / None; National audience; This paper presents a prototypical tool for sound selection driven by users' gestures. Sound selection by gestures is a particular case of "query by content" in multimedia databases. Gesture-to-Sound matching is based on computing the similarity between both gesture and sound parameters' temporal evolution. The tool presents three algorithms for matching gesture query to sound target. The system leads to several applications in sound design, virtual instrument design and interactive installation. more...

cote interne IRCAM: FdiliAlaoui11a; None / None; National audience; In this paper we address the question of extending user experience in large scale tactile displays. Our contribution is a non task-oriented interaction technique based on modern dance for the creation of aesthetically pleasant large scale tactile interfaces. This approach is based on dance movement qualities applied to touch interaction allowing for natural gestures in large touch displays. We used specific movements from a choreographic glossary and developed a robust movement quality recognition process. To illustrate our approach, we propose a media installation called A light touch, where touch is used to control a light spot reacting to movement qualities. more...

cote interne IRCAM: Zamborlin11b; National audience; (land)moves is an interactive installation: the user’s gestures control the multimedia processing with a total synergy between audio and video synthesis and treatment. more...

cote interne IRCAM: Zamborlin11b; National audience; (land)moves is an interactive installation: the user’s gestures control the multimedia processing with a total synergy between audio and video synthesis and treatment. more...

cote interne IRCAM: Zamborlin11b; National audience; (land)moves is an interactive installation: the user’s gestures control the multimedia processing with a total synergy between audio and video synthesis and treatment. more...

cote interne IRCAM: Cont11c; None / None; International audience; Applications de la géométrie de l'information au traitement des flux audio temps-réel

cote interne IRCAM: Bevilacqua11a; None / None; National audience; Gesture capture: Paradigms in interactive music/dance systems

1 Abstract. This chapter presents a general framework for gesture-controlled audio processing. The gesture parameters are assumed to be multi-dimensional temporal profiles obtained from movement or sound capture systems. The analysis is based on machine learning techniques, comparing the incoming dataflow with stored templates. The mapping procedures between the gesture and the audio processing include a specific method we called temporal mapping. In this case, the temporal evolution of the gesture input is taken into account in the mapping process. We describe an example of a possible use of the framework that we experimented with in various contexts, including music and dance performances, music pedagogy and installations. more...

cote interne IRCAM: Guedy11a; None / None; National audience; L’expérimentation musicale et pédagogique sur le geste de l’Atelier des Feuillantines avec l’équipe Interaction Musicale Temps Réel de l’Ircam more...

cote interne IRCAM: Schnell11a; None / None; National audience; Playing and Replaying – Sound, Gesture and Music Analysis and Re-Synthesis for the Interactive Control and Re-Embodiment of Recorded Music more...