Search

Abstract Playing music is a complex task that relies on the combination of musicians’ technical and expressive skills. While the literature has investigated the effects of musical expressivity on the listeners, the way how technical difficulty and emotional expressivity affect musicians during playing has surprisingly received no attention. In an attempt to fill this gap in the literature, we collected behavioral and physiological data from twelve violinists playing 29 pieces that included both technical exercises and excerpts from classical repertoire for violin. After playing each stimulus, participants rated it for emotional expressivity and technical difficulty. During the entire session, cardiac parameters and electrodermal activity were collected, together with the kinematic parameters of the bowing gesture. A set of Linear Mixed-Effect (LME) models suggested that emotional expressivity attributed to music affected the fluidity of bowing (i.e., smoothness), with the excerpts rated as more expressive being performed in a less smooth way. In contrast, LME models revealed no effects of expressivity and technical difficulty on any of the physiological parameters of violinists. Our results offer novel insights into the psychophysiological dynamics that link motor parameters with musical expressivity. These findings could influence educational practices in music and deepen our understanding of aesthetic emotions.

The minimal, essential condition for individuals to interact is that they exchange information via at least one sensory channel. Once informational coupling is established, it enables basic forms of coordinated behavior to spontaneously emerge from the interaction. Our previous study revealed different coordination dynamics in dyads engaged in a joint finger-tapping task based on visual versus auditory coupling. This observation led us to propose the ‘modality-dependent hypothesis', which posits that coordination dynamics are influenced by the sensory modality mediating informational coupling. However, recognizing that different modalities have inherent differences in accessing spatiotemporal features of perceived movement, we formulated the alternative ‘kinematic hypothesis'. This hypothesis posits that differences in dynamics would vanish given equivalent kinematic information across modalities. The study involved forty (N = 40) participants, grouped into twenty (N = 20) dyads, who engaged in a joint finger-tapping task. This task was conducted under varying conditions of visual and auditory coupling, with manipulations in the access to kinematic information, categorized as discrete and continuous. Contrary to our initial predictions, the results strongly supported the ‘modality-dependent hypothesis'. We observed that visual and auditory coupling consistently yielded distinct attractor dynamics, regardless of the access to kinematic information. Furthermore, all conditions of auditory coupling resulted in higher levels of synchronization than their visual counterparts. These findings suggest that the differences in interpersonal synchronization are predominantly influenced by the sensory modality, rather than the continuity of kinematic information. Our study highlights the significance of sensorimotor interactions in interpersonal synchronization and addresses the potential of sonification strategies in supporting motor training and rehabilitation.

Abstract Sensorimotor synchronization strategies have been frequently used for gait rehabilitation in different neurological populations. Despite these positive effects on gait, attentional processes required to dynamically attend to the auditory stimuli needs elaboration. Here, we investigate auditory attention in neurological populations compared to healthy controls quantified by EEG recordings. Literature was systematically searched in databases PubMed and Web of Science. Inclusion criteria were investigation of auditory attention quantified by EEG recordings in neurological populations in cross-sectional studies. In total, 35 studies were included, including participants with Parkinson’s disease (PD), stroke, Traumatic Brain Injury (TBI), Multiple Sclerosis (MS), Amyotrophic Lateral Sclerosis (ALS). A meta-analysis was performed on P3 amplitude and latency separately to look at the differences between neurological populations and healthy controls in terms of P3 amplitude and latency. Overall, neurological populations showed impairments in auditory processing in terms of magnitude and delay compared to healthy controls. Consideration of individual auditory processes and thereafter selecting and/or designing the auditory structure during sensorimotor synchronization paradigms in neurological physical rehabilitation is recommended.

In this perspective paper, we explore the use of haptic feedback to enhance human-human interaction during musical tasks. We start by providing an overview of the theoretical foundation that underpins our approach, which is rooted in the embodied music cognition framework, and by briefly presenting the concepts of action-perception loop, sensorimotor coupling and entrainment. Thereafter, we focus on the role of haptic information in music playing and we discuss the use of wearable technologies, namely lightweight exoskeletons, for the exchange of haptic information between humans. We present two experimental scenarios in which the effectiveness of this technology for enhancing musical interaction and learning might be validated. Finally, we briefly discuss some of the theoretical and pedagogical implications of the use of technologies for haptic communication in musical contexts, while also addressing the potential barriers to the widespread adoption of exoskeletons in such contexts.

Abstract Music performance requires high levels of motor control. Professional musicians use body movements not only to accomplish and help technical efficiency, but to shape expressive interpretation. Here, we recorded motion and audio data of twenty participants performing four musical fragments varying in the degree of technical difficulty to analyze how knee flexion is employed by expert saxophone players. Using a computational model of the auditory periphery, we extracted emergent acoustical properties of sound to inference critical cognitive patterns of music processing and relate them to motion data. Results showed that knee flexion is causally linked to tone expectations and correlated to rhythmical density, suggesting that this gesture is associated with expressive and facilitative purposes. Furthermore, when instructed to play immobile, participants tended to microflex (>1 Hz) more frequently compared to when playing expressively, possibly indicating a natural urge to move to the music. These results underline the robustness of body movement in musical performance, providing valuable insights for the understanding of communicative processes, and development of motor learning cues.

This dataset comprises motion capture, audio, and questionnaire data from violinists who underwent four augmented reality training sessions spanning a month. The motion capture data was meticulously recorded using a 42-marker Qualisys Animation marker set, capturing movement at a high rate of 120 Hz. Audio data was captured using two condenser microphones, boasting a bit depth of 24 and a sampling rate of 48 kHz. The dataset encompasses recordings from 2 violin orchestra section leaders and 11 participants.Initially, we collected motion capture (MoCap) and audio data from the section leaders, who performed 2 distinct musical pieces. These recordings were then utilized to create 2 avatars, each representing a section leader and their respective musical piece. Subsequently, each avatar was assigned to a group of violinists, forming groups of 5 and 6 participants. Throughout the experiment, participants rehearsed one piece four times using a 2D representation of the avatar, and the other piece four times using a 3D representation.During the practice sessions, participants were instructed to meticulously replicate the avatar's bowing techniques, encompassing gestures related to bowing, articulation, and dynamics. For each trial, we collected motion capture, audio data, and self-reported questionnaires from all participants. The questionnaires included the Witmer presence questionnaire, a subset of the Makransky presence questionnaire, the sense of musical agency questionnaire, as well as open-ended questions for participants to express their thoughts and experiences.Additionally, participants completed the Immersive Tendencies questionnaire, the Music Sophistication Index questionnaire, and provided demographic information before the first session commenced.

Summary: Humans exhibit a strong tendency to synchronize movements with each other, with visual perspective potentially influencing interpersonal synchronization. By manipulating the visual scenes of participants engaged in a joint finger-tapping task, we examined the effects of 1st person and 2nd person visual perspectives on their coordination dynamics. We hypothesized that perceiving the partner’s movements from their 1st person perspective would enhance spontaneous interpersonal synchronization, potentially mediated by the embodiment of the partner’s hand. We observed significant differences in attractor dynamics across visual perspectives. Specifically, participants in 1st person coupling were unable to maintain de-coupled trajectories as effectively as in 2nd person coupling. Our findings suggest that visual perspective influences coordination dynamics in dyadic interactions, engaging error-correction mechanisms in individual brains as they integrate the partner’s hand into their body representation. Our results have the potential to inform the development of applications for motor training and rehabilitation.

Neural entrainment, defined as unidirectional synchronization of neural oscillations to an external rhythmic stimulus, is a topic of major interest in the field of neuroscience. Despite broad scientific consensus on its existence, on its pivotal role in sensory and motor processes, and on its fundamental definition, empirical research struggles in quantifying it with non-invasive electrophysiology. To this date, broadly adopted state-of-the-art methods still fail to capture the dynamic underlying the phenomenon.Here, we present event-related frequency adjustment (ERFA) as a methodological framework to induce and to measure neural entrainment in human participants, optimized for multivariate EEG datasets. By applying dynamic phase and tempo perturbations to isochronous auditory metronomes during a finger-tapping task, we analyzed adaptive changes in instantaneous frequency of entrained oscillatory components during error correction. Spatial filter design allowed us to untangle, from the multivariate EEG signal, perceptual and sensorimotor oscillatory components attuned to the stimulation frequency.Both components dynamically adjusted their frequency in response to perturbations, tracking the stimulus dynamics by slowing down and speeding up the oscillation over time. Source separation revealed that sensorimotor processing enhanced the entrained response, supporting the notion that the active engagement of the motor system plays a critical role in processing rhythmic stimuli. In the case of phase shift, motor engagement was a necessary condition to observe any response, whereas sustained tempo changes induced frequency adjustment even in the perceptual oscillatory component. Although the magnitude of the perturbations was controlled across positive and negative direction, we observed a general bias in the frequency adjustments towards positive changes, which points at the effect of intrinsic dynamics constraining neural entrainment.We conclude that our findings provide compelling evidence for neural entrainment as mechanism underlying overt sensorimotor synchronization, and highlight that our methodology offers a paradigm and a measure for quantifying its oscillatory dynamics by means of non-invasive electrophysiology, rigorously informed by the fundamental definition of entrainment.

Within the broader context of improving interactions between artificial intelligence and humans, the question has arisen regarding whether auditory and rhythmic support could increase attention for visual stimuli that do not stand out clearly from an information stream. To this end, we designed an experiment inspired by pip-and-pop but more appropriate for eliciting attention and P3a-event-related potentials (ERPs). In this study, the aim was to distinguish between targets and distractors based on the subject’s electroencephalography (EEG) data. We achieved this objective by employing different machine learning (ML) methods for both individual-subject (IS) and cross-subject (CS) models. Finally, we investigated which EEG channels and time points were used by the model to make its predictions using saliency maps. We were able to successfully perform the aforementioned classification task for both the IS and CS scenarios, reaching classification accuracies up to 76%. In accordance with the literature, the model primarily used the parietal–occipital electrodes between 200 ms and 300 ms after the stimulus to make its prediction. The findings from this research contribute to the development of more effective P300-based brain–computer interfaces. Furthermore, they validate the EEG data collected in our experiment.

Several studies have addressed motor coordination in dance, but few have addressed the influence of musical context on micro-timing during sensorimotor synchronization (SMS) in classical ballet. In this study, we analyze the Promenade in Arabesque of the Odile variations, first as a dance-music fragment non-embedded in a musical context, then as a dance-music fragment embedded in a musical context at two different instances. Given the musical structure of the fragments, there are repeats of patterns between and within the fragments. Four dancers were invited to perform the three fragments in twelve successive performances. The beats of the music were extracted and compared with the timing of the dancers' heel movements, using circular-linear smooth regression modelling, and circular statistics. The results reveal an effect of repeat within fragments, and an effect of musical context between fragments, on micro-timing anticipation in SMS. The methodology offers a framework for future work on dynamical aspects of SMS.

Abstract Rhythmic joint coordination is ubiquitous in daily-life human activities. In order to coordinate their actions towards shared goals, individuals need to co-regulate their timing and move together at the collective level of behavior. Remarkably, basic forms of coordinated behavior tend to emerge spontaneously as long as two individuals are exposed to each other’s rhythmic movements. The present study investigated the dynamics of spontaneous dyadic entrainment, and more specifically how they depend on the sensory modalities mediating informational coupling. By means of a novel interactive paradigm, we showed that dyadic entrainment systematically takes place during a minimalistic rhythmic task despite explicit instructions to ignore the partner. Crucially, the interaction was organized by clear dynamics in a modality-dependent fashion. Our results showed highly consistent coordination patterns in visually-mediated entrainment, whereas we observed more chaotic and more variable profiles in the auditorily-mediated counterpart. The proposed experimental paradigm yields empirical evidence for the overwhelming tendency of dyads to behave as coupled rhythmic units. In the context of our experimental design, it showed that coordination dynamics differ according to availability and nature of perceptual information. Interventions aimed at rehabilitating, teaching or training sensorimotor functions can be ultimately informed and optimized by such fundamental knowledge.

In musical instrument training, researchers have gradually started exploring the potential of interactive technologies supporting learning processes and teaching methods. Although numerous technological tools have been proposed to enhance instrument learning and teaching, these methods rarely find their way into daily practice and teaching routines. In this study, we report data from an online survey study administered to violin and drum kit teachers. Results reveal distinct learning profiles of novice violin and drum kit students and exhibit a variety of teaching approaches toward adults and children. Furthermore, they provide more insight into teachers’ opinions on the use of virtual reality (VR) and smart wearable technologies in early instrumental training, as well as their attitudes regarding technology design. Overall, our findings highlight the importance of involving teachers in the initial stages of technology design to facilitate technology acceptance and adoption, prevent potential mismatches between requirements regarding technological functionality and actual user needs, and promote musical growth and skill acquisition.

Across a broad spectrum of interactions, humans exhibit a prominent tendency to synchronize their movements with one another. Traditionally, this phenomenon has been explained from the perspectives of predictive coding or dynamical systems theory. While these theories diverge with respect to whether individuals hold internal models of each other, they both assume a predictive or anticipatory mechanism enabling rhythmic interactions. However, the neural bases underpinning interpersonal synchronization are still a subject under active investigation. Here we provide evidence that the brain relies on a common oscillatory mechanism to pace self-generated rhythmic movements and to track the movements produced by a partner. By performing dual-electroencephalography recordings during a joint finger-tapping task, we identified an oscillatory component in the beta range (∼ 20 Hz), which was significantly modulated by both self-generated and other-generated movement. In conditions where the partners perceived each other, we observed periodic fluctuations of beta power as a function of the reciprocal movement cycles. Crucially, this modulation occurred both in visually and in auditorily coupled conditions, and was accompanied by recurrent periods of dyadic synchronized behavior. Our results show that periodic beta power modulations may be a critical mechanism underlying interpersonal synchronization, possibly enabling mutual predictions between coupled individuals, leading to co-regulation of timing and overt mutual adaptation. Our findings thus provide a potential bridge between influential theories attempting to explain interpersonal coordination, and a concrete connection to its neurophysiological bases.

Abstract Methods to reduce impact in distance runners have been proposed based on real-time auditory feedback of tibial acceleration. These methods were developed using treadmill running. In this study, we extend these methods to a more natural environment with a proof-of-concept. We selected ten runners with high tibial shock. They used a music-based biofeedback system with headphones in a running session on an athletic track. The feedback consisted of music superimposed with noise coupled to tibial shock. The music was automatically synchronized to the running cadence. The level of noise could be reduced by reducing the momentary level of tibial shock, thereby providing a more pleasant listening experience. The running speed was controlled between the condition without biofeedback and the condition of biofeedback. The results show that tibial shock decreased by 27% or 2.96 g without guided instructions on gait modification in the biofeedback condition. The reduction in tibial shock did not result in a clear increase in the running cadence. The results indicate that a wearable biofeedback system aids in shock reduction during over-ground running. This paves the way to evaluate and retrain runners in over-ground running programs that target running with less impact through instantaneous auditory feedback on tibial shock.

In a century where humans and machines—powered by artificial intelligence or not—increasingly work together, it is of interest to understand human processing of multi-sensory stimuli in relation to attention and working memory. This paper explores whether and when supporting visual information with rhythmic auditory stimuli can optimize multi-sensory information processing. In turn, this can make the interaction between humans or between machines and humans more engaging, rewarding and activating. For this purpose a novel working memory paradigm was developed where participants are presented with a series of five target digits randomly interchanged with five distractor digits. Their goal is to remember the target digits and recall them orally. Depending on the condition support is provided by audio and/or rhythm. It is expected that the sound will lead to a better performance. It is also expected that this effect of sound is different in case of rhythmic and non-rhythmic sound. Last but not least, some variability is expected across participants. To make correct conclusions, the data of the experiment was statistically analyzed in a classic way, but also predictive models were developed in order to predict outcomes based on a range of input variables related to the experiment and the participant. The effect of auditory support could be confirmed, but no difference was observed between rhythmic and non-rhythmic sounds. Overall performance was indeed affected by individual differences, such as visual dominance or perceived task difficulty. Surprisingly a music education did not significantly affect the performance and even tended toward a negative effect. To better understand the underlying processes of attention, also brain activation data, e.g., by means of electroencephalography (EEG), should be recorded. This approach can be subject to a future work.

Life and social sciences often focus on the social nature of music (and language alike). In biology, for example, the three main evolutionary hypotheses about music (i.e., sexual selection, parent-infant bond, and group cohesion) stress its intrinsically social character (Honing et al., 2015). Neurobiology thereby has investigated the neuronal and hormonal underpinnings of musicality for more than two decades (Chanda and Levitin, 2013; Salimpoor et al., 2015; Mehr et al., 2019). In line with these approaches, the present paper aims to suggest that the proper way to capture the social interactive nature of music (and, before it, musicality), is to conceive of it as an embodied language, rooted in culturally adapted brain structures (Clarke et al., 2015; D’Ausilio et al., 2015). This proposal heeds Ian Cross’ call for an investigation of music as an “interactive communicative process” rather than “a manifestation of patterns in sound” (Cross, 2014), with an emphasis on its embodied and predictive (coding) aspects (Clark, 2016; Leman, 2016; Koelsch et al., 2019). In the present paper our goal is: (i) to propose a framework of music as embodied language based on a review of the major concepts that define joint musical action, with a particular emphasis on embodied music cognition and predictive processing, along with some relevant neural underpinnings; (ii) to summarize three experiments conducted in our laboratories (and recently published), which provide evidence for, and can be interpreted according to, the new conceptual framework. In doing so, we draw on both cognitive musicology and neuroscience to outline a comprehensive framework of musical interaction, exploring several aspects of making music in dyads, from a very basic proto-musical action, like tapping, to more sophisticated contexts, like playing a jazz standard and singing a hocket melody. Our framework combines embodied and predictive features, revolving around the concept of joint agency (Pacherie, 2012; Keller et al., 2016; Bolt and Loehr, 2017). If social interaction is the “default mode” by which human brains communicate with their environment (Hari et al., 2015), music and musicality conceived of as an embodied language may arguably provide a route toward its navigation.

Understanding rhythmic behavior in the context of coupled auditory and motor systems has been of interest to neurological rehabilitation, in particular, to facilitate walking. Recent work based on behavioral measures revealed an entrainment effect of auditory rhythms on motor rhythms. In this study, we propose a method to compute the neural component of such a process from an electroencephalographic (EEG) signal. A simple auditory-motor synchronization paradigm was used, where 28 healthy participants were instructed to synchronize their finger-tapping with a metronome. The computation of the neural outcome measure was carried out in two blocks. In the first block, we used Generalized Eigendecomposition (GED) to reduce the data dimensionality to the component which maximally entrained to the metronome frequency. The scalp topography pointed at brain activity over contralateral sensorimotor regions. In the second block, we computed instantaneous frequency from the analytic signal of the extracted component. This returned a time-varying measure of frequency fluctuations, whose standard deviation provided our “stability index” as a neural outcome measure of auditory-motor coupling. Finally, the proposed neural measure was validated by conducting a correlation analysis with a set of behavioral outcomes from the synchronization task: resultant vector length, relative phase angle, mean asynchrony, and tempo matching. Significant moderate negative correlations were found with the first three measures, suggesting that the stability index provided a quantifiable neural outcome measure of entrainment, with selectivity towards phase-correction mechanisms. We address further adoption of the proposed approach, especially with populations where sensorimotor abilities are compromised by an underlying pathological condition. The impact of using stability index can potentially be used as an outcome measure to assess rehabilitation protocols, and possibly provide further insight into neuropathological models of auditory-motor coupling.

Musical life became disrupted in 2020 due to the COVID-19 pandemic. Many musicians and venues turned to online alternatives, such as livestreaming. In this study, three livestreamed concerts were organized to examine separate, yet interconnected concepts—agency, presence, and social context—to ascertain which components of livestreamed concerts facilitate social connectedness. Hierarchical Bayesian modeling was conducted on 83 complete responses to examine the effects of the manipulations on feelings of social connectedness with the artist and the audience. Results showed that in concert 1, where half of the participants were allowed to vote for the final song to be played, this option did not result in the experience of more agency. Instead, if their preferred song was played (regardless of voting ability) participants experienced greater connectedness to the artist. In concert 2, participants who attended the concert with virtual reality headsets experienced greater feelings of physical presence, as well as greater feelings of connectedness with the artist, than those that viewed a normal YouTube livestream. In concert 3, attendance through Zoom led to greater experience of social presence, but predicted less connectedness with the artist, compared to a normal YouTube livestream. Crucially, a greater negative impact of COVID-19 (e.g., loneliness) predicted feelings of connectedness with the artist, possibly because participants fulfilled their social needs with this parasocial interaction. Examining data from all concerts suggested that physical presence was a predictor of connectedness with both the artist and the audience, while social presence only predicted connectedness with the audience. Correlational analyses revealed that reductions in loneliness and isolation were associated with feelings of shared agency, physical and social presence, and connectedness to the audience. Overall, the findings suggest that in order to reduce feelings of loneliness and increase connectedness, concert organizers and musicians could tune elements of their livestreams to facilitate feelings of physical and social presence.

Taking regular walks when living with Parkinson’s disease (PD) has beneficial effects on movement and quality of life. Yet, patients usually show reduced physical activity compared to healthy older adults. Using auditory stimulation such as music can facilitate walking but patients vary significantly in their response. An individualized approach adapting musical tempo to patients’ gait cadence, and capitalizing on these individual differences, is likely to provide a rewarding experience, increasing motivation for walk-in PD. We aim to evaluate the observance, safety, tolerance, usability, and enjoyment of a new smartphone application. It was coupled with wearable sensors (BeatWalk) and delivered individualized musical stimulation for gait auto-rehabilitation at home. Forty-five patients with PD underwent a 1-month, outdoor, uncontrolled gait rehabilitation program, using the BeatWalk application (30 min/day, 5 days/week). The music tempo was being aligned in real-time to patients’ gait cadence in a way that could foster an increase up to +10% of their spontaneous cadence. Open-label evaluation was based on BeatWalk use measures, questionnaires, and a six-minute walk test. Patients used the application 78.8% (±28.2) of the prescribed duration and enjoyed it throughout the program. The application was considered “easy to use” by 75% of the patients. Pain, fatigue, and falls did not increase. Fear of falling decreased and quality of life improved. After the program, patients improved their gait parameters in the six-minute walk test without musical stimulation. BeatWalk is an easy to use, safe, and enjoyable musical application for individualized gait rehabilitation in PD. It increases “walk for exercise” duration thanks to high observance.Clinical Trial Registration: ClinicalTrials.gov Identifier: NCT02647242.

Music is believed to work as a bio-social tool enabling groups of people to establish joint action and group bonding experiences. However, little is known about the quality of the group members’ interaction needed to bring about these effects. To investigate the role of interaction quality, and its effect on joint action and bonding experience, we asked dyads (two singers) to perform music in medieval “hocket” style, in order to engage their co-regulatory activity. The music contained three relative inter-onset-interval (IOI) classes: quarter note, dotted quarter note and eight note, marking time intervals between successive onsets (generated by both singers). We hypothesized that singers co-regulated their activity by minimizing prediction errors in view of stable IOI-classes. Prediction errors were measured using a dynamic Bayesian inference approach that allows us to identify three different types of error called fluctuation (micro-timing errors measured in milliseconds), narration (omission errors or misattribution of an IOI to a wrong IOI class), and collapse errors (macro-timing errors that cause the breakdown of a performance). These three types of errors were correlated with the singers’ estimated quality of the performance and the experienced sense of joint agency. We let the singers perform either while moving or standing still, under the hypothesis that the moving condition would have reduced timing errors and increased We-agency as opposed to Shared-agency (the former portraying a condition in which the performers blend into one another, the latter portraying a joint, but distinct, control of the performance). The results show that estimated quality correlates with fluctuation and narration errors, while agency correlates (to a lesser degree) with narration errors. Somewhat unexpectedly, there was a minor effect of movement, and it was beneficial only for good performers. Joint agency resulted in a “shared,” rather than a “we,” sense of joint agency. The methodology and findings open up promising avenues for future research on social embodied music interaction.

Claims made in many Music Information Retrieval (MIR) publications are hard to verify due to the fact that (i) often only a textual description is made available and code remains unpublished – leaving many implementation issues uncovered; (ii) copyrights on music limit the sharing of datasets; and (iii) incentives to put effort into reproducible research – publishing and documenting code and specifics on data – is lacking. In this article the problems around reproducibility are illustrated by replicating an MIR work. The system and evaluation described in ‘A Highly Robust Audio Fingerprinting System’ is replicated as closely as possible. The replication is done with several goals in mind: to describe difficulties in replicating the work and subsequently reflect on guidelines around reproducible research. Added contributions are the verification of the reported work, a publicly available implementation and an evaluation method that is reproducible.

Beating-time gestures are movement patterns of the hand swaying along with music, thereby indicating accented musical pulses. The spatiotemporal configuration of these patterns makes it diÿcult to analyse and model them. In this paper we present an innovative modelling approach that is based upon imitation learning or Programming by Demonstration (PbD). Our approach - based on Dirichlet Process Mixture Models, Hidden Markov Models, Dynamic Time Warping, and non-uniform cubic spline regression - is particularly innovative as it handles spatial and temporal variability by the generation of a generalised trajectory from a set of periodically repeated movements. Although not within the scope of our study, our procedures may be implemented for the sake of controlling movement behaviour of robots and avatar animations in response to music.

Scores of complex, 20th century, solo piano pieces can be difficult to perform and may even include elements that are physically impossible to play. This article investigates the role of music notation in the Opus Clavicembalisticum of Sorabji, which is a rather extreme case in terms of virtuosity and length. To analyze the effect of score notation on learning and performing, 9 pianists were asked to practice music fragments in 3 different score editions, namely the original Urtext edition (a 4-staff score), performance edition (same notes but organized according to an "embodied" performance viewpoint), and study edition (further simplified and with added analytical reading aids). The hypothesis was that the "embodied notation" would have an effect on study time (shorter study time) and errors (fewer errors). Objective features of the study process and performance, such as study time, error ratio and markings on the score (fingerings, hand distribution, synchronization) were compared. Subjective remarks the performers made about the scores were also analyzed. Findings indicate a significant positive influence of the score type on the study time. These results suggest that players draw on ideomotor principles, which include processes based on learned and "embodied" associations between perceived images of the scores and the motor activity that is directly associated with it.

L’approche classique – désincarnée – de la cognition musicale conçoit l’action et la perception comme des processus séparés, périphériques. En revanche, une vision incarnée de la cognition musicale donne une place centrale au couplage étroit entre action et perception. Il est un fait communément établi que la perception oriente l’action. Nous présentons un cadre de travail théorique qui aborde la manière dont le système moteur humain et ses actions peuvent influencer la perception de la musique et réciproquement. La pierre angulaire de ce cadre est la théorie du codage commun, postulant que les représentations de la planification, de l’exécution et de la perception du mouvement s’imbriquent dans le cerveau. L’intégration de l’action et de la perception dans les modèles dits internes est expliquée comme le résultat de processus d’apprentissage associatif. Les modèles internes ont pour caractéristiques de permettre aux états sensoriels attendus ou perçus d’être transférés dans les aires du contrôle moteur correspondantes (modélisation inverse), et vice versa, de prédire les réponses sensorielles d’actions planifiées (modélisation prédictive). La vision incarnée se réfère habituellement à la modélisation inverse afin d’expliquer les effets de l’action sur la perception de la musique (Leman, 2007). Nous prolongeons ce point de vue en désignant la modélisation prédictive comme un mécanisme alternatif par lequel l’action peut moduler la perception. Nous proposons un aperçu détaillé des résultats de travaux empiriques récents qui soutiennent cette idée. De plus, nous démontrons que des dysfonctionnements moteurs peuvent causer des déficiences perceptives, soutenant l’idée principale de l’article selon laquelle le système moteur humain joue un rôle fonctionnel dans la perception auditive. Le constat selon lequel la perception de la musique est forgée par le système moteur humain et ses actions suggère que la cognition musicale est fortement incarnée. Toutefois, nous défendons une approche plus radicale de la cognition musicale incarnée dans le sens où celle-ci a besoin d’être considérée comme un processus dynamique, dans lequel les aspects de l’action, de la perception, de l’introspection et de l’interaction sociale sont d’une importance cruciale.

In this study, we assumed that correct functional movements for weightlifting can be learned with the help of a music-based biofeedback system. We compared musical feedback with verbal feedback from experienced trainers using two independent groups. The focus was on one specific movement called deadlift. Physical parameters under considerations were the spine (i.e. loss of midline stability resulting in flexion) and the forward displacement of the barbell during the repetitions relative to the mid-foot. We recruited 31 recreational weight lifters (21-42 years of age). Results revealed that both feedback types are effective in improving the movements for deadlift. No significant differences were found across the two feedback types, neither in terms of movement, nor in terms of clarity and motivation. The results suggest that the proposed feedback system is a valid tool for technology-aided training and self-training practices.

The design of urban public spaces is typically performed by architects and urban planners, which often only focus on the visual aesthetics of the urban space. Yet, a visually pleasing public open space designed for relaxing will be underused if it sounds unpleasant. Ideally, sonic design should be integrated with visual design, a need the soundscape approach answers. The current trend of co-creating the urban space together with all stakeholders, including local residents, opens up new opportunities to account for all senses in the urban design process. Unfortunately, architects and urban planners struggle to incorporate the soundscape approach in the urban design process and to use it in the context of co-creation. In this work, a hackathon is proposed to generate creative concepts, methods and tools to co-create the urban public space. A soundscape hackathon was organized in the spring of 2019. Participants were challenged to apply their own immersive approaches or virtual and/or augmented reality solutions on selected urban soundscapes. They presented their results to colleagues in the field and to a professional jury. This paper describes the process and results of the event and shows that a hackathon is a viable approach to accelerate the co-creation of the urban public space.

A method is presented for detecting changes in the axial peak tibial acceleration while adapting to self-discovered lower-impact running. Ten runners with high peak tibial acceleration were equipped with a wearable auditory biofeedback system. They ran on an athletic track without and with real-time auditory biofeedback at the instructed speed of 3.2 m·s−1. Because inter-subject variation may underline the importance of individualized retraining, a change-point analysis was used for each subject. The tuned change-point application detected major and subtle changes in the time series. No changes were found in the no-biofeedback condition. In the biofeedback condition, a first change in the axial peak tibial acceleration occurred on average after 309 running gait cycles (3′40″). The major change was a mean reduction of 2.45 g which occurred after 699 running gait cycles (8′04″) in this group. The time needed to achieve the major reduction varied considerably between subjects. Because of the individualized approach to gait retraining and its relatively quick response due to a strong sensorimotor coupling, we want to highlight the potential of a stand-alone biofeedback system that provides real-time, continuous, and auditory feedback in response to the axial peak tibial acceleration for lower-impact running.

Background: Interdisciplinary work is needed for scientific progress, and with this review, our interest is in the scientific progress toward understanding the underlying mechanisms of auditory-motor coupling, and how this can be applied to gait rehabilitation. Specifically we look into the process of entrainment and synchronization; where entrainment is the process that governs the dynamic alignments of the auditory and motor domains based on error-prediction correction, whereas synchronization is the stable maintenance of timing during auditory-motor alignment.Methodology: A systematic literature search in databases PubMed and Web of Science were searched up to 9th of August 2017. The selection criteria for the included studies were adult populations, with a minimum of five participants, investigating walking to an auditory stimulus, with an outcome measure of entrainment, and synchronization. The review was registered in PROSPERO as CRD42017080325.Objectives: The objective of the review is to systematically describe the metrics which measure entrainment and synchronization to auditory stimuli during walking in healthy and neurological populations.Results: Sixteen articles were included. Fifty percent of the included articles had healthy controls as participants (N = 167), 19% had neurological diseases such as Huntington's and Stroke (N = 76), and 31% included both healthy and neurological [Parkinson's disease (PD) and Stroke] participants (N = 101). In the included studies, six parameters were found to capture the interaction between the human movement and the auditory stimuli, these were: cadence, relative phase angle, resultant vector length, interval between the beat and the foot contact, period matching performance, and detrended fluctuation analysis.Conclusion: In this systematic review, several metrics have been identified, which measure the timing aspect of auditory-motor coupling and synchronization of auditory stimuli in healthy and neurological populations during walking. The application of these metrics may enhance the current state of the art and practice across the neurological gait rehabilitation. These metrics also have current shortcomings. Of particular pertinence is our recommendation to consider variability in data from a time-series rather than time-windowed viewpoint. We need it in view of the promising practical applications from which the studied populations may highly benefit in view of personalized medical care.

Since its breakthrough at the beginning of the twenty-first century, the embodied music cognition theory has inspired empirical research on the role of actionperception couplings in musical activities. The integration of novel technologies and analysis methods inspired empirical research advancing knowledge regarding the role of embodiment in music perception and musical signification processes. In this paper, we present recent and on-going research in the field of embodied music cognition, with a focus on studies conducted at IPEM, the research laboratory in systematic musicology at Ghent University, Belgium. Attention is devoted to encoding/decoding principles underlying musical expressiveness, synchronization and entrainment, and action-based effects on music perception. The discussed empirical findings demonstrate that embodiment is only one component in an interconnected network of sensory, motor, affective, and cognitive systems involved in music perception. Currently, these findings drive the embodiment theory towards a more dynamical approach in which the interaction between various internal processes and the external environment are of central importance. Additionally, this approach envisions practical outcomes in the field of music affect research, wellbeing, healing, sports, music engineering, and brain studies.

The use of music and specifically tempo-matched music has been shown to affect running performance. But can we maximize the synchronization of movements to music and does maximum synchronization influence kinematics and motivation? In this study, we explore the effect of different types of music-to-movement alignment strategies on phase coherence, cadence and motivation. These strategies were compared to a control condition where the music tempo was deliberately not aligned to the running cadence. Results show that without relative phase alignment, a negative mean asynchrony (NMA) of footfall timings with respect to the beats is obtained. This means that footfalls occurred slightly before the beat and that beats were anticipated. Convergence towards this NMA or preferred relative phase angle was facilitated when the first music beat of a new song started close to the step, which means that entrainment occurred. The results also show that using tempo and phase alignment, the relative phase can be manipulated or forced in a certain angle with a high degree of accuracy. Ensuring negative angles larger than NMA (step before beat) results in increased motivation and decreasing cadence. Running in NMA or preferred relative phase angles results in a null effect on cadence. Ensuring a positive phase angle with respect to NMA results in higher motivation and higher cadence. None of the manipulations resulted in change in perceived exhaustion or a change in velocity. Results also indicate that gender plays an important role when using forced phase algorithms: effects were more pronounced for the female population than for the male population. The implementation of the proposed alignment strategies and control of beat timing while running opens possibilities optimizing the individual running cadence and motivation.

The paper by Lara Pearson shows that a case study based on qualitative description may reveal interesting aspects about the co-occurrence of hand gestures and singing in a particular music culture. However, above all the paper lets us dream about what could be possible if forces from cultural studies and music cognition research were to be combined. A cross-cultural empirical musicology holds the promise of scientific work that goes far beyond qualitative descriptions.

In the domain of sports and motor rehabilitation, it is of major importance to regulate and control physiological processes and physical motion in most optimal ways. For that purpose, real-time auditory feedback of physiological and physical information based on sound signals, often termed `sonification’, has been proven particularly useful. However, the use of music in biofeedback systems has been much less explored. In the current article, we assert that the use of music, and musical principles, can have a major added value, with respect to mere sound signals, to the benefit of psychological and physical optimization of sports and motor rehabilitation tasks. In this article, we present the 3Mo model to describe three main functions of music that contribute to these benefits. These functions relate the power of music to Motivate, and to Monitor and Modify physiological and physical processes. The model brings together concepts and theories related to human sensorimotor interaction with music, and specifies the underlying psychological and physiological principles. This 3Mo model is intended to provide a conceptual framework that guides future research on musical biofeedback systems in the domain of sports and motor rehabilitation.

In his paper “What is entrainment? Definition and applications in musical research” (this issue), M. Clayton offers a definition of entrainment that is based on a timing dimension (relative phase relationships). However, this definition may be too limited when applied to musical entrainment. Based on the idea that human engagement with music is embodied and that gestures may condition entrainment, I suggest that the definition of entrainment be broadened so as to include a spatiotemporal dimension.

The expressive features of music can influence the velocity of walking. So far, studies used instructed (and intended) synchronization. But is this velocity effect still present with non-instructed (spontaneous) synchronization? To figure that out, participants were instructed to walk in their own comfort tempo on an indoor track, first in silence and then with tempo-matched music. We compared velocities of silence and music conditions. The results show that some music has an activating influence, increasing velocity and motivation, while other music has a relaxing influence, decreasing velocity and motivation. The influence of musical expression on the velocity of self-paced walking can be predicted with a regression model using only three sonic features explaining 56% of the variance. Phase-coherence between footfall and beat did not contribute to the velocity effect, due to its implied fixed pacing. The findings suggest that the velocity effect depends on vigor entrainment that influences both stride length and pacing. Our findings are relevant for preventing injuries, for gait improvement in walking rehabilitation, and for improving performance in sports activities.

Recent research revealed that emotional content can be successfully decoded from human dance movement. Most previous studies made use of videos of actors or dancers portraying emotions through choreography. The current study applies emotion induction techniques and free movement in order to examine the recognition of emotional content from dance. Observers (N = 30) watched a set of silent videos showing depersonalized avatars of dancers moving to an emotionally neutral musical stimulus after emotions of either sadness or happiness had been induced. Each of the video clips consisted of two dance performances which were presented side-by-side and were played simultaneously; one of a dancer in the happy condition and one of the same individual in the sad condition. After every film clip, the observers were asked to make forced-choices concerning the emotional state of the dancer. Results revealed that observers were able to identify the emotional state of the dancers with a high degree of accuracy. Moreover, emotions were more often recognized for female dancers than for their male counterparts. In addition, the results of eye tracking measurements unveiled that observers primarily focus on movements of the chest when decoding emotional information from dance movement. The findings of our study show that not merely portrayed emotions, but also induced emotions can be successfully recognized from free dance movement.

In this study we explore how music can entrain human walkers to synchronise to the musical beat without being instructed to do so. For this, we use an interactive music player, called D-Jogger, that senses the user's walking tempo and phase. D-Jogger aligns the music by manipulating the timing difference between beats and footfalls. Experiments are reported that led to the development and optimisation of four alignment strategies. The first strategy matched the music's tempo continuously to the runner's pace. The second strategy matched the music's tempo at the beginning of a song to the runner's pace, keeping the tempo constant for the remainder of the song. The third alignment starts a song in perfect phase synchrony and continues to adjust the tempo to match the runner's pace. The fourth and last strategy additionally adjusts the phase of the music so each beat matches a footfall. The first two strategies resulted in a minor increase of steps in phase synchrony with the main beat when compared to a random playlist, the last two strategies resulted in a strong increase in synchronised steps. These results may be explained in terms of phase-error correction mechanisms and motor prediction schemes. Finding the phase-lock is difficult due to fluctuations in the interaction, whereas strategies that automatically align the phase between movement and music solve the problem of finding the phase-locking. Moreover, the data show that once the phase-lock is found, alignment can be easily maintained, suggesting that less entrainment effort is needed to keep the phase-lock, than to find the phase-lock. The different alignment strategies of D-Jogger can be applied in different domains such as sports, physical rehabilitation and assistive technologies for movement performance.

Inspired by a theory of embodied music cognition, we investigate whether music can entrain the speed of beat synchronized walking. If human walking is in synchrony with the beat and all musical stimuli have the same duration and the same tempo, then differences in walking speed can only be the result of music-induced differences in stride length, thus reflecting the vigor or physical strength of the movement. Participants walked in an open field in synchrony with the beat of 52 different musical stimuli all having a tempo of 130 beats per minute and a meter of 4 beats. The walking speed was measured as the walked distance during a time interval of 30 seconds. The results reveal that some music is 'activating' in the sense that it increases the speed, and some music is 'relaxing' in the sense that it decreases the speed, compared to the spontaneous walked speed in response to metronome stimuli. Participants are consistent in their observation of qualitative differences between the relaxing and activating musical stimuli. Using regression analysis, it was possible to set up a predictive model using only four sonic features that explain 60% of the variance. The sonic features capture variation in loudness and pitch patterns at periods of three, four and six beats, suggesting that expressive patterns in music are responsible for the effect. The mechanism may be attributed to an attentional shift, a subliminal audio-motor entrainment mechanism, or an arousal effect, but further study is needed to figure this out. Overall, the study supports the hypothesis that recurrent patterns of fluctuation affecting the binary meter strength of the music may entrain the vigor of the movement. The study opens up new perspectives for understanding the relationship between entrainment and expressiveness, with the possibility to develop applications that can be used in domains such as sports and physical rehabilitation.

The theory of embodied music cognition states that the perception and cognition of music is firmly, although not exclusively, linked to action patterns associated with that music. In this regard, the focus lies mostly on how music promotes certain action tendencies (i.e., dance, entrainment, etc.). Only recently, studies have started to devote attention to the reciprocal effects that people's body movements may exert on how people perceive certain aspects of music and sound (e.g., pitch, meter, musical preference, etc.). The present study positions itself in this line of research. The central research question is whether expressive body movements, which are systematically paired with music, can modulate children's perception of musical expressiveness. We present a behavioral experiment in which different groups of children (7-8 years, N = 46) either repetitively performed a happy or a sad choreography in response to expressively ambiguous music or merely listened to that music. The results of our study show indeed that children's perception of musical expressiveness is modulated in accordance with the expressive character of the dance choreography performed to the music. This finding supports theories that claim a strong connection between action and perception, although further research is needed to uncover the details of this connection.