Search

Sound Actions: Conceptualizing Musical Instruments. By Alexander Refsum Jensenius. Cambridge, MA: MIT Press, 2022. [xx, 283 p. ISBN 9780262544634 (paperback), $45; ISBN 9780262372213 (ebook), price varies; ISBN 9780262372206 (open access [...]

This paper investigates the potential of the intrinsically motivated reinforcement learning (IMRL) approach for robotic drumming. For this purpose, we implemented an IMRL-based algorithm for a drumming robot called ZRob , an underactuated two-DoF robotic arm with flexible grippers. Two ZRob robots were instructed to play rhythmic patterns derived from MIDI files. The RL algorithm is based on the deep deterministic policy gradient (DDPG) method, but instead of relying solely on extrinsic rewards, the robots are trained using a combination of both extrinsic and intrinsic reward signals. The results of the training experiments show that the utilization of intrinsic reward can lead to meaningful novel rhythmic patterns, while using only extrinsic reward would lead to predictable patterns identical to the MIDI inputs. Additionally, the observed drumming patterns are influenced not only by the learning algorithm but also by the robots' physical dynamics and the drum's constraints. This work suggests new insights into the potential of embodied intelligence for musical performance. [ABSTRACT FROM AUTHOR]

This paper presents an innovative experimental setup that employs the principles of audio technology to subject adherent cells to rhythmic vertical vibrations. We employ a novel approach that combines three-axis acceleration measurements and particle tracking velocimetry to evaluate the setup's performance. This allows us to estimate crucial parameters such as root mean square acceleration, fluid flow patterns, and shear stress generated within the cell culture wells when subjected to various vibration types. The experimental conditions consisted of four vibrational modes: No Vibration, Continuous Vibration, Regular Pulse, and Variable Pulse. To evaluate the effects on cells, we utilized fluorescence microscopy and a customized feature extraction algorithm to analyze the F-actin filament structures. Our findings indicate a consistent trend across all vibrated cell cultures, revealing a reduction in size and altered orientation (2D angle) of the filaments. Furthermore, we observed cell accumulations in the G1 cell cycle phase in cells treated with Continuous Vibration and Regular Pulse. Our results demonstrate a negative correlation between the magnitude of mechanical stimuli and the size of F-actin filaments, as well as a positive correlation with the accumulations of cells in the G1 phase of the cell cycle. By unraveling these analyses, this study paves the way for future investigations and provides a compelling framework for comprehending the intricate cellular responses to rhythmic mechanical stimulation. [ABSTRACT FROM AUTHOR]

In this article, we introduce the coadaptive audiovisual instrument, CAVI. This instrument uses deep learning to generate control signals based on muscle and motion data of a performer's actions. The generated signals control time-based live sound-processing modules. How does a performer perceive such an instrument? Does it feel like a machine learning–based musical tool? Or is it an actor with the potential to become a musical partner? We report on an evaluation of CAVI after it had been used in two public performances. The evaluation is based on interviews with the performers, audience questionnaires, and the creator's self-analysis. Our findings suggest that the perception of CAVI as a tool or actor correlates with the performer's sense of agency. The perceived agency changes throughout a performance based on several factors, including perceived musical coordination, the balance between surprise and familiarity, a "common sense," and the physical characteristics of the performance setting. [ABSTRACT FROM AUTHOR]

As formal theoretical linguistic methodology has matured, recent years have seen the advent of applying it to objects of study that transcend language, e.g. to the syntax and semantics of music (Lerdahl & Jackendoff 1983 , Schlenker 2017a ; see also Rebuschat et al. 2011). One of the aims of such extensions is to shed new light on how meaning is construed in a range of communicative systems. In this paper, we approach this goal by looking at narrative dance in the form of Bharatanatyam. We argue that a semantic approach to dance can be modeled closely after the formal semantics of visual narrative proposed by Abusch (2013 , 2014 , 2021). A central conclusion is that dance not only shares properties of other fundamentally human means of expression, such as visual narrative and music, but that it also exhibits similarities to sign languages and the gestures of non-signers (see, e.g. Schlenker 2020) in that it uses space to track individuals in a narrative and performatively portray the actions of those individuals. From the perspective of general human cognition, these conclusions corroborate the idea that linguistic investigations beyond language (see Patel-Grosz et al. forthcoming) can yield insights into the very nature of the human mind and of the communicative devices that it avails. [ABSTRACT FROM AUTHOR]

The primary aim of this article is to provide a biological rhythm model based on previous theoretical and experimental findings to promote more comprehensive studies of rhythmic mechanical stimulation of cell cultures, which relates to tissue engineering and regenerative medicine fields. Through an interdisciplinary approach where different standpoints from biology and musicology are combined, we explore some of the core rhythmic features of biological and cellular rhythmic processes and present them as a trio model that aims to afford a basic but fundamental understanding of the connections between various biological rhythms. It is vital to highlight such links since rhythmic mechanical stimulation and its effect on cell cultures are vastly underexplored even though the cellular response to mechanical stimuli (mechanotransduction) has been studied widely and relevant experimental evidence suggests mechanotransduction processes are rhythmic. [ABSTRACT FROM AUTHOR]

Ensemble musicians engage with each other visually through glances and body motion. We conducted a case study to test how string quartet musicians would respond to playing conditions that were meant to discourage or promote visually communicative behaviour. A quartet performed in different seating configurations under rehearsal and concert conditions. Quantity of head motion was reduced when musicians' gaze was constrained. Differences in gaze and body motion between musicians reflected their musical roles in the ensemble. Overall, our findings suggest that gaze and motion dynamics vary within and between performances in response to changing musical, situational and social factors. [ABSTRACT FROM AUTHOR]

Music performance can be cognitively and physically demanding. These demands vary across the course of a performance as the content of the music changes. More demanding passages require performers to focus their attention more intensity, or expend greater "mental effort." To date, it remains unclear what effect different cognitive-motor demands have on performers' mental effort. It is likewise unclear how fluctuations in mental effort compare between performers and perceivers of the same music. We used pupillometry to examine the effects of different cognitive-motor demands on the mental effort used by performers and perceivers of classical string quartet music. We collected pupillometry, motion capture, and audio-video recordings of a string quartet as they performed a rehearsal and concert (for live audience) in our lab. We then collected pupillometry data from a remote sample of musically-trained listeners, who heard the audio recordings (without video) that we captured during the concert. We used a modelling approach to assess the effects of performers' bodily effort (head and arm motion; sound level; performers' ratings of technical difficulty), musical complexity (performers' ratings of harmonic complexity; a score-based measure of harmonic tension), and expressive difficulty (performers' ratings of expressive difficulty) on performers' and listeners' pupil diameters. Our results show stimulating effects of bodily effort and expressive difficulty on performers' pupil diameters, and stimulating effects of expressive difficulty on listeners' pupil diameters. We also observed negative effects of musical complexity on both performers and listeners, and negative effects of performers' bodily effort on listeners, which we suggest may reflect the complex relationships that these features share with other aspects of musical structure. Looking across the concert, we found that both of the quartet violinists (who exchanged places halfway through the concert) showed more dilated pupils during their turns as 1st violinist than when playing as 2nd violinist, suggesting that they experienced greater arousal when "leading" the quartet in the 1st violin role. This study shows how eye tracking and motion capture technologies can be used in combination in an ecological setting to investigate cognitive processing in music performance. [ABSTRACT FROM AUTHOR]

Music researchers work with increasingly large and complex data sets. There are few established data handling practices in the field and several conceptual, technological, and practical challenges. Furthermore, many music researchers are not equipped for (or interested in) the craft of data storage, curation, and archiving. This paper discusses some of the particular challenges that empirical music researchers face when working towards Open Research practices: handling (1) (multi)media files, (2) privacy, and (3) copyright issues. These are exemplified through MusicLab, an event series focused on fostering openness in music research. It is argued that the "best practice" suggested by the FAIR principles is too demanding in many cases, but "good enough practice" may be within reach for many. A four-layer data handling "recipe" is suggested as concrete advice for achieving "good enough practice" in empirical music research. [ABSTRACT FROM AUTHOR]

We investigated how the action-sound relationships found in electric guitar performance can be used in the design of new instruments. Thirty-one trained guitarists performed a set of basic sound-producing actions (impulsive, sustained, and iterative) and free improvisations on an electric guitar. We performed a statistical analysis of the muscle activation data (EMG) and audio recordings from the experiment. Then we trained a long short-term memory network with nine different configurations to map EMG signal to sound. We found that the preliminary models were able to predict audio energy features of free improvisations on the guitar, based on the dataset of raw EMG from the basic soundproducing actions. The results provide evidence of similarities between body motion and sound in music performance, compatible with embodied music cognition theories. They also show the potential of using machine learning on recorded performance data in the design of new musical instruments. [ABSTRACT FROM AUTHOR]

The links between music and human movement have been shown to provide insight into crucial aspects of human's perception, cognition, and sensorimotor systems. In this study, we examined the influence of music on movement during standstill, aiming at further characterizing the correspondences between movement, music, and perception, by analyzing head sway fractality. Eighty seven participants were asked to stand as still as possible for 500 seconds while being presented with alternating silence and audio stimuli. The audio stimuli were all rhythmic in nature, ranging from a metronome track to complex electronic dance music. The head position of each participant was captured with an optical motion capture system. Long-range correlations of head movement were estimated by detrended fluctuation analysis (DFA). Results agree with previous work on the movement-inducing effect of music, showing significantly greater head sway and lower head sway fractality during the music stimuli. In addition, patterns across stimuli suggest a two-way adaptation process to the effects of music, with musical stimuli influencing head sway while at the same time fractality modulated movement responses. Results indicate that fluctuations in head movement in both conditions exhibit long-range correlations, suggesting that the effects of music on head movement depended not only on the value of the most recent measured intervals, but also on the values of those intervals at distant times. [ABSTRACT FROM AUTHOR]

The present study investigates how people move and relate to each other – and to the dance music – in a club-like setting created within a motion capture laboratory. Three groups of participants (29 in total) each danced to a 10-minute-long DJ mix consisting of four tracks of electronic dance music (EDM). Two of the EDM tracks had little structural development, while the two others included a typical "break routine" in the middle of the track, consisting of three distinct passages: (a) "breakdown", (b) "build-up" and (c) "drop". The motion capture data show similar bodily responses for all three groups in the break routines: a sudden decrease and increase in the general quantity of motion. More specifically, the participants demonstrated an improved level of interpersonal synchronization after the drop, particularly in their vertical movements. Furthermore, the participants' activity increased and became more pronounced after the drop. This may suggest that the temporal removal and reintroduction of a clear rhythmic framework, as well as the use of intensifying sound features, have a profound effect on a group's beat synchronization. Our results further suggest that the musical passages of EDM efficiently lead to the entrainment of a whole group, and that a break routine effectively "re-energizes" the dancing. [ABSTRACT FROM AUTHOR]

The relationships between human body motion and music have been the focus of several studies characterizing the correspondence between voluntary motion and various sound features. The study of involuntary movement to music, however, is still scarce. Insight into crucial aspects of music cognition, as well as characterization of the vestibular and sensorimotor systems could be largely improved through a description of the underlying links between music and involuntary movement. This study presents an analysis aimed at quantifying involuntary body motion of a small magnitude (micromotion) during standstill, as well as assessing the correspondences between such micromotion and different sound features of the musical stimuli: pulse clarity, amplitude, and spectral centroid. A total of 71 participants were asked to stand as still as possible for 6 min while being presented with alternating silence and music stimuli: Electronic Dance Music (EDM), Classical Indian music, and Norwegian fiddle music (Telespringar). The motion of each participant's head was captured with a marker-based, infrared optical system. Differences in instantaneous position data were computed for each participant and the resulting time series were analyzed through cross-correlation to evaluate the delay between motion and musical features. The mean quantity of motion (QoM) was found to be highest across participants during the EDM condition. This musical genre is based on a clear pulse and rhythmic pattern, and it was also shown that pulse clarity was the metric that had the most significant effect in induced vertical motion across conditions. Correspondences were also found between motion and both brightness and loudness, providing some evidence of anticipation and reaction to the music. Overall, the proposed analysis techniques provide quantitative data and metrics on the correspondences between micromotion and music, with the EDM stimulus producing the clearest music-induced motion patterns. The analysis and results from this study are compatible with embodied music cognition and sensorimotor synchronization theories, and provide further evidence of the movement inducing effects of groove-related music features and human response to sound stimuli. Further work with larger data sets, and a wider range of stimuli, is necessary to produce conclusive findings on the subject. [ABSTRACT FROM AUTHOR]

Background: Previous evidence suggests that the variability of the spatial center of infant movements, calculated by computer-based video analysis software, can identify fidgety general movements (GMs) and predict cerebral palsy.Aim: To evaluate whether computer-based video analysis quantifies specific characteristics of normal fidgety movements as opposed to writhing general movements.Methods: A longitudinal study design was applied. Twenty-seven low-to moderate-risk preterm infants (20 boys, 7 girls; mean gestational age 32 [SD 2.7, range 27–36] weeks, mean birth weight 1790 grams [SD 430g, range 1185–2700g]) were videotaped at the ages of 3–5 weeks (period of writhing GMs) and 10–15 weeks (period of fidgety GMs) post term. GMs were classified according to Prechtl’s general movement assessment method (GMA) and by computer-based video analysis. The variability of the centroid of motion (CSD), derived from differences between subsequent video frames, was calculated by means of computer-based video analysis software; group mean differences between GM periods were reported.Results: The mean variability of the centroid of motion (CSD) determined by computer-based video analysis was 7.5% lower during the period of fidgety GMs than during the period of writhing GMs (p = 0.004).Conclusion: Our findings support that the variability of the centroid of motion reflects small and variable movements evenly distributed across the body, and hence shows that computer-based video analysis qualifies for assessment of direction and amplitude of FMs in young infants. [ABSTRACT FROM PUBLISHER]

In this paper, we report on a free-hand motion capture study in which 32 participants 'traced' 16 melodic vocal phrases with their hands in the air in two experimental conditions. Melodic contours are often thought of as correlated with vertical movement (up and down) in time, and this was also our initial expectation. We did find an arch shape for most of the tracings, although this did not correspond directly to the melodic contours. Furthermore, representation of pitch in the vertical dimension was but one of a diverse range of movement strategies used to trace the melodies. Six different mapping strategies were observed, and these strategies have been quantified and statistically tested. The conclusion is that metaphorical representation is much more common than a 'graph-like' rendering for such a melodic sound-tracing task. Other findings include a clear gender difference for some of the tracing strategies and an unexpected representation of melodies in terms of a small object for some of the Hindustani music examples. The data also show a tendency of participants moving within a shared 'social box'. [ABSTRACT FROM AUTHOR]