Search

This document is a companion to the electronic 'music boxes' sitting beside you. It supports, and comprehensively explains, a portfolio of over an hour of 1-bit compositions, represented by nearly 40 kilobytes of program memory (roughly 400 times smaller than this PDF document). With this project I have set out to compose the tiniest music I can using the most minimal instrumentational environment possible. This project is an exploration of conceptual purism: a personal exploration of the compositional voice in an increasingly restricted environment. The danger with this process however, is aptly communicated by the following : 'I thought using loops was cheating, so I programmed my own samples. I then thought using samples was cheating, so I recorded real drums. I then thought that programming it was cheating, so I learned to play drums for real. I then thought using bought drums was cheating, so I learned to make my own. I then thought using pre-made skins was cheating too, so I grew my own goat from a baby goat. I also think that is cheating, but I'm not sure where to go from here. I haven't made much music lately, what with the goat farming and all'. Even so, the most compelling thing about this process is, in my experience, once you start distilling your composition and stripping away superfluous components to produce musically dense micro-constructions, you realise that you can keep going. The attraction of embedded music, music hard-coded for a particular computational platform, is that the composer can control all elements of the composition, its encapsulation and its environment. How many composers can say that they constructed the platform, the synthesis, the instruments, the score and the composition? The sonic landscape of microcontroller music is dictated entirely by its creator. Using the 8-bit AVR microcontroller range to enforce both absolute memory restriction and hardware simplicity, and through custom synthesis software and algorithms, I have created a series of musical investigations focused on the central research question: • Using 1-bit synthesis, how does constructing music within one kilobyte (or a similarly limited memory environment) inform or change compositional methodology? Furthermore, the following two questions arise in sympathy: • Is it actually possible to create complex, engaging music in less than a kilobyte of program space using 1-bit synthesis? (Including generation and synthesis code of course; no hiding behind decoders and interpreters!) • What is the most economical method of creating 1-bit complex musical compositions in relation to memory footprint? By the end of this commentary, the reader will understand: • How this project fits within a wider context: the history of 1-bit music and the chipmusic scene. • The sonic and musical properties of a 1-bit waveform. • How a 1-bit waveform might be synthesised in software. • Approaches to organise and structure 1-bit voices musically. • Potential compositional solutions to employ in low memory environments.

This thesis examines the use of sonification (the transformation of non-musical data into sound) as a means of creating generative music (algorithmic music which is evolving in real time and is of potentially infinite length). It consists of a portfolio of ten works where the possibilities of sonification as a strategy for creating generative works is examined. As well as exploring the viability of sonification as a compositional strategy toward infinite work, each work in the portfolio aims to explore the notion of how artistic coherency between data and resulting sound is achieved – rejecting the notion that sonification for artistic means leads to the arbitrary linking of data and sound. In the accompanying written commentary the definitions of sonification and generative music are considered, as both are somewhat contested terms requiring operationalisation to correctly contextualise my own work. Having arrived at these definitions each work in the portfolio is documented. For each work, the genesis of the work is considered, the technical composition and operation of the piece (a series of tutorial videos showing each work in operation supplements this section) and finally its position in the portfolio as a whole and relation to the research question is evaluated. The body of work is considered as a whole in relation to the notion of artistic coherency. This is separated into two main themes: the relationship between the underlying nature of the data and the compositional scheme and the coherency between the data and the soundworld generated by each piece.

Music is used to enhance the experience of participants and visitors in a range of settings including theatre, film, video games, installations and theme parks. These experiences may be interactive, contrastingly episodic and with variable duration. Hence, the musical accompaniment needs to be dynamic and to transition between contrasting music passages. In these contexts, computer generation of music may be necessary for practical reasons including distribution and cost. Automated and dynamic composition algorithms exist but are not well-suited to a highly interactive episodic context owing to transition-related problems including discontinuity, abruptness, extended repetitiveness and lack of musical granularity and musical form. Addressing these problems requires algorithms capable of reacting to participant behaviour and episodic change in order to generate formic music that is continuous and coherent during transitions. This thesis presents the Form-Aware Transitioning and Recovering Algorithm (FATRA) for realtime, adaptive, form-aware music generation to provide continuous musical accompaniment in episodic context. FATRA combines stochastic grammar adaptation and grammar merging in real time. The Form-Aware Transition Engine (FATE) implementation of FATRA estimates the time-occurrence of upcoming narrative transitions and generates a harmonic sequence as narrative accompaniment with a focus on coherent, form-aware music transitioning between music passages of contrasting character. Using FATE, FATRA has been evaluated in three perceptual user studies: An audioaugmented real museum experience, a computer-simulated museum experience and a music-focused online study detached from narrative. Music transitions of FATRA were benchmarked against common approaches of the video game industry, i.e. crossfading and direct transitions. The participants were overall content with the music of FATE during their experience. Transitions of FATE were significantly favoured against the crossfading benchmark and competitive against the direct transitions benchmark, without statistical significance for the latter comparison. In addition, technical evaluation demonstrated capabilities of FATRA including form generation, repetitiveness avoidance and style/form recovery in case of falsely predicted narrative transitions. Technical results along with perceptual preference and competitiveness against the benchmark approaches are deemed as positive and the structural advantages of FATRA, including form-aware transitioning, carry considerable potential for future research.

Throughout this thesis we will explore automatic music source separation by utilizing modern (at the time of writing) techniques and tools from machine learning and big data processing. The bulk of this work was carried out between 2016 and 2019. In Chapter 2 we conduct a review of source separation literature. We start by outlining a subset of applications of source separation in some depth. We describe some of the early, pioneering work in automatic source separation: Auditory Scene Analysis, and its digital counterpart, Computational Auditory Scene Analysis. We then introduce matrix decomposition-based methods such as Independent Component Analysis and Non-Negative Matrix factorization, and pitch informed methods where the separation algorithm is guided by pitch information that is known a priori. We brie y discuss user-guided methods, before conducting a thorough review of Deep Learning based source separation, including recurrent, convolutional, deep clustering-based, and Generative Adversarial Networks. We then proceed to describe common evaluation metrics and training datasets. Finally, we list a number of current challenges and drawbacks of current systems. Chapter 3 focuses on datasets for musical source separation. First we show the growth of dataset sizes for both machine learning in general and music information retrieval specifically. We give several examples of the complexities and idiosyncrasies that are intrinsic to music datasets. We then proceed to present a method for extracting ground truth data for source separation from large unstructured musical catalogs. In Chapter 4 we design a novel deep learning-based source separation algorithm. Motivation is provided by means of a musicological study1 that showed the high importance of vocals relative to other musical factors, in the minds of listeners. At the core of the vocal separation algorithm is the U-Net, a deep learning architecture that uses skip connections to preserve fine-grained detail. It was originally developed in the biomedical imaging domain, and later adapted to image-to-image translation. We adapt it to the source separation domain by treating spectrograms as images, and we use the dataset mining methods from Chapter 3 to generate sufficiently large training data. We evaluate our model objectively using standard evaluation metrics, subjectively using \crowdsourced" human subjects. To the best of our knowledge, this is the first use of U-Nets for source separation. In the introduction above we proposed joint learning to optimize source separation and other objectives. In Chapter 5 we investigate one such instance: multi-task learning of vocal removal and vocal pitch tracking. We combine the vocal separation model from Chapter 4 with a state of the art pitch salience estimation model2, exploring several ways of combining the two models. We find that vocal pitch estimation benefits from joint learning when the two tasks are trained in sequence, with the source separation model preceding the pitch estimation model. We also report benefits from fine-tuning by iteratively applying the model. Chapter 6 extends the U-Net model to multiple instruments. In order to minimize the phase artifacts that were a common issue in Chapter 4, we modify the model to operate in the complex domain. We run experiments with several loss functions: Time-domain loss, magnitude-only frequency domain loss, and joint time and frequency-domain loss. Our experiments are evaluated both objectively and subjectively, and we carry out extensive qualitative analysis to investigate the effects of complex masking. Finally, we conclude the thesis in Chapter 7 by summarizing this work and highlighting several future directions of research.

Technologies are fundamental to music and its marketing and dissemination, as is the categorisation of music by genre. In this research we examine the relationship between musical genre and technology by examining genre proliferation, fragmentation and hybridity. We compare the movement of musical artists between genres in various technological eras, and evaluate the connections between the dissemination of music and its categorisation. Cultural hybridity and fragmentation is thought to be the norm in the globalised era by many scholars, and the online music environment appears to be populated by hybrid genres and micro-genres. To examine this we study the representation of musical genre on the Internet. We acquire data from three main sources: The Echo Nest, a music-intelligence system, and two collectively constructed knowledge-bases, Wikidata and MusicBrainz. We discover geographical and commercial biases. We calculate genre inception dates in order to examine category proliferation, and construct networks from these data, using the relationships between artists and genres to establish structure. Using network analyses to quantify genre hybridity we find increasing hybridisation, peaking at various periods in different datasets. Statistical analyses, comparing hybridity within our various data, validates our method and reveals a relationship between the activity of editing music information and the movement of musical artists between musical genres. We also find evidence for the fragmentation of genre and the appearance of micro- genres. We consider artists that are invisible in mainstream systems using data from three alternative platforms, Bandcamp, CD Baby and SoundCloud, and examine rapid genre proliferation in Spotify. We then discuss hybridity and fragmentation in relation to postmodernity, hypermodernity and unimodernity, music and genre within society, and the ways genre intersects with technology.

This thesis was motivated by an experiment carried out in the 1960s that studied the relationship between vocal performance practice and society by means of statistical analysis. Using a comprehensive corpus of audio recordings of singing from around the world collected over several decades, the ethnomusicologist Alan Lomax devised the Cantometrics project, the largest comparative study of music, in which 36 performance practice characteristics were rated for each recording. With particular interest in vocal production, we intended to formalise the knowledge of vocal production to enable statistical and computational approaches in the spirit of Cantometrics. Three models of vocal production were investigated: the perceptual model from Cantometrics, a physical model from voice science and a physiological model from singing education. We built on Johan Sundberg's vocal source parameters and Jo Estill's physiological building blocks as the basis to develop an ontology of vocal production. Two approaches to automated characterisation of the ontological descriptors were considered. For the incremental approach a proof-of-concept experiment on automatic labelling of phonation modes was presented, based on reconstructing the vocal source waveform by means of inverse filtering. We created a dataset of sustained sung vowels with annotations on pitch, vowel and phonation mode on which our model was trained. Steps to generalise this experiment to more complex data were outlined, discussing the challenges of such generalisation. The integrated approach addressed the full variance in the data, turning to the methodology of expert knowledge elicitation in order to annotate the original Cantometrics dataset with our descriptors. We performed an investigative mixed-methods study in which 13 vocal physiology experts from different professional backgrounds were interviewed; they used our ontology to analyse vocal production in the Cantometrics dataset. The goal of the study was to: a) validate the acceptance of our ontological terms, b) verify the consensus between experts on the values of the descriptors, c) collect reliable annotations. While the acceptance of the ontology was good for most terms, quantitative analysis showed good agreement between experts for only two out of 11 descriptors (larynx height, aryepiglottic sphincter). A detailed qualitative analysis of the interview data (over 33 hours) was followed by a meta-analysis extracting common themes and confounding issues which point to probable reasons for the disagreement. For aryepiglottic sphincter and larynx height we collected the average ratings, which constitute the first set of reliable annotations on vocal production. A strong correlation was found between larynx height and the vocal width parameter from Cantometrics; larynx height was therefore a good candidate to replace vocal width as a more objective descriptor. The current work was based on knowledge from a number of research disciplines, and its results are discussed from the viewpoint of several fields - MIR, vocal pedagogy, Cantometrics - for which they present significant implications. Future research is suggested for each of the fields. Based on the meta-analysis, we account for the reasons for disagreement between experts on the subject of vocal production, from music information retrieval (MIR) and singing education perspectives. We further explain the various kinds of bias that affect raters. We conclude that vocal physiology, though offering a more objective language than perceptual descriptors, is not well-suited as an ontological middle layer for statistical approaches to singing given the current state of knowledge. A mixed perceptual-objective path to ontology building is suggested and ways to collect reliable annotations are outlined. In the domain of vocal pedagogy we touch on the issue of communication on vocal physiology between experts, between teacher and student; we consider the future of teaching vocal technique and make suggestions for new experiments in the field. A plan is presented for revising and scaling up Cantometrics as an interdisciplinary collaboration. Possible contributions of MIR, ethnomusicologists and vocal production specialists are specified.

Based on online and offline ethnographic fieldwork among the Spanish community in London, this thesis investigates why people circulate and share music on social media platforms, particularly within a context of migration. Written from the perspective of an insider-outsider, it combines research data from ethnographic participant observation and interviews to understand how music media circulates online, studying the different roles it fulfils within these contexts as different forms of musicking. The thesis responds to these questions by addressing six areas. Chapter 1 explores the uses of music to perform and articulate cultural and gender identity on social media profiles. Chapter 2 investigates how music sharing and compiling can be used to maintain relationships and social capital and to participate in transnational sociality. Chapter 3 addresses how the circulation of music parody as citizen engagement in contexts of political conflict such as the Brexit and Catalonian referendums gives rise to temporary political alliances in the form of citizen assemblages. Chapter 4 considers the circulation of music on social media from the perspective of fandom and how it is influenced by post-object ephemeral and reflexive practices. Chapter 5 analyses how the ubiquitous and imagined character of online music media fosters its circulation as a silent and visual element of online sociality, effectively generating practices of imagined listening. Chapter 6 examines ritualistic practices of music exchange on social media and their relationship with emerging moral economies of music circulation as foundational elements of online sociality and citizenship. I conclude by arguing that, in circulating music online through their social media and streaming profiles, users develop new forms of (im)material culture-making. Online musicking enables new ways of being in the world, turning the intangible, time-bound aural and visual experience of online music into something that has a materialised impact in the social lives of users. Through the circulation of music online users form, dissolve and inhabit temporary music-based alliances and expand their social worlds.

Automatic music transcription (AMT) is commonly defined as the process of converting an acoustic musical signal into some form of musical notation, and can be split into two separate phases: (1) multi-pitch detection, the conversion of an audio signal into a time-frequency representation similar to a MIDI file; and (2) converting from this time-frequency representation into a musical score. A substantial amount of AMT research in recent years has concentrated on multi-pitch detection, and yet, in the case of the transcription of polyphonic music, there has been little progress. There are many potential reasons for this slow progress, but this thesis concentrates on the (lack of) use of music language models during the transcription process. In particular, a music language model would impart to a transcription system the background knowledge of music theory upon which a human transcriber relies. In the related field of automatic speech recognition, it has been shown that the use of a language model drawn from the field of natural language processing (NLP) is an essential component of a system for transcribing spoken word into text, and there is no reason to believe that music should be any different. This thesis will show that a music language model inspired by NLP techniques can be used successfully for transcription. In fact, this thesis will create the blueprint for such a music language model. We begin with a brief overview of existing multi-pitch detection systems, in particular noting four key properties which any music language model should have to be useful for integration into a joint system for AMT: it should (1) be probabilistic, (2) not use any data a priori, (3) be able to run on live performance data, and (4) be incremental. We then investigate voice separation, creating a model which achieves state-of-the-art performance on the task, and show that, used as a simple music language model, it improves multi-pitch detection performance significantly. This is followed by an investigation of metrical detection and alignment, where we introduce a grammar crafted for the task which, combined with a beat-tracking model, achieves state-of-the-art results on metrical alignment. This system's success adds more evidence to the long-existing hypothesis that music and language consist of extremely similar structures. We end by investigating the joint analysis of music, in particular showing that a combination of our two models running jointly outperforms each running independently. We also introduce a new joint, automatic, quantitative metric for the complete transcription of an audio recording into an annotated musical score, something which the field currently lacks.

The comparison of world music cultures has been considered in musicological research since the end of the 19th century. Traditional methods from the field of comparative musicology typically involve the process of manual music annotation. While this provides expert knowledge, the manual input is timeconsuming and limits the potential for large-scale research. This thesis considers computational methods for the analysis and comparison of world music cultures. In particular, Music Information Retrieval (MIR) tools are developed for processing sound recordings, and data mining methods are considered to study similarity relationships in world music corpora. MIR tools have been widely used for the study of (mainly) Western music. The first part of this thesis focuses on assessing the suitability of audio descriptors for the study of similarity in world music corpora. An evaluation strategy is designed to capture challenges in the automatic processing of world music recordings and different state-of-the-art descriptors are assessed. Following this evaluation, three approaches to audio feature extraction are considered, each addressing a different research question. First, a study of singing style similarity is presented. Singing is one of the most common forms of musical expression and it has played an important role in the oral transmission of world music. Hand-designed pitch descriptors are used to model aspects of the singing voice and clustering methods reveal singing style similarities in world music. Second, a study on music dissimilarity is performed. While musical exchange is evident in the history of world music it might be possible that some music cultures have resisted external musical influence. Low-level audio features are combined with machine learning methods to find music examples that stand out in a world music corpus, and geographical patterns are examined. The last study models music similarity using descriptors learned automatically with deep neural networks. It focuses on identifying music examples that appear to be similar in their audio content but share no (obvious) geographical or cultural links in their metadata. Unexpected similarities modelled in this way uncover possible hidden links between world music cultures. This research investigates whether automatic computational analysis can uncover meaningful similarities between recordings of world music. Applications derive musicological insights from one of the largest world music corpora studied so far. Computational analysis as proposed in this thesis advances the state-of-the-art in the study of world music and expands the knowledge and understanding of musical exchange in the world.

Sound art is a difficult to categorise and broad genre description that draws together modes of creative practice which use sound as a medium or a subject. The field is considered to be critically underrepresented and under-theorised despite an increase of attention and popularity since the 1990s (Licht 2007, 2001, Cox 2009). This is partly as a consequence of an analytical and historical emphasis on textual and conceptual approaches which dominated the arts through the 1970s and 1980s (Cox 2011, 2013). In particular, acknowledgement of the influence of object-based and kinetic sculpture within the field of sound art is found to be inadequate (Chau 2014, Keylin 2015). This thesis presents an original body of sound art practice as a means through which to uncover and explore connections between sound art, experimental composition, kinetic art and sculpture. The term 'electromechanical' is used to identify this work, highlighting its particular concerns with the use of electrically animated or amplified materials. Through the production, exhibition, critical appraisal and contextualisation of the work new observations and distinctions within the field are presented. These include the identification of a 'closed system aesthetic' and the distinction between robotic and process driven approaches to electromechanical sound art. A further contribution to the field consists of a detailed consideration of sound art emerging from an intersection of experimental music and sculptural practices during the 1960s. The original works produced for the project, and their production are documented and described in detail alongside existing canonical and contemporary examples of sound art. Analysis of these works is informed by materialist and object-orientated critical positions, and science and technology studies. The method of art practice as research is described and extended in an original way that encompasses and applies a systems approach to creative practice.

This thesis discusses research conducted towards the development of an objective model that predicts punch in musical signals. Punch is a term often used by engineers and producers when describing a particular perceptual sensation found in produced music. Music is often characterised by listeners as being punchier yet the term is subjective, in terms of its meaning and the subsequent auditory effect on the listener. An objective model of punch would therefore prove useful for both music classification purposes and as a possible further metric that could be employed in music production and mastering metering tools. The literature reviewed within this body of work encompasses both subjective and objective audio evaluation methods in addition to low-level signal extraction and measurement techniques. The review concludes that whilst there has been a great deal of work in the area of semantic description and audio quality measurement, low-level analysis with respect to the perception of punch remains largely unexplored. The project was completed in a number of phases each designed to investigate the perceptual effects resulting from manipulation of test stimuli. The rationale behind this testing was to establish the key low-level descriptors relating to the punch attribute with the aim of producing a final objective and perceptually based model. The listening tests in each phase were conducted according to the ITU-R BS 1534-1 recommendation. In producing an objective model for the prediction of punch, listener perception to the attribute shows a strong correlation to the signal onset times, octave frequency band, signal duration and dynamic range. The punch measure obtained using the model is named PM95, where 95 indicates the upper percentile used in the measurement. Secondary measures were also obtained as a result of the iterative approach adopted. These are Inter-Band-Ratio (IBR), Transient to Steady-state Ratio (TSR) and Transient to Steady-state Ratio+Residual (TSR+R). These measures are useful in quantifying overall audio quality with respect to its dynamic range across frequency bands in addition to being a more reliable metric for defining the overall compression being applied to a piece of music. In addition, the latter two measures proposed may be useful in highlighting perceptual masking artefacts. The completed perceptual punch model was validated using the scores obtained from a large scale and independently conducted forced pairwise comparison test using expert listeners and a wide range of musical stimuli. From the results obtained, the PM95 measure showed a ‘very strong’ positive correlation with listener punch perception. Both r and rho coefficients (0.849 and 0.833) being significant at the 0.01 level (2-tailed). The PM95M measure, which is the PM95 measure divided by the mean value of punch frames also correlated very well with the perceptual punch scale having both r and rho coefficients (0.707 and -0.750) being significant at the 0.05 level (2-tailed). A real-time implementation of the punch model (and other measures proposed in this thesis) could be utilised as extensions to the metrics currently being used in Music Information Retrieval.

Virtual bands have been present in popular culture for decades, and they have become the topic of increased scholarly interest over the past few years. Despite this new work, however, there remains a need for more indepth critical studies into gender and ethnicity in virtual phenomena, as this approach promises to open up new areas of enquiry. Focussing on questions of gender and ethnicity, my thesis will investigate the mechanics through which identity is constructed in animated, puppet, and hologram virtual band characters. My thesis will draw on a range of empirical, theoretical, and ethnographic approaches in order to analyse how and to what extent virtual band characters are created and disseminated by those in the industry, on the one hand, and by fans, on the other. In particular, I consider this question through the concepts of agency (perceived and attributed), authorship, and authenticity, and in relation to the notion of suspension of disbelief, examining ways in which animation affords greater potential for forms of ‘layered awareness.’ Following a historical overview of virtual bands, and a critical appraisal of relevant theoretical perspectives on this topic, the thesis moves to a close reading of two case studies that reinforce and subvert gender and ethnic stereotypes commonly found in popular culture: Gorillaz and Vocaloid. These examples present different aspects of identity construction in virtual media, the former apparently led by the band’s creators, the other by its fans. Within Gorillaz, my discussion centres around the female guitarist Noodle, who, I will argue, is a modern-day Orientalist construction. By contrast, the chapters on Vocaloid draw on fan studies techniques to show that Vocaloid’s fan base contains a large, unexpected demographic, and that part of the fans’ dedication stems from their confirmed expectations of gender and ethnic identity in the Vocaloid characters.