Search

In multiagent reinforcement learning scenarios, it is often the case that independent agents must jointly learn to perform a cooperative task. This paper focuses on such a scenario in which agents have individual preferences regarding how to accomplish the shared task. We consider a framework for this setting which balances individual preferences against task rewards using a linear mixing scheme. In our theoretical analysis we establish that agents can reach an equilibrium that leads to optimal shared task reward even when they consider individual preferences which aren't fully aligned with this task. We then empirically show, somewhat counter-intuitively, that there exist mixing schemes that outperform a purely task-oriented baseline. We further consider empirically how to optimize the mixing scheme.

Recent years have seen a growing focus on automated personalized services, with music recommendations a particularly prominent domain for such contributions. However, while most prior work on music recommender systems has focused on preferences for songs and artists, a fundamental aspect of human music perception is that music is experienced in a temporal context and in sequence. Hence, listeners’ preferences also may be affected by the sequence in which songs are being played and the corresponding song transitions. Moreover, a listener’s sequential preferences may vary across circumstances, such as in response to different emotional or functional needs, so that different song sequences may be more satisfying at different times. It is therefore useful to develop methods that can learn and adapt to individuals’ sequential preferences in real time, so as to adapt to a listener’s contextual preferences during a listening session. Prior work on personalized playlists either considered batch learning from large historical data sets, attempted to learn preferences for songs or artists irrespective of the sequence in which they are played, or assumed that adaptation occurs over extended periods of time. Hence, this prior work did not aim to adapt to a listener’s current song and sequential preferences in real time, during a listening session. This paper develops and evaluates a novel framework for online learning of and adaptation to a listener’s current song and sequence preferences exclusively by interacting with the listener, during a listening session. We evaluate the framework using both real playlist datasets and an experiment with human listeners. The results establish that the framework effectively learns and adapts to a listeners’ transition preferences during a listening session, and that it yields a significantly better listener experience. Our research also establishes that future advances of online adaptation to listener’s temporal preferences is a valuable avenue for research, and suggests that similar benefits may be possible from exploring online learning of temporal preferences for other personalized services.

In order to better understand the contributions of this book it is important to put them in context. Given the breadth and complexity of research work at the intersection of AI research and music informatics, a need arises to find useful ways of breaking down and organizing previous research. Indeed, in this section I will not only introduce a broad overview of the related literature, but also propose a unified framework for parsing this varied and complex body of work, delineating a taxonomy for music AI tasks, and mapping out the overall state of the art. In doing so, this chapter constitutes Contribution 1 of this book.

In Chap. 5 I discussed how the impact of music on human decision-making can be modeled. Subsequently in Chap. 6 I discussed how this impact can be leveraged by an agent to engender better interaction with a person. However, that is only one facet of person-agent interaction in musical context. The other includes a scenario in which people and machines actively collaborate in music generation. What would such an interaction be like? An important aspect of person-agent interaction, or of agents interacting with multiple people and/or other agents, is that of reasoning about preferences. Particularly in a domain such as music generation, people’s subjective tastes play a pivotal role, and reasoning about them when trying to collaborate is critical. This train of thought leads to a deeper question: how can multiple agents reason with each other in a shared task while also maintaining individual preferences that may be at odds with the shared task and with the preferences of others? Studying the balance between shared tasks and individual preferences in multiagent interaction is a significant step in fulfillment of Contribution 4 of this book, building towards multiagent music interaction as a meaningful step towards person-agent music generation.

As described in Sect. 1.1, one of the core problems in music recommendation over time, and in content recommendation in general, is that the distributional properties of music, and people’s musical tastes, change over time in ways that are nontrivial to track and predict. This informational challenge is a special case of concept drift—the change, either abrupt or gradual, in the underlying structure of data. Concept drift is a common and fundamental problem in machine learning in general, so any solutions designed to combat drift in content recommendation, even ones specific for that setting, are broadly applicable to ML research in general. In this chapter I focus on the application of reinforcement learning approaches to handle concept drift, specifically through model retraining, both in its general context and directly with respect to tracking people’s temporal listening habits as reflected by a real world dataset.

In the previous chapter I established, in fulfillment of Contribution 3 of this work, that music indeed alters people’s decision-making process in a nontrivial way, and that this effect can be modeled computationally. In this chapter I take this research thread two steps further: first to provide evidence that the impact of music on human decision-making goes beyond binary decisions; and second to show results suggesting that this effect can be modeled in real-time by a learning agent in order to induce better human-agent interaction. One of the core contributions of this book, described in Sect. 1.1, is the development of approaches and techniques for multiagent collaboration in musical environments, and specifically human-agent collaboration, constituting Contribution 4. Providing evidence that an agent can represent music as part of its world state and utilize this knowledge for better interaction is a substantial component of fulfilling Contribution 4. The other aspect of this contribution, which is the study of how multiple agents with varying tastes (which are not directly known to the other agents) can collaborate jointly, is studied in Chap. 7.

UT Austin Villa is a robot soccer team that has competed in the annual RoboCup soccer competitions since 2003. The team has won several championships and has inspired research contributions spanning many topics in robotics and artificial intelligence. This article summarizes some of these research contributions and provides a snapshot into the current development status of the team. Educational uses of the team's code bases are also presented.

In many learning and optimization tasks, the sample cost of performing the task is prohibitively expensive or time consuming. Learning is instead often performed on a less expensive task that is believed to be a reasonable approximation or surrogate of the actual target task. This paper focuses on the challenging open problem of performing learning on an approximation of a true target task, while simultaneously adapting the surrogate task used for learning to be a better representation of the true target task. Our work is evaluated in the RoboCup 3D simulation environment where we attempt to learn configuration parameters for an omnidirectional walk engine used by humanoid soccer playing robots.

This paper presents a novel algorithmic approach to music performance analysis. Previous attempts to use algorithmic tools in this field focused typically on tempo and dynamics alone. We base our analysis on ten different performance categories (such as bowing, vibrato and durations). We adapt phylogenetic analysis tools to resolve the inherent inconsistencies between these categories, and describe the relationships between performances. Taking samples from 29 different performances of two pieces from Bach's sonatas for solo violin, we construct a ‘phylogenetic’ tree, representing the relationship between those performances. The tree supports several interesting relations previously conjectured by the musicology community, such as the importance of date of birth and recording period in determining interpretative style. Our work also highlights some unexpected inter-connections between performers, and challenges previous assumptions regarding the significance of educational background and affiliat...

In this paper we study a bin-based estimation method of the amount of effort associated with code development. We investigate the following 3 variants to define the bins: (1) the same amount of data in a bin (SVM same #), (2) the same range for each bin (SVM same range) and (3) the bins made by Ward’s method (SVM Ward). We carry out evaluation experiments to compare the accuracy of the proposed SVM models with that of the \(\varepsilon \)-SVR using Welch’s t-test and effect sizes. These results indicate that the methods SVM same # (1) and SVM Ward (3) can improve the accuracy of estimating the amount of effort in terms of the mean percentage of predictions that fall within 25 % of the actual value.

This paper considers the problem of representative selection: choosing a subset of data points from a dataset that best represents its overall set of elements. This subset needs to inherently reflect the type of information contained in the entire set, while minimizing redundancy. For such purposes, clustering may seem like a natural approach. However, existing clustering methods are not ideally suited for representative selection, especially when dealing with non-metric data, where only a pairwise similarity measure exists. In this paper we propose $\delta$-medoids, a novel approach that can be viewed as an extension to the $k$-medoids algorithm and is specifically suited for sample representative selection from non-metric data. We empirically validate $\delta$-medoids in two domains, namely music analysis and motion analysis. We also show some theoretical bounds on the performance of $\delta$-medoids and the hardness of representative selection in general.