Search

We gathered ethnographic evidence from surgeons about the concept of gesture-based control over the display of their patients’ radiographic scan data during surgery. This would give the surgeons direct access to their patients’ scan data without compromising their sterile working field and without needing to rely on other clinicians to interpret display instructions. Our approach involved interviewing surgeons and observing them in the operating theatre. We included evidence from earlier publications in this field and we used a grounded theory approach to analyse our data. Our findings address diversity across the surgical specialties, preoperative versus intraoperative use of the data, preferences for simple natural gestures, the role of another person controlling the display, broad system constraints and the willingness of surgeons to collaborate with their time and effort in this research.

We present the results of two controlled studies of free-improvised ensemble music-making on touch-screens. In our system, updates to an interface of harmonically-selected pitches are broadcast to every touch-screen in response to either a performer pressing a GUI button, or to interventions from an intelligent agent. In our first study, analysis of survey results and performance data indicated significant effects of the button on performer preference, but of the agent on performance length. In the second follow-up study, a mixed-initiative interface, where the presence of the button was interlaced with agent interventions, was developed to leverage both approaches. Comparison of this mixed-initiative interface with the always-on button-plus-agent condition of the first study demonstrated significant preferences for the former. The different approaches were found to shape the creative interactions that take place. Overall, this research offers evidence that an intelligent agent and a networked GUI both improve aspects of improvised ensemble music-making.

We consider tasks that require users to be aware of the proximity of two 3D surfaces and where one or both of these surfaces is changing over time. We consider situations where users need to quickly and accurately assess when and where the two surfaces approach each other and eventually intersect. Because occlusion in 3D visualisations remains an issue in the perception of such data, a complete, simultaneous perception of the proximity of two such surfaces could be helpful. We propose and implement a new, interactive, visualisation technique, “Proximity Map Projection” (PMP), to provide this assistance to users and describe a user study to investigate the effectiveness of PMP in a static scenario. This study found that PMP enabled faster and more accurate identification of regions of nearest proximity and greatest protrusion. As well as affirming the potential benefits of PMP, this study motivates several areas of further investigation of the technique

We present a study where a small group of experienced iPad musicians evaluated a system of three musical touch-screen apps and two server-based agents over 18 controlled improvisations. The performers’ perspectives were recorded through surveys, interviews, and interaction data. Our agent classifies the touch gestures of the performers and identifies new sections in the improvisations while a control agent returns similar messages sourced from a statistical model. The three touch-screen apps respond according to design paradigms of reward, support, and disruption. In this study of an ongoing musical practice, significant effects were observed due to the apps’ interfaces and how they respond to agent interactions. The “reward” app received the highest ratings. The results were used to iterate the app designs for later performances.

The group experience of synchronisation is a key aspect of ensemble musical performance. This paper presents a number of strategies for syncing performance information across networked iPad-instruments to enable creativity among an ensemble of improvising musicians. Acoustic instrumentalists sync without mechanical intervention. Electronic instruments frequently synchronise rhythm using MIDI or OSC connections. In contrast, our system syncs other aspects of performance, such as tonality, instrument functions, and gesture classifications, to support and enhance improvised performance. Over a number of performances with an iPad and percussion group, Ensemble Metatone, various syncing scenarios have been explored that support, extend, and disrupt ensemble creativity.