Search

Extracting rich contextual information from study participants presents an interesting challenge when the expected results are uncertain. This article presents the design of a contextual geographic information system (GIS) to extract platial information from a multimodal data set (audio, video, and GPS) collected during a ‘think-aloud’ leisure walking study. WalkGIS enables transcriptions, labelling, and platial analysis to be performed within one system, with data being linked and coordinated to form linked video narratives.

As Human-Computer Interaction (HCI) engages with technologies that sense and actuate the body, there is a need to reconsider the human bodily experience. We present three case studies that each involve different forms of bodily experience: a breath-controlled amusement ride, a brain-controlled film, and an interactive musical duet with a physically actuated piano. We introduce a conceptual framework to describe how control becomes contested between human and computer in such experiences, using the three dimensions of: surrender of control, self-awareness of control, and looseness of control. We reveal how our experiences took users on journeys through control that traversed the space of these dimensions. We propose that our framework is not only relevant to playful cultural experiences, such as those charted in our case studies, but can also inform the design of embodied interaction more widely by emphasising the human experience of control when engaging with autonomous and bodily-focused systems, from future robots and vehicles to today’s gaze, speech and gestural interfaces.

JL: Kysytty McGookinilta, onko hän saanut luvan tiedoston avaamiseen (2.2.2018) Audio-based mobile technology is opening up a range of new interactive possibilities. This paper brings some of those possibilities to light by offering a range of perspectives based in this area. It is not only the technical systems that are developing, but novel approaches to the design and understanding of audio-based mobile systems are evolving to offer new perspectives on interaction and design and support such systems to be applied in areas, such as the humanities.

Audio-based mobile technology is opening up a range of new interactive possibilities. This paper brings some of those possibilities to light by offering a range of perspectives based in this area. It is not only the technical systems that are developing, but novel approaches to the design and understanding of audio-based mobile systems are evolving to offer new perspectives on interaction and design and support such systems to be applied in areas, such as the humanities.

“Climb!” is a musical composition that combines the ideas of a classical virtuoso piece and a computer game. We present a case study of the composition process and realization of “Climb!”, written for Disklavier and a digital interactive engine, which was co-developed together with the musical score. Specifically, the engine combines a system for recognising and responding to musical trigger phrases along with a dynamic digital score renderer. This tool chain allows for the composer’s original scoring to include notational elements such as trigger phrases to be automatically extracted to auto-configure the engine for live performance. We reflect holistically on the development process to date and highlight the emerging challenges and opportunities. For example, this includes the potential for further developing the workflow around the scoring process and the ways in which support for musical triggers has shaped the compositional approach.

Audio-based content, location and mobile technologies can offer a multitude of interactional possibilities when combined in innovative and creative ways. It is important not to underestimate impact of the interplay between location, place and sound. Even if intangible and ephemeral, sounds impact upon the way in which we experience the built as well as the natural world. As technology offer us the opportunity to augment and access the world, mobile technologies offer us the opportunity to interact while moving though the world. They are technologies that can mediate, provide and locate experience in the world. Vision, and to some extent the tactile senses have been dominant modalities discussed in experiential terms within HCI. This workshop suggests that there is a need to better understand how sound can be used for shaping and augmenting the experiential qualities of places through mobile computing.

Interpersonal touch is a key element of social behaviour, yet is largely ignored in HCI. In this paper, we describe two games which explore two extremes of interpersonal touch, and discuss key research questions relating to them.

Digital locative music technologies are transforming the ways in which we are able to manipulate and re-frame the meaning of architecture, landscape and art. In this note we explore and outline some of the key features that are associated with this. Defining the future possibilities and challenges that we have identified through our re-search in the area. Our work is supported by examples and critically assesses the relationship between physical and aural presence, examining the manipulation and re-framing of meaning through audio augmentation. “Is Space Audible?” [7].

Locative music experiences are often non-linear and as such they are co-created, as the final arrangement of the music heard is guided by the movements of the user. We note an absence of principles and guidelines regarding how composers should approach the structuring of such locative soundtracks. For instance, how does one compose for a non-linear, indeterminate experience using linear pre-composed placed sounds, where fixed musical time is situated into the indeterminate time of the user's experience? Furthermore, how does one create a soundtrack that is suitable for the location, but also functions as a coherent musical structure? We explore these questions by analyzing an existing `placed sound' work from a traditional music theory perspective and in doing so reveal some structural principals from `fixed' musical forms can also support the composition of contemporary locative music experiences.

We explore how to digitally augment musical instruments by connecting them to their social histories. We describe the use of Internet of Things technologies to connect an acoustic guitar to its digital footprint -- a record of how it was designed, built and played. We introduce the approach of crafting interactive decorative inlay into the body of an instrument that can then be scanned using mobile devices to reveal its digital footprint. We describe the design and construction of an augmented acoustic guitar called Carolan alongside activities to build its digital footprint through documented encounters with twenty-seven players in a variety of settings. We reveal the design challenge of mapping the different surfaces of the instrument to various facets of its footprint so as to afford appropriate experiences to players, audiences and technicians. We articulate an agenda for further research on the topic of connecting instruments to their social histories, including capturing and performing digital footprints and creating personalized and legacy experiences.

Musical soundtracks have great potential to enhance mobile walking experiences such as tours and guides, much as they already do for films and games. They also raise new challenges for composers as the music must fit a given landscape and respond to walkers’ trajectories. We present a design for an interactive mobile soundtrack to accompany a visit to a sculpture park. We describe the motivating factors used to structure soundtrack and drive the compositional process.

Music plays a vital role in accompanying all manner of our experiences. Soundtracks within films, video games and ceremonies possess a unique ability to enhance a narrative, suggest emotional content and mark key transitions. Moreover, soundtracks often achieve all of this without being the primary focus, on the contrary they typically assume a supporting role. The proliferation of mobile devices increasingly leads us to listen to music while on the move and musicians are seizing on locative technologies as a tool for creating new kinds of music that directly respond to people's movements through space. In light of these trends, we consider the interesting question of how composers might set about creating musical soundtracks to accompany mobile experiences. What we have in mind are experiences such as guided walks, tours and even pervasive games. The novelty of our research here is in the music serving as an accompaniment to enhance a location specific activity, much as a soundtrack does for a film. This calls for composers to take into account the key features of the experience, and its setting, to gently complement them through the music. We examine this process from a composer's perspective by presenting `from the field' an account of how they address the multifaceted challenges of designing a soundtrack for public sculpture park. We chart a composer's rationale as they developed a soundtrack for this site over multiple iterations of design, testing and refinement. We expose key relationships between the raw materials of music (melody, harmony, timbre, rhythm and dynamics) and those of the physical setting, that enable the composer to gracefully mesh the music into the fabric of the space. The result is to propose a set of recommendations to inform the composition of mobile soundtracks that we intend to guide future practice and research.

Digital musical experiences are commonplace, everyday occurrences for many of us. Digital technologies facilitate where, how and what we access, and they increasingly offer new methods for capturing, sharing, enhancing and supporting such musical experiences. Presented here is a sample of distinct research activities where we have engaged with a cross section of digital tools to augment novel musical experiences in the physical world, harnessing the mobile and the tangible.