Search

We explore transdisciplinary collaborations between artists and roboticists across a portfolio of artworks. Brendan Walker's Broncomatic was a breath controlled mechanical rodeo bull ride. Blast Theory's Cat Royale deployed a robot arm to play with a family of three cats for twelve days. Different Bodies is a prototype improvised dance performance in which dancers with disabilities physically manipulate two mirrored robot arms. We reflect on these to explore how artists shape robotics research through the two key strategies of improvisation and provocation. Artists are skilled at improvising extended robot experiences that surface opportunities for technology-focused design, but which also require researchers to improvise their research processes. Artists may provoke audiences into reflecting on the societal implications of robots, but at the same time challenge the established techno-centric concepts, methods and underlying epistemology of robotics research., Comment: 16th International Conference on Social Robotics (ISCR 2024)

The recent release of the Apple Vision Pro has reignited interest in the metaverse, showcasing the intensified efforts of technology giants in developing platforms and devices to facilitate its growth. As the metaverse continues to proliferate, it is foreseeable that everyday environments will become increasingly saturated with its presence. Consequently, uncovering links to these metaverse items will be a crucial first step to interacting with this new augmented world. In this paper, we address the problem of establishing connections with virtual worlds within everyday environments, especially those that are not readily discernible through direct visual inspection. We introduce a vision-based approach leveraging Artcode visual markers to uncover hidden metaverse links embedded in our ambient surroundings. This approach progressively localises the access points to the metaverse, transitioning from coarse to fine localisation, thus facilitating an exploratory interaction process. Detailed experiments are conducted to study the performance of the proposed approach, demonstrating its effectiveness in Artcode localisation and enabling new interaction opportunities., Comment: This paper has been accepted by The 48th IEEE International Conference on Computers, Software, and Applications (COMPSAC 2024) for publication. It includes around 5600 words, 11 pages, 15 figures, and 1 table

Ongoing efforts to turn Machine Learning (ML) into a design material have encountered limited success. This paper examines the burgeoning area of AI art to understand how artists incorporate ML in their creative work. Drawing upon related HCI theories, we investigate how artists create ambiguity by analyzing nine AI artworks that use computer vision and image synthesis. Our analysis shows that, in addition to the established types of ambiguity, artists worked closely with the ML process (dataset curation, model training, and application) and developed various techniques to evoke the ambiguity of processes. Our finding indicates that the current conceptualization of ML as a design material needs to reframe the ML process as design elements, instead of technical details. Finally, this paper offers reflections on commonly held assumptions in HCI about ML uncertainty, dependability, and explainability, and advocates to supplement the artifact-centered design perspective of ML with a process-centered one.

While ethical challenges are widely discussed in HCI, far less is reported about the ethical processes that researchers routinely navigate. We reflect on a multispecies project that negotiated an especially complex ethical approval process. Cat Royale was an artist-led exploration of creating an artwork to engage audiences in exploring trust in autonomous systems. The artwork took the form of a robot that played with three cats. Gaining ethical approval required an extensive dialogue with three Institutional Review Boards (IRBs) covering computer science, veterinary science and animal welfare, raising tensions around the welfare of the cats, perceived benefits and appropriate methods, and reputational risk to the University. To reveal these tensions we introduce beneficiary-epistemology space, that makes explicit who benefits from research (humans or animals) and underlying epistemologies. Positioning projects and IRBs in this space can help clarify tensions and highlight opportunities to recruit additional expertise., Comment: Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24), May 11--16, 2024, Honolulu, HI, USA

We reflect on the design of a multispecies world centred around a bespoke enclosure in which three cats and a robot arm coexist for six hours a day during a twelve-day installation as part of an artist-led project. In this paper, we present the project's design process, encompassing various interconnected components, including the cats, the robot and its autonomous systems, the custom end-effectors and robot attachments, the diverse roles of the humans-in-the-loop, and the custom-designed enclosure. Subsequently, we provide a detailed account of key moments during the deployment and discuss the design implications for future multispecies systems. Specifically, we argue that designing the technology and its interactions is not sufficient, but that it is equally important to consider the design of the `world' in which the technology operates. Finally, we highlight the necessity of human involvement in areas such as breakdown recovery, animal welfare, and their role as audience., Comment: Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24), May 11--16, 2024, Honolulu, HI, USA

The recent Facebook rebranding to Meta has drawn renewed attention to the metaverse. Technology giants, amongst others, are increasingly embracing the vision and opportunities of a hybrid social experience that mixes physical and virtual interactions. As the metaverse gains in traction, it is expected that everyday objects may soon connect more closely with virtual elements. However, discovering this "hidden" virtual world will be a crucial first step to interacting with it in this new augmented world. In this paper, we address the problem of connecting physical objects with their virtual counterparts, especially through connections built upon visual markers. We propose a unified recognition framework that guides approaches to the metaverse access points. We illustrate the use of the framework through experimental studies under different conditions, in which an interactive and visually attractive decoration pattern, an Artcode, is used as the approach to enable the connection. This paper will be of interest to, amongst others, researchers working in Interaction Design or Augmented Reality who are seeking techniques or guidelines for augmenting physical objects in an unobtrusive, complementary manner., Comment: This paper includes 6 pages, 4 figures, and 1 table, and has been accepted to be published by the 2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC), Los Alamitos, CA, USA

Museums are interested in designing emotional visitor experiences to complement traditional interpretations. HCI is interested in the relationship between Affective Computing and Affective Interaction. We describe Sensitive Pictures, an emotional visitor experience co-created with the Munch art museum. Visitors choose emotions, locate associated paintings in the museum, experience an emotional story while viewing them, and self-report their response. A subsequent interview with a portrayal of the artist employs computer vision to estimate emotional responses from facial expressions. Visitors are given a souvenir postcard visualizing their emotional data. A study of 132 members of the public (39 interviewed) illuminates key themes: designing emotional provocations; capturing emotional responses; engaging visitors with their data; a tendency for them to align their views with the system's interpretation; and integrating these elements into emotional trajectories. We consider how Affective Computing can hold up a mirror to our emotions during Affective Interaction., Comment: Accepted for publication in CHI 2022

Software testing is often hindered where it is impossible or impractical to determine the correctness of the behaviour or output of the software under test (SUT), a situation known as the oracle problem. An example of an area facing the oracle problem is automatic image classification, using machine learning to classify an input image as one of a set of predefined classes. An approach to software testing that alleviates the oracle problem is metamorphic testing (MT). While traditional software testing examines the correctness of individual test cases, MT instead examines the relations amongst multiple executions of test cases and their outputs. These relations are called metamorphic relations (MRs): if an MR is found to be violated, then a fault must exist in the SUT. This paper examines the problem of classifying images containing visually hidden markers called Artcodes, and applies MT to verify and enhance the trained classifiers. This paper further examines two MRs, Separation and Occlusion, and reports on their capability in verifying the image classification using one-way analysis of variance (ANOVA) in conjunction with three other statistical analysis methods: t-test (for unequal variances), Kruskal-Wallis test, and Dunnett's test. In addition to our previously-studied classifier, that used Random Forests, we introduce a new classifier that uses a support vector machine, and present its MR-augmented version. Experimental evaluations across a number of performance metrics show that the augmented classifiers can achieve better performance than non-augmented classifiers. This paper also analyses how the enhanced performance is obtained., Comment: 32 pages, 11 figures

The global pandemic has driven musicians online. We report an ethnographic account of how two traditional folk clubs with little previous interest in digital platforms transitioned to online experiences. They followed very different approaches: one adapted their existing singaround format to video conferencing while the other evolved a weekly community-produced, pre-recorded show that could be watched together. However, despite their successes, participants ultimately remained unable to sing in chorus due to network constraints. We draw on theories of liveness from performance studies to explain our findings, arguing that HCI might orientate itself to online liveness as being co-produced through rich participatory structures that dissolve traditional distinctions between live and recorded and performer and audience. We discuss how participants appropriated existing platforms to achieve this, but these in turn shaped their practices in unforeseen ways. We draw out implications for the design and deployment of future live performance platforms.

This article brings together two distinct, but related perspectives on playful museum experiences: Critical play and hybrid design. The article explores the challenges involved in combining these two perspectives, through the design of two hybrid museum experiences that aimed to facilitate critical play with/in the collections of the Museum of Yugoslavia and the highly contested heritage they represent. Based on reflections from the design process as well as feedback from test users we describe a series of challenges: Challenging the norms of visitor behaviour, challenging the role of the artefact, and challenging the curatorial authority. In conclusion we outline some possible design strategies to address these challenges., Comment: Accepted for publication in the ACM Journal on Computing and Cultural Heritage, https://dl.acm.org/journal/jocch

We reflect on two museum visiting experiences that adopted the strategy of interpersonalization in which one visitor creates an experience for another. In the Gift app, visitors create personal mini-tours for specific others. In Never let me go, one visitor controls the experience of another by sending them remote instructions as they follow them around the museum. By reflecting on the design of these experiences and their deployment in museums we show how interpersonalization can deliver engaging social visits in which visitors make their own interpretations. We contrast the approach to previous research in customization and algorithmic personalization. We reveal how these experiences relied on intimacy between pairs of visitors but also between visitors and the museum. We propose that interpersonalization requires museums to step back to make space for interpretation, but that this then raises the challenge of how to reintroduce the museum's own perspective. Finally, we articulate strategies and challenges for applying this approach., Comment: Provisionally accepted for publication in the International Journal of Human-Computer Interaction (IJHCI)