Search

Recent forms of virtual reality (VR) have changed the way people interact with moving images in the entertainment and the games industry, as well as the way the content is created. Technological advances in VR have given an opportunity to create simulated environments that users can immerse themselves in, and sense almost as a real experience by combining film techniques and interactive media approaches. Storytelling in VR presents various challenges due to the spatial properties of the medium. Research suggests that engaging Non-Player Characters (NPCs) enhance storytelling and can do so by communicating emotions. Most VR war experiences use the concept of morale and emotions applied to a group of soldiers or individual characters. To address the need for more believable AI characters in VR, this project will investigate how emotions can be communicated more effectively in a VR war application. VR companies are increasingly using Artificial Intelligence (AI) and cloud technologies to develop a stronger ecosystem for NPCs. However, there are still significant number of limitations in terms of technology and immersive storytelling for VR with characters and props paying significant role for creating convincing VR experiences. This project will therefore aim to enhance storytelling in VR by inducing emotions through AI characters in a war environment inspired by realistic events from WWII.

As the technology progresses, VR becomes more widespread and finds uses outside of gaming. One of these applications is virtual museums (Alatrash et al.2021). As it has been shown that active participation with art pieces plays an important role in audience experience (Passebois Ducros & Euzéby 2021), virtual museums are gaining popularity all over the world. This makes VR a great tool that is potentially able to increase audience engagement with art, increasing its outreach. Museums all over the world have launched virtual tours, allowing audiences to wander through representations of gallery spaces and take a closer look at famous art pieces. In addition to digital representations of real museums, there are also completely digital museums that are accessible only through VR. However, these completely virtual museums rarely take advantage of their freedom from the restrictions of the physical world. Harnessing the potential of VR in this domain, we present Art(NET)work: an application able to visualise data in an interactive and immersive manner that can provide a much richer and more intuitive understanding of the data.

Review of Ludified, Edited by Marko Ciciliani, Barbara Lüneburg, and Andreas Pirchner (The Green Box, 2021, 432 pp, €38.00).

This chapter discusses practice-led and interdisciplinary methods in sound design research. Sound design can be undertaken as a creative activity in combination with theoretical investigation, so that theory informs practice, while conversely, the generation of new practical approaches provides new theoretical insights. The authors discuss how this may provide a suitable methodology for research in sound design, where sound artifacts themselves may transmit and advance knowledge in the field, but may also be complemented and disseminated through means of papers and publications. Furthermore, the authors also argue that practice-led methods can be used in combination with other empirical approaches, which may help to reinforce the outcomes. For instance, it is possible to develop artifacts of sound design or prototype systems, which are informed by, or provide a basis for, quantitative, qualitative, or mixed methods evaluations. How such practice-led and interdisciplinary studies may be formulated, is specifically explored in this chapter with reference to the authors’ previous research in affective sound design, which considers aspects of human emotion and perception as they relate to sound. Through the discussion of various projects, the chapter explores how practice-led and interdisciplinary approaches may lead to positive research outcomes, and may also have an enriching effect on the skillsets, knowledge and attributes of individual researchers and their teams.

This paper discusses the design of audio-visual sound toys in Cyberdream, a virtual reality music visualization. While an earlier version of this project for Oculus GearVR provided a journey through audio-visual environments related to 1990s rave culture, the most recent iteration for Oculus Quest provides the addition of three audio-visual sound toys, the discussion of which is the main focus of this paper. In the latest version, the user flies through synaesthetic environments, while using the interactive controllers to manipulate the audio-visual sound toys and 'paint with sound'. These toys allow the user to playfully manipulate sound and image in a way that is complementary to, and interfaces with, the audio-visual backdrop provided by the VR music visualization. Through the discussion of novel approaches to design, the project informs new strategies in the field of VR music visualizations.

In 2019 the virtual reality experience Cyberdream VR was presented at various events including the Event Two exhibition of computer art held at the Royal College of Art. This short demo for Oculus Gear VR provided a c.5 minute sonic journey, in which the user moves through a series of symbolic environments based on the futuristic techno-utopian or dystopian imagery of 90s rave fliers. These environments accompanied an original soundtrack of rave music and vaporwave, allowing users to enjoy the music whilst feeling as though they are inside synaesthetic virtual spaces related to the symbolic imagery of rave culture. This paper will discuss the subsequent development of this project, which is now being adapted for the Oculus Quest VR headset. Rajmil Fischman’s concept of ‘Music in the Holodeck’; the fictional ’holophonor’ instrument from the TV series Futurama; or the video game Rez Infinite, are among those that suggest paradigms for performing sound and visuals in VR. Drawing upon these ideas, the current iteration of Cyberdream allows the user to trigger ‘sound instruments’ with the Oculus Quest Touch controllers. This is conceived as a means through which the user can ‘paint with sound’ in 3D space, improvising with the music, whilst also generating synaesthetic imagery. The design of these sound instruments presents interesting challenges, in order for sound to be generated in correspondence with visual imagery, where these materials may complement both the music and spatial environments. Drawing on the author’s VJ work, at the macro level of compositional structure, the latest version of the project also seeks ways to provide a more continuous experience analogous to a DJ or VJ set, through blending of music tracks between scenes. This paper will discuss on-going work on this project, advancing the discourse regarding the visualisation of hardcore rave music in virtual reality.