Search

We have designed, developed, and tested an immersive virtual reality (VR) platform to practise protocols of the Self-attachment psychotherapy for treatment of depression and anxiety. The VR platform consists of both a high-end version based on Facebook's Oculus and a low-end mobile version based on Google's cardboard. Under the Self-attachment therapeutic framework, the causes of mental disorders are traced back to the individual's insecure attachment with their primary caregiver during childhood. Self-attachment offers a self-administrable protocol in which the user imaginatively creates an internal affectional bond with their childhood self, based on mental imagery, and then simulates optimal parent-child interactions to re-parent the childhood self to emotional maturity, thus earning themselves secure attachment. The platform offers a complementary visual tool for administering Self-attachment protocols; it enables the user to interact with their customised photorealistic childhood avatar (childhood self). The platform is developed in Unity 3D, a cross-platform game engine, and takes advantage of the itSeez3D Avatar SDK for generating a customised photorealistic 3D avatar head from a 2D childhood image of the user. The platform also offers facial and body animations for some of the basic emotions such as happy, sad, scared and joyful and it allows modifications to the avatar body and clothing colour. A pilot study to compare the use of the avatar-based approach (VR) to Self-attachment with the conventional photo based approach showed promising results. More than 85% of the participants reported that their photorealistic childhood avatar in VR, in both the high-end version and the low-end version, was more relatable than their ordinary childhood photo. Participants also reported that the high-end version of the VR platform was more realistic and immersive than the low-end mobile version.

The three core components of Affective Computing (AC) are emotion expression recognition, emotion processing, and emotional feedback. Affective states are typically characterized in a two-dimensional space consisting of arousal, i.e., the intensity of the emotion felt; and valence, i.e., the degree to which the current emotion is pleasant or unpleasant. These fundamental properties of emotion can not only be measured using subjective ratings from users, but also with the help of physiological and behavioural measures, which potentially provide an objective evaluation across users. Multiple combinations of measures are utilised in AC for a range of applications, including education, healthcare, marketing, and entertainment. As the uses of immersive Virtual Reality (VR) technologies are growing, there is a rapidly increasing need for robust affect recognition in VR settings. However, the integration of affect detection methodologies with VR remains an unmet challenge due to constraints posed by the current VR technologies, such as Head Mounted Displays. This EngD project is designed to overcome some of the challenges by effectively integrating valence and arousal recognition methods in VR technologies and by testing their reliability in seated and room-scale full immersive VR conditions. The aim of this EngD research project is to identify how affective states are elicited in VR and how they can be efficiently measured, without constraining the movement and decreasing the sense of presence in the virtual world. Through a three-years long collaboration with Emteq labs Ltd, a wearable technology company, we assisted in the development of a novel multimodal affect detection system, specifically tailored towards the requirements of VR. This thesis will describe the architecture of the system, the research studies that enabled this development, and the future challenges. The studies conducted, validated the reliability of our proposed system, including the VR stimuli design, data measures and processing pipeline. This work could inform future studies in the field of AC in VR and assist in the development of novel applications and healthcare interventions.

Real-time solutions to reducing the gap between virtual and physical worlds for photorealistic interactive Augmented Reality (AR) are presented. First, a method of texture deformation with image inpainting, provides a proof of concept to convincingly re-animate fixed physical objects through digital displays with seamless visual appearance. This, in combination with novel methods for image-based retargeting of real shadows to deformed virtual poses and environment illumination estimation using in conspicuous flat Fresnel lenses, brings real-world props to life in compelling, practical ways. Live AR animation capability provides the key basis for interactive facial performance capture driven deformation of real-world physical facial props. Therefore, Intermediated Reality (IR) is enabled; a tele-present AR framework that drives mediated communication and collaboration for multiple users through the remote possession of toys brought to life. This IR framework provides the foundation of prototype applications in physical avatar chat communication, stop-motion animation movie production, and immersive video games. Specifically, a new approach to reduce the number of physical configurations needed for a stop-motion animation movie by generating the in-between frames digitally in AR is demonstrated. AR-generated frames preserve its natural appearance and achieve smooth transitions between real-world keyframes and digitally generated in-betweens. Finally, the methods integrate across the entire Reality-Virtuality Continuum to target new game experiences called Multi-Reality games. This gaming experience makes an evolutionary step toward the convergence of real and virtual game characters for visceral digital experiences.

Facial performance capture (or face capture), a process of reconstructing, tracking and analysing the deformable geometry and appearance of the human face from visual input (e.g. RGB or RGB-D images), is a long-standing research topic in the fields of computer graphics and vision. Over the past two decades, the field of face capture has witnessed rapid progress, which has pushed the capture method's accuracy, speed and ease of use to a new level, and benefited a wide range of applications such as personalized facial avatar generation, face identification and facial animation. Nevertheless, it remains an open problem of improving the method's capturing robustness while keeping the method compute and data efficient. Moreover, the emerging virtual reality (VR) technologies for immersive interactions have posed new challenges to facial performance capture. The VR head-mounted display (HMD) occludes a large portion of the user's face, which makes conventional vision-based face capture methods less effective. Targeting at solving the aforementioned problems, this thesis first develops two novel face capture approaches for detecting sparse 2D facial landmarks and tracking dense 3D facial geometry respectively from a monocular RGB camera. Both approaches have been thoroughly evaluated on benchmark face image and video datasets. In comparison with the previous methods, they showcase improved capturing performance at very low data and computational cost. The proposed approaches have further been implemented into mobile and desktop facial tracking interfaces and validated on live video streams. For capturing the VR HMD user's facial expression with high-fidelity, the thesis proposes to combine a classic monocular 3D face reconstruction algorithm with a pioneering facial biosensing technique - Faceteq, which uses advanced electromyographic (EMG) sensors to capture facial muscle activities. This extends the facial performance capture from the traditional visual scene to the novel VR context, thereby providing a practical solution to achieve face-to-face communication with compelling facial expressions in virtual environment. Besides developing robust facial performance capture approaches, this thesis explores a new direction for applying those approaches to solve real-world problems. Specifically, it identifies the problem of automated facial nerve function assessment from visual face capture for facial palsy management. By systematically reviewing the principal studies on related topics, the thesis points out the challenges in the field and indicates promising directions for future work. What's more, it proposes a promising pathway to apply the face capture methods proposed in previous chapters onto automated facial nerve function assessment. To the best of my knowledge, this is the first review of its kind to be reported so far. Due to the interdisciplinary nature of the review, it can benefit multiple areas, including visual face capture, clinical facial palsy diagnosis and facial bioengineering.

Virtual Environments (VEs), as media providing high-level immersion, o er people an opportunity to mimic natural interpersonal interactions digitally. As a multi-player version of VEs, Shared Virtual Environments (SVEs) inherit VEs' advantages in enabling natural interactions and generating a high level of immersion, and will possibly play an increasingly important role in supporting digitally-mediated collaboration. Though SVEs have been extensively explored for education, entertainment, work, and training, as yet, few SVEs exist in the eld of supporting creative collaboration and as a result, research on the creative aspect of collaboration in SVEs remains very poor. This raises questions about how to design the user experience to support creative collaboration in SVEs. This thesis starts with an introduction and related work. An SVE called Let's Move (LeMo) will then be briefed. LeMo allows two people to interact with each other and create music collaboratively in its virtual environment. Three studies based on LeMo will then be presented: Study I explores how free-form visual 3D annotations and work identity in uence the collaboration, Study II and Study III explore how working space con gurations a ect the collaboration. Results indicate that: (1) 3D annotations can support people's collaborative music making (CMM) in SVEs through ve classes of use; (2) group territory, personal territory, and territorial behaviour emerge during collaborative music making in SVEs; (3) manipulating characteristics of personal space a ected collaborative behaviour, formation of territory, work e ciency, sense of contribution, preference, and so on. Then an overall discussion between studies is made and further implications for SVEs supporting collaborative music making (and other types of collaboration) in SVEs are given. The ndings of this thesis contribute towards the design of Human-Computer Interaction of Shared Virtual Environments focusing on supporting collaborative music making.

The aim of this thesis is to address and contribute to the research evidence on what it means to implement usable security interfaces in mobile devices given their increasing importance in our daily lives. We focused on user interfaces for authentication while incorporating HCI principles to develop and explore design and usability issues. We have done so by directly observing and reporting on our three different works, with special attention given to the user practices that create security lapses and usability drawbacks. The novelty of this research is the confirmation of the integral relationship across our three works showing that understanding and exploiting inherent human-factors such as memorability, tactile attributes, kinesiology and other inherent properties for human computer interface (HCI) designs improves security, usability and acceptance. One of our tasks is re-conceptualizing mobile device interfaces to make them both secure and usable. Our research indicates that interfaces that combine tactile and behavioural human characteristics into their basic design paradigm are more usable, and systems based on core graphical tokens with mnemonic properties result in higher memorability and familiarity values, while error recovery is strongly influenced by system design. Therefore system interfaces that can accommodate other familiar compound activities by users greatly reduce errors. One of our works proposes a new graphical authentication prototype interface to evaluate our research questions empirically. Our findings indicated that our first work, SemanticLock, had superior performance on key metrics such as password entry speed, memorability, encumbrance, user acceptance, usability, and likeability when compared with the PATTERN and PIN authentication techniques. Secondly, we explored the acceptable levels of complexity an interface could have in order to be secure and yet still usable by shifting the focus of our second work to the popular virtual reality device (VR) ecosystem. This work involved immersing participants in a virtual reality environment where they created passwords on virtual reality versions of popular mobile device authentication systems with different virtual reality interaction methods. The virtual reality system allowed us to evaluate the interface and interaction challenges by providing numerous heterogeneous interaction methods. We explored the outcome of porting the popular PATTERN authentication system into the virtual reality environment. We used the mobile device version of PATTERN as a control and the report indicates that PATTERN in VR is moderately fast, functionally usable and highly resistant to shoulder-surfing. For our third work, we examined various technological impediments that make it difficult to develop secure interfaces and proposed alternatives such as transparent interfaces that rely solely on the user's biometric signatures. We explored this challenge with a virtual reality (VR) based prototype based on kinesiology, effectively capturing the biometric movement of the participants in VR, and collecting the discerning identifying factors from each person via machine-learning assisted processes. We evaluated large datasets of head, eyes and hand movements using machine learning to create a continuous transparent biometric authentication system. We attained a classification accuracy of 99.7% and determined that kinesiologically replicating a valid participant with false-positive data is extremely difficult, thus making this system highly secure and usable. Our three major works conceptually explore the significant effects of user interactions on the effective security of their mobile devices. In our first study we determined various baselines in HCI that were used across the other works. The second work examined the practical effect of different interaction modes, while the third work explored using the interactions itself as a resultant effect of security. The compounding relevance among the works is the user.

This thesis presents research at the intersection of Extended Reality (XR) and Cultural Heritage - Virtual Heritage (VH). Prior research in VH has used Virtual Reality (VR) for immersive visualisation and Augmented Reality (AR) has been used for enhancing exhibitions. These reality technologies were primarily configured for use within museums, which are limited in terms of physical touch and handling capabilities and remote accessibility. Recent advances in XR and its rapid growth and affordability across various consumer devices are providing new opportunities for cultural heritage to be accessed outside the walls of the museums. However, virtual exhibitions and physical visits are different in many ways. Challenges remain for XR in its ability to emulate physicality and materiality, and the social environments within which interpretation, knowledge and ideas are exchanged. Meanwhile, the diversity of devices that are adopted as a consequence of user preferences necessarily means that technologies developed for users should take into account such a need as audiences are central and core to the activities of cultural institutions. In view of such a need, XR presents many challenges that are unsolved, one aspect of which is related to the concept of Extended Reality itself in its broad spectrum between physical reality and virtual reality. As users adopt their preferred devices within the XR spectrum, it has become essential to investigate the intersection of such use if communication within the spectrum is to be made possible. In addressing the need and challenge, this research designed a Hybrid Virtual and Augmented Reality (HVAR) environment and evaluated what could work if users were to socialise between the spectrum of realities. This research presents two frameworks that conceptualise virtual objects and virtual exhibitions for the connections between the spectrum of realities. Three technological developments were made for experimenting with hybridity in terms of interaction, communication and engagement for users with different XR devices. The initial experimental study indicated that users prefer to have control of movement and object information over guided tours and automatically triggered information, providing insights into the interaction design in virtual exhibitions. Follow up studies found that user acceptance of HVAR is strongly affected by social influences, and their communication within HVAR can be explained through the understanding of perceived presence. Building upon these, this research further investigated factors influencing user engagement within HVAR with an innovated objective measure of engagement using a wearable electroencephalogram (EEG) device combined with user activity monitoring at system runtime, suggesting the effects of object interactivity, User-Generated Content (UGC) and avatar proximity on user engagement. This research innovates the use of Hybrid Virtual and Augmented Reality environments. It has implications for the future design, evaluation and application of XR in various domains that necessitates sociality.

The use of VR has seen significant developments in recent years. The use of VR simulation tools offers new opportunities to develop, assess and train for decommissioning activities. Training for the decommissioning of nuclear reactors and facilities represent challenges that are unique to the nuclear industry. The protection of individuals from the hazardous effects of ionising radiation requires that mechanical assemblies are dismantled through the use of remote handling tooling. In this thesis, the use of simulation and VR to support the training of staff in the use of remote handling tools was studied. The decommissioning of the Consort research reactor provides a real-world example of a decommissioning project. Two simulators were created that were used to examine the development of remote handling skills using VR. The first simulator was used to explore the development of skills using remote handling tongs. This was in support of the cropping of the Consort reactor control rods. The second simulator was created to explore the development of skills using MSM. This was in support of the dismantling of the Consort reactor core assembly. The results from this research show that using VR; it is possible to develop skills using remote handling tools. The research further shows that the skills developed are transferable between the real and virtual worlds. The research also shows that VR can be a powerful tool for the development of processes. Improved processes lead to reduced costs, improvements to safety and lower the required level of skill for performing remote handling tasks.

The widespread use of virtual environments in today’s society leads to the importance of researching how using these virtual environments affect us, as well as how we can best use them. Video games are a very commonly used type of virtual environment/application of virtual environments. Video game research is rife with conflicting results, from studies into training, effects on emotion, to effects on visual attention. Chapter 2 considers the impacts of playing video games on visual attention and shows that the effects depend on the type of attentional process measured, and the video game genres played. Chapter 3 looks at how studies measure video game experience, and suggests a more sophisticated measure, including video game genres and platforms. This chapter also considers to what extent different video game genres are linked to different cognitive skills. Chapter 4 covers research between video game playing, task switching, and impulsivity. Chapter 5 shows that home video game playing (i.e. on home console platforms) affects both implicit memory and explicit memory, but mobile video game playing does not. Recent technological advances allowed the development of a newer form of virtual environment, virtual reality. Virtual reality has become more popular over the last few years in manufacturing and entertainment industries. However, studies into applying virtual reality to educational settings are limited. Chapter 6 presents a study that tests the effects of virtual reality on learning. The results show increased motivation and engagement with learning materials, when xxiv compared to learning with textbook-style or video materials. Chapter 7 compares learning in virtual reality, mixed reality and traditional lecture style modalities, and finds that participants report higher levels of engagement in both Virtual Reality and Mixed Reality conditions, and higher levels of positive emotions in the Virtual Reality condition. Implications for how individuals are affected by both of these types of virtual environments is discussed, including how they can be applied to learning.

360-degree immersive video applications for Head Mounted Display (HMD) devices offer great potential in providing engaging forms of experiential media solutions. Design challenges emerge though by this new kind of immersive media due to the 2D form of resources used for their construction, the lack of depth, the limited interaction, and the need to address the sense of presence. In addition, the use of Virtual Reality (VR) is related to cybersickness effects imposing further implications in moderate motion design tasks. This research project provides a systematic methodological approach in addressing those challenges and implications in 360-degree immersive video applications design. By studying and analysing methods and techniques efficiently used in the area of VR and Games design, a rigorous methodological design process is proposed. This process is introduced by the specification of the iVID (Immersive Video Interaction Design) framework. The efficiency of the iVID framework and the design methods and techniques it proposes is evaluated through two phases of user studies. Two different 360-degree immersive video prototypes have been created to serve the studies purposes. The analysis of the purposes of the studies ed to the definition of a set of design guidelines to be followed along with the iVID framework for designing 360-degree video-based experiences that are engaging and immersive.

Augmented reality (AR) is the enhancement of our perception of the world around us. It promises the ability to simultaneously interact with a virtual environment and real objects, providing an intuitive method to interact with data. However, AR displays must be lightweight and compact, and be able to display bright, high-resolution 3D images. Current AR displays are limited by bulky-form factors with a limited field of view (FOV), and the inability to display high quality 3D images without introducing side effects such as nausea and eyestrain. In the first part of this thesis an afocal Maxwellian display is analysed and modelled, the small eyebox limitation examined and a solution proposed. A prototype head mounted display unit was constructed as a demonstration. The display performance was analysed objectively and subjectively, and found to perform well over a wide accommodation range. However, the system is mechanically complicated to provide for a large IPD demographic, and the implementation of the eyebox expansion is too limited. An alternative curved reflector system was proposed to reduce weight, increase FOV, and increase the eyebox. The final part of this research examines waveguides as an AR display technology. The limitations of current systems are evaluated, and a novel switchable waveguide architecture proposed where light can be selectively directed towards the eye to form an image time sequentially. This enables higher quality images, with a wider FOV and increased efficiency, from a more transparent waveguide. A liquid crystal diffraction grating is developed as a switching element with a simulated diffraction efficiency of 20% and a geometry optimised for easy industrial manufacture. The possibility for such a system to include an integrated lens function is also considered, and an 80° FOV design presented.

The central hypothesis of this thesis is that probabilistic user interface design provides an effective methodology for delivering productive and enjoyable applications in virtual reality (VR) and augmented reality (AR). This investigation is timely given the recent emergence of mass-market virtual and augmented reality head-mounted displays and growing demand for tailored applications and content. The design guidance for building compelling and productive applications for these environments is, however, currently lagging the pace at which the underlying technology is maturing. This is problematic given important differences between designing conventional 2D interfaces and interactions and their embodied 3D counterparts. This dissertation investigates probabilistic user interface design as a method for solving many of the novel challenges encountered when developing applications for VR and AR. Probabilistic user interface design seeks to model the uncertain events in a system and identify, implement and validate strategies that drive improved system performance. This thesis addresses four research questions by applying a probabilistic treatment in four distinct but closely related case studies. These four case studies are selected to illustrate the flexibility and unique benefits offered by this method. Research Question 1 asks how the probabilistic qualities of an interface can be determined and how this can inform design. This question is investigated in the context of text entry in VR with a probabilistic characterisation performed on two fundamental design choices. Research Question 2 relates to the challenge of adapting AR applications to deployment contexts not knowable at design time. A study in which crowdworkers are employed to build a probabilistic understanding of the requirements for contextually adaptive AR answers this question. The text entry theme is revisited in answering Research Questions 3 which asks how high levels of input noise can be mitigated through inference. A probabilistic text entry method specifically tailored for use in AR is implemented and evaluated. Finally, Research Question 4 asks how the high dimensional design space in AR and VR applications can be efficiently explored to support ideal design choices. Interface refinement through probabilistic optimisation and crowdsourcing is shown to be highly efficient and effective for this purpose. A probabilistic treatment in the design process has many potential benefits, principle among which is increased robustness to circumstances unanticipated at design time. This thesis contributes to the toolset and guidance available to designers and supports the development of next generation user interfaces specifically tailored to virtual and augmented reality.

The aim of this doctoral thesis is to present a body of work aimed at improving performance and developing new methods for animating physical interactions using simulation in virtual environments. To this end we develop a number of novel parallel collision detection and fracture simulation algorithms. Methods for traversing and constructing bounding volume hierarchies (BVH) on graphics processing units (GPU) have had a wide success. In particular, they have been adopted widely in simulators, libraries and benchmarks as they allow applications to reach new heights in terms of performance. Even with such a development however, a thorough adoption of techniques has not occurred in commercial and practical applications. Due to this, parallel collision detection on GPUs remains a relatively niche problem and a wide number of applications could benefit from a significant boost in proclaimed performance gains. In fracture simulations, explicit surface tracking methods have a good track record of success. In particular they have been adopted thoroughly in 3D modelling and animation software like Houdini [124] as they allow accurate simulation of intricate fracture patterns with complex interactions, which are generated using physical laws. Even so, existing methods can pose restrictions on the geometries of simulated objects. Further, they often have tight dependencies on implicit surfaces (e.g. level sets) for representing cracks and performing cutting to produce rigid-body fragments. Due to these restrictions, catering to various geometries can be a challenge and the memory cost of using implicit surfaces can be detrimental and without guarantee on the preservation of sharp features. We present our work in four main chapters. We first tackle the problem in the accelerating collision detection on the GPU via BVH traversal - one of the most demanding components during collision detection. Secondly, we show the construction of a new representation of the BVH called the ostensibly implicit tree - a layout of nodes in memory which is encoded using the bitwise representation of the number of enclosed objects in the tree (e.g. polygons). Thirdly, we shift paradigm to the task of simulating breaking objects after collision: we show how traditional finite elements can be extended as a way to prevent frequent re-meshing during fracture evolution problems. Finally, we show how the fracture surface–represented as an explicit (e.g. triangulated) surface mesh–is used to generate rigid body fragments using a novel approach to mesh cutting.

In recent years, a number of large enterprises have gradually begun to use various Augmented Reality technologies to prominently improve the audiences’ view of their products. Among them, the creation of an immersive virtual interactive scene through the projection has received extensive attention, and this technique refers to projection SAR, which is short for projection spatial augmented reality. However, as the existing projection-SAR systems have immobility and limited working range, they have a huge difficulty to be accepted and used in human daily life. Therefore, this thesis research has proposed a technically feasible optimization scheme so that it can be practically applied to AR broadcasting and demonstrations. Based on three main techniques required by state-of-art projection SAR applications, this thesis has created a novel mobile projection SAR cameraman for AR broadcasting and demonstration. Firstly, by combining the CNN scene parsing model and multiple contour extractors, the proposed contour extraction pipeline can always detect the optimal contour information in non-HD or blurred images. This algorithm reduces the dependency on high quality visual sensors and solves the problems of low contour extraction accuracy in motion blurred images. Secondly, a plane-based visual mapping algorithm is introduced to solve the difficulties of visual mapping in these low-texture scenarios. Finally, a complete process of designing the projection SAR cameraman robot is introduced. This part has solved three main problems in mobile projection-SAR applications: (i) a new method for marking contour on projection model is proposed to replace the model rendering process. By combining contour features and geometric features, users can identify objects on colourless model easily. (ii) a camera initial pose estimation method is developed based on visual tracking algorithms, which can register the start pose of robot to the whole scene in Unity3D. (iii) a novel data transmission approach is introduced to establish a link between external robot and the robot in Unity3D simulation work-space. This makes the robotic cameraman can simulate its trajectory in Unity3D simulation work-space and project correct virtual content. Our proposed mobile projection SAR system has made outstanding contributions to the academic value and practicality of the existing projection SAR technique. It firstly solves the problem of limited working range. When the system is running in a large indoor scene, it can follow the user and project dynamic interactive virtual content automatically instead of increasing the number of visual sensors. Then, it creates a more immersive experience for audience since it supports the user has more body gestures and richer virtual-real interactive plays. Lastly, a mobile system does not require up-front frameworks and cheaper and has provided the public an innovative choice for indoor broadcasting and exhibitions.

The pace of advancement in emerging display and interface technologies supporting the development of mixed reality systems - those that exploit the existence of real-world objects to enhance the believability of virtual objects - is rapidly increasing. However, the availability of relevant human-system design standards underpinning the exploitation of interfaces is significantly lagging behind. To provide supporting principles to aid in the development and deployment of mixed reality systems, a series of studies was conducted to systematically investigate a range of design parameters relevant to mixed reality, and to determine the impact of those parameters on human-system performance, including cognitive and physical demands. An assessment of specific design standards was undertaken related to the performance of fundamental human-system interaction tasks in a mixed reality system. It was found that mixed reality is most suited to selection tasks, as opposed to more complex interaction tasks such as repositioning and rescaling virtual objects in 3D space. An evaluation was also made of the effects on presence of introducing physical "tangible" interface elements co-located with virtual content. The findings show that tangible interface objects have a significant positive effect on presence, in addition to usability and workload. Finally, an investigation was undertaken to assess the effects of vibration - a common, uncontrollable environmental condition - on human-system performance. Vibration is shown to have a significantly larger impact on accuracy for eye-based input than on head-based input when performing dwell-based interaction. The lowest frequencies have the greatest effect on accuracy, with higher frequencies producing similar effects to instances of zero vibration.

The purpose of this research project was to examine how immersive digital virtual technologies have the potential to expand the genre of interactive film into new forms of audience engagement and narrative production. Aside from addressing the limitations of interactive film, I have explored how interactive digital narratives can be reconfigured in the wake of immersive media. My contribution to knowledge stems from using a transdisciplinary synthesis of the interactive systems in film and digital media art, which is embodied in the research framework and theoretical focal point that I have titled Cynematics (chapter 2). Using a methodology that promotes iterative experimentation I developed a series of works that allowed me to practically explore the limitations of interactive film systems that involve non-haptic user interaction. This is evidenced in the following series of works: Virtual Embodiment, Narrative Maze, Eye Artefact Interactions and Routine Error - all of which are discussed in chapter 4 of this thesis. Each of these lab experiments collectively build towards the development of novel interactive 360° film practices. Funneling my research towards these underexplored processes I focused on virtual gaze interaction (chapters 4-6), aiming to define and historically contextualise this system of interaction, whilst critically engaging with it through my practice. It is here that gaze interaction is cemented as the key focus of this thesis. The potential of interactive 360° film is explored through the creation of three core pieces of practice, which are titled as follows: Systems of Seeing (chapter 5), Mimesis (chapter 6), Vanishing Point (chapter 7). Alongside the close readings in these chapters and the theoretical developments explored in each are the interaction designs included in the appendix of the thesis. These provide useful context for readers unable to experience these site-specific installations as virtual reality applications. After creating these systems, I established terms to theoretically unpack some of the processes occurring within them. These include Datascape Mediation (chapter 2), which frames agency as a complex entanglement built on the constantly evolving relationships between human and machine - and Live-Editing Practice (chapter 7), which aims to elucidate how the interactive 360° film practice designed for this research leads to new way of thinking about how we design, shoot and interact with 360° film. Reflecting on feedback from exhibiting Mimesis I decided to define and evaluate the key modes of virtual gaze interaction, which led to the development of a chapter and concept referred to as The Reticle Effect (chapter 6). This refers to how a visual overlay that is used to represent a user's line of sight not only shapes their experience of the work, but also dictates their perception of genre. To navigate this, I combined qualitative and quantitative analysis to explore user responses to four different types of gaze interaction. In preparing to collect this data I had to articulate these different types of interaction, which served to demarcate the difference between each of these types of gaze interaction. Stemming from this I used questionnaires, thematic analysis and data visualisation to explore the use and response to these systems. The results of this not only supports the idea of the reticle effect, but also gives insight into how these different types of virtual gaze interaction shape whether these works are viewed as games or as types of interactive film. The output of this allowed me to further expand on interactive 360° film as a genre of immersive media and move beyond the realm of interactive film into new technological discourses, which serves to validate the nascent, yet expansive reach of interactive 360° film as a form of practice. The thesis is concluded by framing this research within the wider discourse of posthuman theory as given that the technologies of immersive media perpetuate a state of extended human experience - how we interact and consider the theories that surround these mediums needs to be considered in the same way. The practice and theory developed throughout this thesis contribute to this discourse and allow for new ways of considering filmic language in the wake of interactive 360° film practice.

Virtual Reality has always represented a fascinating yet powerful opportunity that has attracted studies and technology developments, especially since the latest release on the market of powerful high-resolution and wide field-of-view VR headsets. While the great potential of such VR systems is common and accepted knowledge, issues remain related to how to design systems and setups capable of fully exploiting the latest hardware advances. The aim of the proposed research is to study and understand how to increase the perceived level of realism and sense of presence when remotely observing real places through VR headset displays. Hence, to produce a set of guidelines that give directions to system designers about how to optimize the display-camera setup to enhance performance, focusing on remote visual observation of real places. The outcome of this investigation represents unique knowledge that is believed to be very beneficial for better VR headset designs towards improved remote observation systems. To achieve the proposed goal, this thesis presents a thorough investigation of existing literature and previous researches, which is carried out systematically to identify the most important factors ruling realism, depth perception, comfort, and sense of presence in VR headset observation. Once identified, these factors are further discussed and assessed through a series of experiments and usability studies, based on a predefined set of research questions. More specifically, the role of familiarity with the observed place, the role of the environment characteristics shown to the viewer, and the role of the display used for the remote observation of the virtual environment are further investigated. To gain more insights, two usability studies are proposed with the aim of defining guidelines and best practices. The main outcomes from the two studies demonstrate that test users can experience an enhanced realistic observation when natural features, higher resolution displays, natural illumination, and high image contrast are used in Mobile VR. In terms of comfort, simple scene layouts and relaxing environments are considered ideal to reduce visual fatigue and eye strain. Furthermore, sense of presence increases when observed environments induce strong emotions, and depth perception improves in VR when several monocular cues such as lights and shadows are combined with binocular depth cues. Based on these results, this investigation then presents a focused evaluation on the outcomes and introduces an innovative eye-adapted High Dynamic Range (HDR) approach, which the author believes to be of great improvement in the context of remote observation when combined with eye-tracked VR headsets. Within this purpose, a third user study is proposed to compare static HDR and eye-adapted HDR observation in VR, to assess that the latter can improve realism, depth perception, sense of presence, and in certain cases even comfort. Results from this last study confirmed the author expectations, proving that eye-adapted HDR and eye tracking should be used to achieve best visual performances for remote observation in modern VR systems.

Literature in the study of human response to immersive virtual reality systems often deals with the phenomenon of presence. It can be shown that audio and imagery with spatial information can interact to affect presence in users of immersive virtual reality. It has also been shown that there is variation between individuals in the experience of presence in VR. The relationship between these effects has hitherto not been fully explored. This thesis aims to identify and evaluate the relation- ships between spatial audio rendering and spatial relationships between audio and visual objects and cognitive and personality differences which account for variation in the experience of presence in VR with spatial audio. This thesis compares mea- sures of audiovisual quality of experience with an existing model of presence in a factor-analytical paradigm. Scores on these dimensions were compared between en- vironments which are similar or dissimilar to pre-exposure conditions and compared between when participants believed they were listening to real-world or headphone rendered audio events. Differences between audiovisual treatments, including au- dio rendering methods and audiovisual spatial relationships, were compared with differences attributed to cognitive and personality factors identified as significant predictors using hierarchical modelling. It was found that audiovisual quality of experience relates to subscales of presence by being independent of reported visual realism and involvement, but combines linearly with these factors to contribute to ’spatial presence’, a dimension of overall presence which is identified as the largest component in the construct. It was also found that, although manipulation of the spatial information content of audiovisual stimuli was a predictor of audiovisual quality of experience, this effect is overshadowed by inter-participant variation. In- teractive effects between extraversion, empathy, ease of resolving visual detail, and systematisation and are better predictors of quality of experience and spatial pres- ence than the changes to spatial information content investigated in this work. An- choring biases are also identified which suggest that novel environments are rated higher on audiovisual quality than those geometrically similar to the pre-exposure environment. These findings constitute support for a novel framework for assessing propensity for presence in terms of an information-processing model.

This thesis concerns itself with an investigation of live performance on an augmented stage in front of an audience, where performers witness themselves as projection mapped virtual characters able to interact with projected virtual scenography. An interactive virtual character is projected onto the body of a performer, its movements congruent with the performer. Through visual feedback via a Head Mounted Display (HMD), the performer is virtually embodied in that they witness their virtualised body interacting with the virtual scenery and props of the augmented stage. The research is informed by a theoretical framework derived from theory on intermediality and performance, virtual embodiment and performative interaction. A literature review of theatrical productions and performances utilising projection identifies a research gap of providing the performer with a visual perspective of themselves in relationship to the projected scenography. The visual perspective delivered via the HMD enables the performer to perform towards the audience and away from the interactive projected backdrop. The resultant 'turn away' from facing an interactive screen and instead performing towards an audience is encapsulated in the concept of the 'Embodied Performative Turn'. The practice-based research found that changing the visual perspective presented to the performer impacted differently on performative interaction and virtual embodiment. A second-person or audience perspective, 'performer-as-observed' prioritises the perception of the virtual body and enhances performative behaviour but challenges effective performative interaction with the virtual scenography. Conversely, a first-person perspective, 'performer-as-observer' prioritises a worldview and enhances performative interaction, but negatively impacts on performative behaviour with the loss of performer-as-observed. The research findings suggest that the presentation of differing perspectives to the performer can be used to selectively enhance performative interaction and performative behaviour on an augmented stage.

There has been a recent development and growth of augmented reality devices across the world. The technology has the potential to enhance a user's perception and experience of the environment which surrounds them. For designers and engineers of 3D displays (e.g. vehicle HUDs) the technology offers an opportunity to form a more enriched and exciting experience for drivers. A key anticipation of AR is the capability to present virtual information at depth, as well as on displays with a wide field of view. Applying psychological theories and models of visual attention (e.g. perceptual load theory) and depth perception (e.g. modified weak fusion), the aim of the study was to establish human abilities with AR advancements in depth and useful field of view (UFOV). Conclusions were applied to the development of psychological theory and helpful design guidelines for AR designers and engineers. This has been done by creating two sets of experimental paradigms. The first set of experiments presented peripheral Landolt C target stimuli on a large display (120o), simultaneously whilst participants controlled a central tracking task (CTT). Over two experiments workload was manipulated on both central and peripheral tasks, including presentation time, contrast, tracking speed, and size. Results indicated that peripheral performance did not decrease as a function eccentricity. Conclusions suggested a demonstration of attentional selection dependent on stimulus parameters, with performance potentially dependent on more temporal characteristics than eccentricity. 'Best-practice' design guidelines are presented for AR HUDs with large display size. The second design paradigm implemented a two-alternative forced-choice psychophysical depth judgment task, assessing thresholds for correctly determining the depth of a virtual diamond in reference to a real-world object. Over a series of experiments a number of parameters were assessed including virtual image height in the visual field, virtual image cue properties (relative size and brightness), and background scene manipulations. Depth thresholds for the AR image were tested with the pedestrian target at 5m, 10m, 20m and 25m. Results demonstrated that depth resolution is poor in augmented reality compared to real-world environments, with conclusions indicating a potential difference in how relative cues of depth interact with each other in AR environments, i.e. in AR cues from virtual imagery and real-world background may not be effectively combined together. Design guidelines regarding where in depth a virtual image needs to be placed to 'match' that of a specific real-world object, and how relative cues to influence depth perception are presented.

Designers are known to use a blend of manual and virtual processes to produce design prototype solutions. For modern designers, computer-aided design (CAD) tools are an essential requirement to begin to develop design concept solutions. CAD, together with augmented reality (AR) systems have altered the face of design practice, as witnessed by the way a designer can now change a 3D concept shape, form, color, pattern, and texture of a product by the click of a button in minutes, rather than the classic approach to labor on a physical model in the studio for hours. However, often CAD can limit a designer's experience of being 'hands-on' with materials and processes. The rise of machine haptic1 (MH) tools have afforded a great potential for designers to feel more 'hands-on' with the virtual modeling processes. Through the use of MH, product designers are able to control, virtually sculpt, and manipulate virtual 3D objects on-screen. Design practitioners are well placed to make use of haptics, to augment 3D concept creation which is traditionally a highly tactile process. For similar reasoning, it could also be said that, non-sighted and visually impaired (NS, VI) communities could also benefit from using MH tools to increase touch-based interactions, thereby creating better access for NS, VI designers. In spite of this the use of MH within the design industry (specifically product design), or for use by the non-sighted community is still in its infancy. Therefore the full benefit of haptics to aid non-sighted designers has not yet been fully realised. This thesis empirically investigates the use of multimodal MH as a step closer to improving the virtual hands-on process, for the benefit of NS, VI and fully sighted (FS) Designer-Makers. This thesis comprises four experiments, embedded within four case studies (CS1-4). Case study 1and2 worked with self-employed NS, VI Art Makers at Henshaws College for the Blind and Visual Impaired. The study examined the effects of haptics on NS, VI users, evaluations of experience. Case study 3 and4, featuring experiments 3 and4, have been designed to examine the effects of haptics on distance learning design students at the Open University. The empirical results from all four case studies showed that NS, VI users were able to navigate and perceive virtual objects via the force from the haptically rendered objects on-screen. Moreover, they were assisted by the whole multimodal MH assistance, which in CS2 appeared to offer better assistance to NS versus FS participants. In CS3 and 4 MH and multimodal assistance afforded equal assistance to NS, VI, and FS, but haptics were not as successful in bettering the time results recorded in manual (M) haptic conditions. However, the collision data between M and MH showed little statistical difference. The thesis showed that multimodal MH systems, specifically used in kinesthetic mode have enabled human (non-disabled and disabled) to credibly judge objects within the virtual realm. It also shows that multimodal augmented tooling can improve the interaction and afford better access to the graphical user interface for a wider body of users.

The current era has seen unrestrained technological progress. New technologies are replacing common work practices and processes in several fields, such as industry, healthcare, and commerce. The main reasons for using these technologies is the reduction of time to develop products, increased quality of products and processes, and increases in security and communication. This thesis focuses on Virtual Reality (VR). VR is currently replacing old systems and modifying practices and processes in fields such as automotive, healthcare, training and psychological therapies. However, when applying technologies, it is fundamental to study the interaction between the technology and the end users. This thesis takes into consideration one aspect of human-computer interaction: trust. Trust has been seen as fundamental in technologies such as e-commerce, e-marketing, autonomous systems and social networks. This is because trust has been found to be associated with the intention to use a technology, and lack of trust could deter users from adopting the technology. This concept is particularly important for VR, since it is only recently gaining widespread adoption. However, studies on users’ trust in VR systems are limited in the literature and there is uncertainty regarding the factors which could influence end user trust. This research aimed at developing a model to investigate trust in VR. The goal was to identify the factors which have a theoretical influence on trust in VR through an analysis of the literature on trust in VR and trust in technology in general. This permitted the creation of a framework with usability, technology acceptance and presence as possible predictors of trust in VR. In order to validate this framework, six user experiments were conducted. The experiments investigated the relationships among the factors identified in the literature and their influence on trust. The first study was designed to explore possible methodological issues. The next three studies, conducted in collaboration with researchers at the University of Nottingham, analysed further the relationship between usability and trust and between technology acceptance and presence with trust. The fifth experiment was conducted to specifically explore the influence of presence on trust. The last study looked at all factors, and validated the framework, demonstrating that technology acceptance and presence are predictors of trust in VR, and usability has an indirect effect on trust, as it is a strong predictor of technology acceptance. This research generated a model which includes well-studied factors in human computer interaction and human factors and could be applied to study trust in VR for different systems. This model increases the amount of information on VR, both on an academic and industrial point of view. In addition, guidelines based on the model were generated to inform the evaluation of existing VR systems and the design of new ones.

360° media experiences have existed for centuries. Viewing painted panoramas, such as those displayed in the 18th-century rotunda in Leicester Square, was a popular Georgian pastime. Recent advances in capture, processing and display technology have created a surge of interest in the medium, with millions of people now viewing captured 360° media immersively. Despite the popularity of 360° media experiences, there are still substantial technical issues associated with production and distribution, and little research has been done that explores the end-user experience. As these experiences become commonplace, understanding the impact of such media becomes critical. In this work, two user studies were conducted that investigated the effects of 360° media of different forms. The first study looked at the impact of the display type when viewing cinematic virtual reality captured as 360° video. The study used three display types: a head-mounted display (HMD); a standard 16:9 TV; and a focus-plus-context display. Several metrics were explored, including spatial awareness, memory and narrative engagement. The second study investigated the impact of different transition types when exploring static scenes captured as multi-view 360° images in a HMD. The three transitions investigated were a linear movement through a 3D model of the scene, an instantaneous teleportation, and an image-based warp using Möbius transformations. Metrics investigated included spatial awareness, preference, and several subjective qualities such as the feeling of moving through the space. Additionally, an enabling technology for such experiences was investigated. Object removal in 360° images was explored in detail, with extensions for video described for simple cases. Taken together, these three projects further our current understanding of how 360° media can be implemented, and examine some of the most critical aspects of how users engage with these experiences.

The research presented in this thesis explores the use of consumer virtual reality technology for training, comparing its validity to more traditional training formats. The need to evaluate the effectiveness of training in virtual environments is critical as a wider audience gains access to an array of emerging virtual reality consumer devices. Training is an obvious use case for these devices. This is motivated by the well-known success of domain-specific training simulators, the ability to train in safe, controlled environments and the potential to launch training programs when the physical components required to complete a task are not readily available. In this thesis, we present four user studies that aim to compare the effectiveness of systems with varying levels of immersion for learning transfer of several tasks, ranging from object location spatial memory to more complex assembly procedures. For every study, evaluation of the effectiveness of training took place in a real-world, physical environment. The first two studies compare geometric and self-motion models in describing human spatial memory through scale distortions of real and virtual environments. The third study examines the effect of level of immersion, self-avatar and environmental fidelity on object location memory in real and virtual environments. The fourth study compares the effectiveness of physical training and virtual training for teaching a bimanual assembly task. Results highlight the validity of virtual environments for training. The overall conclusion is that virtual training can yield a resulting performance that is superior to other, more traditional training formats. Combined, the outcomes of each of the user studies motivate further study of consumer virtual reality systems in training and suggest considerations for the design of such virtual environments.

This thesis documents the design and application of Mixed Reality (MR) within a nuclear manufacturing cell through the creation of a Digitally Assisted Assembly Cell (DAAC). The DAAC is a proof of concept system, combining full body tracking within a room sized environment and bi-directional feedback mechanism to allow communication between users within the Virtual Environment (VE) and a manufacturing cell. This allows for training, remote assistance, delivery of work instructions, and data capture within a manufacturing cell. The research underpinning the DAAC encompasses four main areas; the nuclear industry, Virtual Reality (VR) and MR technology, MR within manufacturing, and finally the 4 th Industrial Revolution (IR4.0). Using an array of Kinect sensors, the DAAC was designed to capture user movements within a real manufacturing cell, which can be transferred in real time to a VE, creating a digital twin of the real cell. Users can interact with each other via digital assets and laser pointers projected into the cell, accompanied by a built-in Voice over Internet Protocol (VoIP) system. This allows for the capture of implicit knowledge from operators within the real manufacturing cell, as well as transfer of that knowledge to future operators. Additionally, users can connect to the VE from anywhere in the world. In this way, experts are able to communicate with the users in the real manufacturing cell and assist with their training. The human tracking data fills an identified gap in the IR4.0 network of Cyber Physical System (CPS), and could allow for future optimisations within manufacturing systems, Material Resource Planning (MRP) and Enterprise Resource Planning (ERP). This project is a demonstration of how MR could prove valuable within nuclear manufacture. The DAAC is designed to be low cost. It is hoped this will allow for its use by groups who have traditionally been priced out of MR technology. This could help Small to Medium Enterprises (SMEs) close the double digital divide between themselves and larger global corporations. For larger corporations it offers the benefit of being low cost, and, is consequently, easier to roll out across the value chain. Skills developed in one area can also be transferred to others across the internet, as users from one manufacturing cell can watch and communicate with those in another. However, as a proof of concept, the DAAC is at Technology Readiness Level (TRL) five or six and, prior to its wider application, further testing is required to asses and improve the technology. The work was patented in both the UK (S. R EDDISH et al., 2017a), the US (S. R EDDISH et al., 2017b) and China (S. R EDDISH et al., 2017c). The patents are owned by Rolls-Royce and cover the methods of bi-directional feedback from which users can interact from the digital to the real and vice versa.

Virtual World (VW) applications are digital 3D-simualtion environments used for military, education, sport and healthcare purposes. In this work, an architecture-inspired method called the Virtual World Applications Design Method (VWADM) is presented for designing such applications. The VWADM combines a generative design grammar (GDG) framework and a generative design agent (GDA) model to form the template of a mechanism that can be used for creating place models of VW applications during the design process. The GDG framework is used to develop GDGs for capturing spatial data that pertains to VW applications, transforming and storing such data and subsequently using it to create placeoriented conceptual models of these applications. In complement, the GDA model is used to develop GDAs for automatically generating physical models of VW applications, whose specifications are derived from the GDGs. The development of the VWADM is grounded in design science research (DSR), which is a research paradigm that provides a set of synthetic and analytical techniques and perspectives for understanding, conducting, evaluating and reporting creative work. DSR involves answering questions that are relevant to human problems through the design of novel and innovative artefacts. The thesis focuses on describing the design of the VWADM, demonstrating its functionality and evaluating its fitness and utility for designing VW applications. Details are presented of software that was developed to establish proof of concept for the VWADM. Through the development of the VWADM, the research establishes (1) a set of modelling concepts that use grammar and rules as the basis for capturing the semantics of the conceptual models of VW applications, (2) a set of visual notations that include basic shapes and symbols, which can be used to present the conceptual models of VW applications to stakeholders for them to manually review, (3) an intelligent component that is used for automatically generating physical models of VW applications and (4) a stepwise process, which involves layout design, object design, navigation design and interaction design and acts as a guide for the process of creating the conceptual and physical models of VW applications. Feedback from developers suggest that the VWADM can provide a significant contribution to automating the process of VW design specifically from the point of view of place-oriented design. Some ideas for future work are presented.

Grasping is a primary form of interaction with the surrounding world, and is an intuitive interaction technique by nature due to the highly complex structure of the human hand. Translating this versatile interaction technique to Augmented Reality (AR) can provide interaction designers with more opportunities to implement more intuitive and realistic AR applications. The work presented in this thesis uses quantifiable measures to evaluate the accuracy and usability of natural grasping of virtual objects in AR environments, and presents methods for improving this natural form of interaction. Following a review of physical grasping parameters and current methods of mediating grasping interactions in AR, a comprehensive analysis of natural freehand grasping of virtual objects in AR is presented to assess the accuracy, usability and transferability of this natural form of grasping to AR environments. The analysis is presented in four independent user studies (120 participants, 30 participants for each study and 5760 grasping tasks in total), where natural freehand grasping performance is assessed for a range of virtual object sizes, positions and types in terms of accuracy of grasping, task completion time and overall system usability. Findings from the first user study in this work highlighted two key problems for natural grasping in AR; namely inaccurate depth estimation and inaccurate size estimation of virtual objects. Following the quantification of these errors, three different methods for mitigating user errors and assisting users during natural grasping were presented and analysed; namely dual view visual feedback, drop shadows and additional visual feedback when adding user based tolerances during interaction tasks. Dual view visual feedback was found to significantly improve user depth estimation, however this method also significantly increased task completion time. Drop shadows provided an alternative, and a more usable solution, to dual view visual feedback through significantly improving depth estimation, task completion time and the overall usability of natural grasping. User based tolerances negated the fundamental problem of inaccurate size estimation of virtual objects, through enabling users to perform natural grasping without the need of being highly accurate in their grasping performance, thus providing evidence that natural grasping can be usable in task based AR environments. Finally recommendations for allowing and further improving natural grasping interaction in AR environments are provided, along with guidelines for translating this form of natural grasping to other AR environments and user interfaces.

In virtual reality, it is important that the user feels immersed, and that both the visual and listening experiences are pleasant and plausible. Whilst it is now possible to accurately model room acoustics using available scene geometry in real time, the perceptual attributes may not always be optimal. Previous research has examined high level control methods over attributes, yet have only been applied to algorithmic reverberators and not geometric types, which can model the acoustics of a virtual scene more accurately. The present thesis investigates methods of perceptual control over apparent source width and tonal colouration in virtual room acoustics, and is an important step towards and intelligent optimisation method for dynamically improving the listening experience. A review of the psychoacoustic mechanisms of spatial impression and tonal colouration was performed. Consideration was given to the effects early of reflections on these two attributes so that they can be exploited. Existing artificial reverb methods, mainly algorithmic, wave-based and geometric types, were reviewed. It was found that a geometric type was the most suitable, and so a virtual acoustics program that gave access to each reflection and their meta-data was developed. The program would allow for perceptual control methods to exploit the reflection meta-data. Experiments were performed to find novel, directional regions to sort and group reflections by how they contribute to an attribute. The first was a region of in the horizontal plane, where any reflection arriving within it will produce maximum perceived apparent source width (ASW). Another discovered two regions of and unacceptable colouration in front of and behind the listener. Any reflection arriving within these will produce unacceptable colouration. Level adjustment of reflections within either region should manipulate the corresponding attributes, forming the basis of the control methods. An investigation was performed where the methods were applied to binaural room impulse responses generated by the custom program in two different virtual rooms at three source-receiver distances. An elicitation test was performed to find out what perceptual differences the control methods caused using speech, guitar and orchestral sources. It was found that the largest differences were in ASW, loudness, distance and phasiness. Further investigation into the effectiveness of the control methods found that level adjustment of lateral reflections was fairly effective for controlling the degree of ASW without affecting tonal colouration. They also found that level adjustment of front-back reflections can affect ASW, yet had little effect on colouration. The final experiment compared both methods, and also investigated their effect on source loudness and distance. Again it was found that level adjustment in both regions had a significant effect on ASW yet little effect on phasiness. It was also found that they significantly affected loudness and distance. Analysis found that the changes in ASW may be linked to changes in loudness and distance.

Olfaction is a key human sense due to its close connection to the brain, specifically memory and decision making, along with its effects on behaviour and emotions. Clearly, in order for virtual environments to reach perceptual equivalence with the real world, olfaction needs to be incorporated. Perceptual equivalence is defined as the same cognitive response occurring with the users whether they are exposed to real or synthesized stimuli regardless of the levels of duration, intensity or nature. While there have been previous attempts to use olfactory stimuli in virtual environments, these have been limited and not addressed some key aspects of olfaction and its corresponding stimuli, such the effects of high odour concentrations and smell habituation. This thesis presents a framework for a physically accurate olfactory pipeline in order to help provide high fidelity perceptually equivalent virtual environments. The pipeline consists of stages for capturing, data storage, reproduction, and finally delivering stimuli to end users. In particular this thesis focuses on the final delivery stage. It presents a novel physical calibration for the olfaction delivery system and a new human perceptual calibration procedure to ensure a robust and repeatable experience. For the first half of the delivery stage, this thesis provides the blueprint for a physically accurate smell display. This is subsequently shown be capable of reproducing smell stimuli with both accuracy, to intended real world concentrations, and precision, such that the outputted olfactory stimuli are consistently presented at the specified level. In addition, similar to other existing olfactory displays, this smell display is a low cost solution as well as a straightforward design with the intrinsic ability to provide virtually, immediate temporal displacement. The validation of the physical calibration of the display is based on fundamental laws of chemistry to provide the same result under the same specified conditions. For the second half of the delivery stage, a perceptual calibration procedure is presented to calculate Just Noticeable Differences for olfactory stimuli. The purpose of this is to be able to create normalised stimuli levels which are perceivable by the general population but are also perceptually similar across different odours. The work provides two Just Noticeable Difference stimuli levels for three target single molecule odours which have been calibrated on the olfactory display based on a sample population (N=10). These determined stimuli levels can subsequently be utilised as generalisable points for further experimental use with participants when investigating both olfactory phenomena and relationships between olfactory and other sensual stimuli. A final experimental study is proposed in this thesis to clearly demonstrate the capabilities of the research and to explore the possibility of olfactory attention-masking phenomena, similar to those seen in the visual domain. Participants were asked to preferentially rank the odours presented to them based on images of the smell source. Odours were the same as those used in the perception calibration, and were presented as randomised dual conditions either in combination with the other odours or with a blank. Participants were consistently able to correctly identify the correct odours with the correct images, ranking the correct answers either primary or secondary. In addition, a number of preferences for some odours over others were identified. In conclusion, this thesis outlines a novel framework for addressing the physical and perceptual aspects of olfaction in order to provide an accurate representation of a real world equivalent olfactory experience. Experiments show humans are readily able to distinguish between odours when presented at the same time and the evidence obtained suggests there may be attention-masking phenomena in the olfactory domain.

An augmented reality (AR) system needs to track the user-view to perform an accurate augmentation registration. The present research proposes a conceptual marker-less, natural feature-based AR framework system, the process for which is divided into two stages - an offline database training session for the application developers, and an online AR tracking and display session for the final users. In the offline session, two types of 3D reconstruction application, RGBD-SLAM and SfM are integrated into the development framework for building the reference template of a target environment. The performance and applicable conditions of these two methods are presented in the present thesis, and the application developers can choose which method to apply for their developmental demands. A general developmental user interface is provided to the developer for interaction, including a simple GUI tool for augmentation configuration. The present proposal also applies a Bag of Words strategy to enable a rapid "loop-closure detection" in the online session, for efficiently querying the application user-view from the trained database to locate the user pose. The rendering and display process of augmentation is currently implemented within an OpenGL window, which is one result of the research that is worthy of future detailed investigation and development.

Virtual environments are increasingly being used for various applications. In recent times, with the advent of consumer grade systems, virtual reality has reached a critical mass and has exploded in terms of application domains. Extending from games and entertainment, VR is also applied in military training, remote surgery, flight simulation, co-operative work, and education. While all of these applications require careful design with respect to the interaction and aesthetics of the environment, they differ in their requirement of veridical realism: the impression of suspending disbelief to the point where perception in the environment is equal to the real world. At the same time, research in human centred disciplines have shown predictable biases and `errors' in perception with respect to the environment intended by the designer. This can be a challenge when certain perceptual phenomena prohibit the applicability of VR due to a discontinuation in what is rendered and what is actually perceived by the observer. This thesis is focused on a specific perceptual phenomenon in VR, namely that of distance compression, a term describing the widespread underestimation of distances that occur in VR relative to the real world. This perceptual anomaly occurs not only in visual based virtual environments, as compression has been observed and studied in auditory only and audiovisual spaces too. The contribution of this thesis is a novel technique for reducing compression, and its effectiveness is demonstrated in a series of empirical evaluations. First, research questions are synthesized from existing literature and the problem is introduced and explained through rigorous review of previous literature in the context of spatial audio, virtual reality technology, psychophysics, and multi-sensory integration. Second, the technique for reducing distance compression is proposed from an extensive literature review. Third, the technique is empirically tested through a series of studies involving human participants, virtual reality hardware, and bespoke software engineered for each study. Finally, the results from the studies are discussed and concluded with respect to the research questions proposed.

The use of virtual reality in academic and industrial research has been rapidly expanding in recent years therefore evaluations of the quality and effectiveness of virtual environments are required. The assessment process is usually done through user evaluation that is being measured whilst the user engages with the system. The limitations of this method in terms of its variability and user bias of pre and post-experience have been recognised in the research literature. Therefore, there is a need to design more objective measures of system effectiveness that could complement subjective measures and provide a conceptual framework for the fidelity assessment in VR. There are many technological and perceptual factors that can influence the overall experience in virtual environments. The focus of this thesis was to investigate how multisensory feedback, provided during VR exposure, can modulate a user’s qualitative and quantitative experience in the virtual environment. In a series of experimental studies, the role of visual, audio, haptic and motion cues on objective and subjective evaluations of fidelity in VR was investigated. In all studies, objective measures of performance were collected and compared to the subjective measures of user perception. The results showed that the explicit evaluation of environmental and perceptual factors available within VR environments modulated user experience. In particular, the results shown that a user’s postural responses can be used as a basis for the objective measure of fidelity. Additionally, the role of augmented sensory cues was investigated during a manual assembly task. By recording and analysing the objective and subjective measures it was shown that augmented multisensory feedback modulated the user’s acceptability of the virtual environment in a positive manner and increased overall task performance. Furthermore, the presence of augmented cues mitigated the negative effects of inaccurate motion tracking and simulation sickness. In the follow up study, the beneficial effects of virtual training with augmented sensory cues were observed in the transfer of learning when the same task was performed in a real environment. Similarly, when the effects of 6 degrees of freedom motion cuing on user experience were investigated in a high fidelity flight simulator, the consistent findings between objective and subjective data were recorded. By measuring the pilot’s accuracy to follow the desired path during a slalom manoeuvre while perceived task demand was increased, it was shown that motion cuing is related to effective task performance and modulates the levels of workload, sickness and presence. The overall findings revealed that multisensory feedback plays an important role in the overall perception and fidelity evaluations of VR systems and as such user experience needs to be included when investigating the effectiveness of sensory feedback signals. Throughout this thesis it was consistently shown that subjective measures of user perception in VR are directly comparable to the objective measures of performance and therefore both should be used in order to obtain a robust results when investigating the effectiveness of VR systems. This conceptual framework can provide an effective method to study human perception, which can in turn provide a deeper understanding of the environmental and cognitive factors that can influence the overall user experience, in terms of fidelity requirements, in virtual reality environments.

In today’s digital world, the use of diverse interconnected physical computer based devices, typified by the Internet-of-Things, has increased, leaving their internal functionalities hidden from people. In education, these hidden computational processes leave learners with s vagueness that obscures how these physical devices function and communicate in order to produce the high-level behaviours and actions they observe. The current approach to revealing these hidden worlds involves the use of debugging tools, visualisation, simulation, or augmented-reality views. Even when such advanced technologies are utilised they fail to construct a meaningful view of the hidden worlds that relate to the learning context, leaving learners with formidable challenges to understand the operation of these deep technologies. In working towards a solution to this challenge, this thesis combines computing and pedagogical models in a novel way to improve learning and teaching of computer science. This framework (a combination of computational and pedagogical models) is the core contribution of my thesis and has been given the name a Pedagogical Virtual Machine or PVM). It aims to extract learning-related information from the underlying computers that make up the education focus by providing a layered analysis of the technical and pedagogical processes that interact together for any given learning activity (in the context of learning about embedded computing). It adopts an object-oriented perspective that deconstructs computation and learning into objects, while taking inspiration from the Java Virtual Machine ideas, thereby building on existing paradigms of 'learning objects', 'object oriented programming' and virtual machines. In this way it addresses the challenge of linking both computing and learning activities in a standardised way across a multiplicity of computing and learning environments. The use of augmented reality (AR), and its ability to reveal deep technologies, further improves the effectiveness of the PVM framework introduced above by superimposing data, in real-time, concerning the invisible computational processes being explored by the learners. Applications that learners and developers might use this PVM tool for are typified by topics such as the Internet of Things, pervasive computing and robotics. The study presented in this thesis is based on the latter, robotics. The learning effectiveness of an AR based PVM approach was evaluated in two educational experiments that concerned students learning to program a desk-based robot (which was used as an example of an embedded-computer). The two experiments were conducted with computer-science students from the University of Essex and differed in the level of complexity. The results showed that PVM with AR significantly improved the students’ learning achievement and performance than those who used traditional learning environments. In addition, the PVM with AR made a positive difference to students learning experience, supporting the use of PVM with AR in educational activities that involved dealing with abstract technologies.

The rigid structure of medical hierarchies within UK hospitals can become the source of dissatisfaction and conflict for medical personnel, the repercussions of which can be disastrous for patients and staff. The research reported herein presents the results of an investigation into the use of Virtual Reality (VR) simulation and conventional story-boarded techniques to empower medical personnel to challenge decisions they feel are inappropriate. Prototype applications were crafted from a selection of transcribed ‘challenge events’ acquired from an opportunistic sample of clinical staff. Data obtained from an initial investigation were used to establish attitudes toward challenging and evaluate the findings of the literature to generate research questions and objectives. Medical personnel who engaged with both media as part of an experimental phase assessed their viability as potential training resources to help foster the ability to challenge. Analysis of this experiment suggested that both techniques are viable tools in the delivery of decision-making training and could potentially deliver impact into other applications within healthcare. To increase the realism of the training material, the technologies should be presented in a format appropriate for those with limited ‘gaming’ experience and allow a credible level of interaction with the environment and characters.

This PhD research project builds on thirteen years of enquiries as an academic practitioner, developing/critiquing interactive audio-visuals. This approach interweaves theory and practice so that both build on each other. It responds to the need for principles that inter-relate people, digital technologies and environments. The concept of "responsive environments" (RE) is offered as a starting point for the development of principles focusing on people within these environments. A responsive environment is "responsive" in the sense that some form of computer technologies are present and sensing/recording/reacting to people, and an "environment" in the sense that these activities are located in a place and that that place matters in terms of setting the scene, housing the technology and providing a context for the users/visitors. Common themes were extracted from the literature review to draw together previous and, for the most part, separate attempts at theory/practice relating to RE. These themes were complemented by research into contemporaneous activities in the areas of Augmented Reality, Mixed Reality and Locative Media to provided enhancements to the development of three practice projects. These enhancements together with the incorporation of Moore and Anderson's concepts of "patient", "actor", "reciprocator" and "referee" as roles available to those encountering REs led to specific research questions regarding roles, positions, opportunities for repurposing content, learning experiences, the use of sound, visuals and presence, and the assessment of values represented in and through a responsive environment. In each case these questions shift the emphasis of the research towards the experiencing of REs and what they enable rather than the technologies used only. The use of Schwartz and Halegoua's concept of the "spatial self" further focuses attention of the value in connecting digital expression with real spaces through an RE. This has led to a proposed conceptual framework and principles of practice that can be applied in the area of study of RE to nurture opportunities for participants and protagonists. The latter term is proposed as a means of acknowledging opportunities to make content/concepts in an RE as well as obtain and use them by participation. These opportunities are supported by both synchronous and asynchronous interactions through digital layers using online social media platforms. These platforms enable the archiving of content in a digital layer and/or possibilities for continued social interaction through a digital social layer in relation to the responsive environment. The incorporation of synchronous and asynchronous interactions through digital layers is a major contribution to the concept of REs. A further contribution is the use of the pioneering work of Gordon Pask in both the practice and theory of cybernetics as informing the concept of REs. Pask provided a formulation that expressed how content/concepts could be produced through relationships between people, computers and environments. This approach has been mirrored in other disciplines thus giving additional credence to its value. This discovery provides the impetus for further research, by academic practitioners and others, in this developing area of study.

The current era of advanced display technologies, such as head-mounted displays,smart glasses and handheld devices, has supported the usage of mixed reality and augmented reality concepts in smart educational classrooms. These advanced technologies have enabled enhanced collaborations and interactive communication between distance learners and local learners. Whilst being present, immersed and engaged are key factors in both the real and virtual worlds, they play particularly important roles in improving students collaborative learning performance during learning activities. However, few empirical studies have considered how using such interfaces may affect learning outcomes and whether students truly feel fully immersed and engaged in such environments. Furthermore, the lack of support and a conceptual architecture for collaborative mixed and augmented reality group learning activities is still a shortcoming for distance learning and teaching, and a significant challenge to researchers. This study demonstrates a conceptual framework that supports group distance learning and teaching collaboration around learning activities using mixed, augmented and virtual reality technologies. The study also explores learning effectiveness based on the following factors: students presence, engagement and immersion in smart environments. To evaluate these factors, we utilise several existing frameworks that have been applied to our mixed reality platform, MiRTLE+, to help us examine the learning outcomes and teaching experience gained from using these environments. The study was divided into two experimental phases, and 40 samples were examined to assess and compare the affordances of mixed reality interfaces within various collaborative learning scenarios, applying a card game activity (Uno) as a learning task. By comparing the real collaborative learning activities with the two-dimensional web-based activity, we found that novices had slightly better learning performance than the individuals using the web-based activity and that both phases mimicked reality. Novices and experts also felt significantly more present and immersed in the MiRTLE+ learning scenarios (due to the impact of the augmented reality interfaces) than in the web-based scenario.

Fidelity is of key importance if virtual environments are to be used as authentic representations of real life scenarios. However, simulating the multitude of senses that comprise the human sensory system is a computationally challenging task. With limited computational resources it is essential to distribute these carefully in order to simulate the most ideal perceptual experience for the user. This thesis investigates this balance of resources across multiple scenarios where combined audio, visual and olfactory cues are delivered to the user. Starting with bi-modal virtual environments where audio and visual stimuli are delivered to the users, a subjective experimental study, denoted as E1, was undertaken where participants (N = 35) allocated five fixed resource budgets for adjusting the quality of the displayed graphics and acoustics stimuli. In this experiment, increasing the quality of one of the sensory stimuli decreased the quality of the other. Findings demonstrate that participants allocate more resources to graphics, however as the computational budget is increased the allocation ratio between graphics and acoustics decreases significantly. Based on the results, an audio-visual quality prediction model is proposed and successfully validated against previously untested budgets and scenarios. The introduction of realistic olfactory stimuli is considered necessary if multisensory virtual environments are to be used as genuine representations of real life experiences. The estimation and delivery of smell impulses includes many challenges and significantly differs from the methods used for computing and displaying auditory and visual cues. Furthermore, the absence of a quality metric for assessing olfactory stimuli makes the introduction of an olfactory quality scale in the resource allocation framework significantly challenging. The work presented in this thesis investigates whether better spatial discretisation of the computational domain, a frequently used technique for increasing numerical stability in fluid transport simulations, can be used as a successful smell quality metric that can elicit a perceptual impact to the users of the virtual environment. In this context, better spatial discretisation levels are evaluated based on the estimation of the Just Noticeable Difference (JND) threshold for smell intensity using an experimental study (N = 20) and implemented in two phases. This experiment is denoted as E2 throughout this thesis. Findings demonstrate that the JND threshold is larger than the concentration differences given by progressively accurate smell transport simulations. This outcome enables computational savings from avoiding exhaustive smell transport simulations that provide no perceptual benefit to the user. Having considered the limitations associated with assessing smell impulses in terms of quality, a third experimental study is proposed, denoted as E3, and is exploited for resource allocation in tri-modal virtual environments. The experimental layout of E3 (N = 25) builds on the framework proposed in E1 including the delivery of physically accurate smell impulses to the user. The display of the smell bursts is implemented in a binary fashion (two levels or ON/OFF smell) along with the quality levels for the senses of vision and hearing as selected in E1. The smell concentration level presented in this experiment follows from the results of the JND threshold estimation for odour concentration presented in the psychophysics study E2. In conclusion, the research presented, shows that human preference criteria can be fully exploited in the design and delivery of multi-sensory virtual experiences. Experimental data can be used to derive computationally inexpensive prediction models that direct resource allocation in rendering pipelines where diverse sensory stimuli are simulated and delivered to the users.

I offer this thesis as an original and substantial contribution to knowledge in virtual natural environment design practice within computer and video games, by identifying areas of strong/weak practice and to develop a new design framework that utilises a cross-disciplinary approach for practitioners/students/researchers. The thesis combines theoretical frameworks as well practical guidance within a new design framework for virtual natural environment design. The themes relating to this work were examined through a contextual review that focused on previous professional practice as well as critical games produced during the last 30 years. The contextual review involved a detailed textual and visual-based historical survey of virtual landscapes, resulting in a practice-based exploration of virtual natural environment design in computer and video games. One of the main artefacts produced in this research, a three-volume book series titled Virtual Landscapes, presents for the first time these virtual spaces in a digitally enhanced manner through high-resolution panoramic imagery. A review of existing literature and current practice revealed that virtual natural environment design has so far been driven by mainly aesthetic principles and hinted that future emergent design practice should involve a cross/multi-disciplinary approach. The research proposes a new design framework for the creation of virtual landscapes that uses Landscape Character Assessments amongst other elements of environmental design. ShadowMoss Island is a practice-based exploration of how virtual natural environmental design can incorporate elements from Environmental Psychology and Geology, as well as personal reflections and observational analysis based on a field trip. The research proposes that psychological elements added to this new design framework can radically improve the success and impact of the final virtual natural environment. Another practice-based artefact, MindFlow, was created as a pre-production tool for the purpose of environmental design. The proposed tool enables the direct visualisation of collated multimedia (audio, images, video, annotations, design and decisions) in much more natural setting of a single visual space, allowing designers/artists to draw and influence the design and creation of virtual natural environments by bringing together all the different aspects in an intuitive and user-friendly manner. MindFlow helps solve the problem of designers/artists having to retain mental maps of image repositories structure by creating a single visual non-folder tree hierarchy virtual space. The research has significance to both professional and pedagogic practitioners working in the area of computer and video game natural environment design.

Mobile augmented reality has become an influential tool for digital content representation and visualization of media content in terms of enhancing users' experience and improving the adaptability and usability of typical augmented reality applications, such as in e-commerce shopping, virtual museum, or digital heritage scenarios. This research proposes a new Service Oriented Mobile AR Architecture called SOMARA, which includes a novel mobile AR client application. SOMARA takes advantage the ability to integrate third party content through service orientation. The SOMARA architecture enhances traditional standalone mobile AR applications with embedded media content by uniquely integrating a web service framework into an augmented reality client application to create more efficient and flexible mobile augmented reality applications that efficiently supports novel media content acquisition and visualization through appropriate access parameters. The proposed architecture requires access to media content through specific media content service providers, e.g. a museum commissioning an augmented reality based museum interactive — predetermined media content, or any third party with their own service APIs, e.g. the Victoria and Albert Museum API — related external media content. This approach allows relevant third party media content to be ‘mashed' via their public API with museums' augmented reality interactive's ‘embedded' media content in the SOMARA mobile AR client. In this way novel mobile AR interactive applications, such as a museum augmented reality interactive, can be created based on particular museum environment scenarios that integrate a museum visitor's experience with the interactive's cultural objects. Such experiences based on a SOMARA type museum augmented reality interactive can also be saved allowing visitors to take home their museum experience. SOMARA thus allows museum interactive experiences based on visualization of museums and third party media content physically located in the museum to be migrated to the visitor's home environment for further study, enjoyment and understanding. This unique feature, ability to effectively replay the experience at home, of the proposed system utilizes service-orientation to integrate third party media content, which is currently deficient from commercial augmented reality solutions.

The past decade has seen the release of numerous software packages aimed at enhancing anatomical education. However, there has been little research undertaken by the manufacturers of these products into the benefit or otherwise of these packages for student learning. In addition, while many of the existing software packages include interactive three-dimensional models, none of them truly offer virtual dissection i.e. the cutting through anatomical layers with a haptic (tactile) interface. This study investigated the haptic ‘dissection’ of a three dimensional digital model of the hand and wrist in anatomy education at both undergraduate (UG) and postgraduate (PG) levels. The model was used as a teaching and revision aid both prior to and after dissection of a real cadaver. A haptic enabled version of the model, allowing for real-time cutting was compared with a non-haptic version, using instead a keyboard and mouse ‘point and click’ style interface. Both versions were tested on students of gross anatomy in relation to test results and student experience. The model was based upon Computerised Tomography (CT) and photographic slice data from the Visible Human Project female data set. It was segmented and reconstructed using Amira® 5.2.2. From here each structure was exported as a separate STL file and imported into Geomagic Freeform® Modelling TM. Once imported into Freeform® Modelling TM, the individual structures each required varying degrees of re-modelling where detail had been lost during the segmentation process. Some smaller structures such as the nerves, veins and arteries were modelled freehand. The final model could be dissected using FreeForm® ModellingTM, the same software in which it was created. Using FreeForm® ModellingTM as a prototype VR dissector, each anatomical structure could be selected and virtually ‘dissected’ with the PHANTOM® Desktop™ haptic tool. Three methods of interacting with the model were identified: 1) using a cutting tool to cut through the selected layer; 2) using a selection paintball to first select and then delete the layer; and 3) using planes to cut the selected structure in standard anatomical views. The study ran over five successive years and was split into three discreet phases. Phase one compared the results of PG students across control, non-haptic and haptic groups. Phase two compared the results of UG students between control and haptic groups. Phase three compared the results of UG students across control, non-haptic and haptic groups. Due to small group sizes and a largely non-normal distributions the results were analysed using Mann-Whitney U tests. Results for all phases indicate that use of the model, both through haptic and non-haptic interfaces produced some significantly improved test results. The non-haptic version of the model performing equal or better than those with access to the haptic version. This is likely due to cognitive load being adversely affected by the addition of the haptic device. Some students reported that the haptic device was not intuitive to use and took some time to get used to, if at all. No student used either version of the model for more than five hours, with over 40% using it for less than one hour. It is possible that with increased exposure to the haptic device students may find it easier and thus beneficial. The findings of this study indicate that when used for a short period of time only ( < 5 hours) the haptic device may impede rather than enhance learning.

Televisions (TVs) and VR Head-Mounted Displays (VR HMDs) are used in shared and social spaces in the home. This thesis posits that these displays do not sufficiently reflect the collocated, social contexts in which they reside, nor do they sufficiently support shared experiences at-a-distance. This thesis explores how the role of TVs and VR HMDs can go beyond presenting a single entertainment experience, instead supporting social and shared use in both collocated and at-a-distance contexts. For collocated TV, this thesis demonstrates that the TV can be augmented to facilitate multi-user interaction, support shared and independent activities and multi-user use through multi-view display technology, and provide awareness of the multi-screen activity of those in the room, allowing the TV to reflect the social context in which it resides. For at-a-distance TV, existing smart TVs are shown to be capable of supporting synchronous at-a-distance activity, broadening the scope of media consumption beyond the four walls of the home. For VR HMDs, collocated proximate persons can be seamlessly brought into mixed reality VR experiences based on engagement, improving VR HMD usability. Applied to at-a-distance interactions, these shared mixed reality VR experiences can enable more immersive social experiences that approximate viewing together as if in person, compared to at-a-distance TV. Through an examination of TVs and VR HMDs, this thesis demonstrates that consumer display technology can better support users to interact, and share experiences and activities, with those they are close to.

In contemporary culture, human-to-human communication is becoming mediated through digital screens and virtual communication1. Our everyday lives are now lived in and between physical and virtual spaces, in a 'hybrid space', augmented with technologies, in which individuals increasingly perform online as digital versions of themselves: avatars. As a result, 'everyday life' has become 'everyday virtual life' in which new communication practices and social behaviours emerge. This research is a critique of everyday (virtual) life. As with Michel de Certeau’s analysis of the practice of everyday life in the 1980’s, in which the day-to-day practices of human behaviour were critiqued, the increased familiarity of 'everyday virtual life' necessitates new critical questioning: How do we live online? What are the common virtual communication practices? And how can this emergent 'hybrid space' be critically questioned through a participatory performance enquiry? This is an embodied practice, in which the contributions to knowledge are gained through the action and reflection of participatory performance; each raising new critical questioning and an embodied understanding of the critique of everyday (virtual) life: specifically the communication practices and human behaviours present in the digital, which are brought to the foreground through their re-framing and reperformance in a physical space. The research is presented as a textual-visual thesis and online platform, which together reveal the context, methodology, documentation and critical analysis of a body of practice-led research carried out by the author. The reader is invited to view both alongside each other: www.thepracticeofeverydayvirtuallife.com.

Technology has been widely implemented in tourism to enhance the tourism product and tourist experience. However, it remains a challenge for many tourism businesses in urban heritage areas to identify technological solutions and successfully implement them into their business strategy. Particularly for urban heritage sites, it is often challenging to provide information within and around a heritage site without disturbing or destroying the heritage image. Augmented Reality (AR) was identified as a suitable technology to overcome this challenge. However, while AR is not considered a new technology in many industries, and mobile tourism applications with AR functions exist, it has provided limited benefits to tourists due to the lack of using AR functions meaningfully. In addition, the development of wearable devices is increasingly focused on the use of augmented and virtual reality to enhance the user experience. Therefore, there is a need to explore methods of meaningful implementation of such technologies for the tourism industry. This study will investigate current standards of AR technology and tourist requirements in order to examine how to develop meaningful mobile AR tourism applications. For the development of a beneficial product, Quality Function Deployment (QFD), as a tool within Total Quality Management (TQM) was identified as the suitable theoretical background for the purpose of this study. QFD was developed in the theory of quality management, in particular TQM. QFD has since been employed across many industries including the hospitality and tourism sector in a number of studies. However, it has not yet been implemented in the urban heritage tourism context, presenting a knowledge gap. As QFD is designed to incorporate the customers’ view into the product design stage, the aim is to design a meaningful product that is valuable for the visitor and benefit the destination. Therefore, this research aims to eliminate the gap by generating a QFD model for the development of mobile AR tourism applications in the context of urban heritage tourism. Dublin was selected as the research site due to its rich urban heritage. After Dublin’s re-branding strategy in 2013, it has marketed itself with the brand image of ‘Digital Dublin’ in order to encourage the implementation of technology for the purpose of promoting urban heritage tourism. Therefore, Dublin was considered as the suitable research site for the purpose of this study. To achieve the research aim, a mixed method approach was employed for the primary research including semi-structured interviews, focus groups and questionnaires. The study was divided into three research phases. Research phase 1 investigates requirements of tourists, mobile AR application developers and industry experts from Dublin’s tourism industry for the development and use of mobile AR tourism applications in urban heritage tourism. 26 in-depth interviews were conducted with international tourists visiting Dublin in addition to 9 interviews with mobile AR application developers and industry experts from Dublin. Findings were analysed using thematic analysis, providing the knowledge base for Research Phase 2 – focus groups. The second research phase was conducted with tourists as a post-experience study. It aimed to confirm identified tourist requirements from the literature and Research Phase 1, and examine additional findings to design the questionnaire for the quantitative Research Phase 3. Research Phase 3 was conducted with domestic and international tourists in Dublin and provided an importance rating of identified tourist requirements for the development of the final QFD model for mobile AR tourism applications in the urban heritage tourism context. The outcomes from the quantitative research were analysed using confirmatory factor analysis (CFA) to reduce tourist requirements for the final QFD model. By identifying tourist requirements and translating them into respective technical characteristics, this study provides a set of technological elements in hierarchical order for the design of meaningful mobile AR tourism applications in urban heritage tourism. The final 18 requirements are in close alignment with the outcomes of the qualitative research of this study highlighting the need of immediate access to information regardless of time and place, while being able to filter the available information to the tourists’ interest and needs. The final QFD model for the development of mobile AR tourism applications in urban heritage tourism was presented as the outcome of this research. The main contribution to theory is the extension in the identification process of customer requirements for the QFD model. While previous studies in QFD have mainly focused on product functions, such as software and hardware capabilities, the QFD model in this study aims to provide a balance between functional requirements and behavioural characteristics of tourists that are driven by psychological aspects. This study not only explored tourist requirements as in previous studies, but furthermore user resistance factors that would result in avoiding mobile AR tourism applications in urban heritage tourism. Including these criteria as customer attributes in the HOQ, the QFD model was extended to include behavioural and psychological attributes of customers, which is valuable for further implementation of QFD particularly in social studies. This research will furthermore contribute to professionals, Destination Marketing Organisations (DMO) as well as mobile AR application developers alike, by providing a model for the development of mobile AR applications in the context of urban heritage tourism that has incorporated tourist requirements as well as mobile AR application developer considerations. Furthermore, it needs to be acknowledged that the requirements are not limited to mobile AR applications, but are considered particularly valuable for further research in the area of mobile application development for tourism purposes and are expected to be partially transferable to other technology implementations for tourism products.

It is known that limitations of the visual presentation and sense of presence in a virtual environment (VE) can result in deficits of spatial perception such as the documented depth compression phenomena. Investigating size and distance percepts in a VE is an active area of research, where different groups have measured the deficit by employing skill-based tasks such as walking, throwing or simply judging sizes and distances. A psychological trait called phenomenal regression (PR), first identified in the 1930s by Thouless, offers a measure that does not rely on either judgement or skill. PR describes a systematic error made by subjects when asked to match the perspective projections of two stimuli displayed at different distances. Thouless’ work found that this error is not mediated by a subject’s prior knowledge of its existence, nor can it be consciously manipulated, since it measures an individual’s innate reaction to visual stimuli. Furthermore he demonstrated that, in the real world, PR is affected by the depth cues available for viewing a scene. When applied in a VE, PR therefore potentially offers a direct measure of perceptual veracity that is independent of participants’ skill in judging size or distance. Experimental work has been conducted and a statistically significant correlation of individuals’ measured PR values (their ‘Thouless ratio’, or TR) between virtual and physical stimuli was found. A further experiment manipulated focal depth to mitigate the mismatch that occurs between accommodation and vergence cues in a VE. The resulting statistically significant effect on TR demonstrates that it is sensitive to changes in viewing conditions in a VE. Both experiments demonstrate key properties of PR that contribute to establishing it as a robust indicator of VE quality. The first property is that TR exhibits temporal stability during the period of testing and the second is that it differs between individuals. This is advantageous as it yields empirical values that can be investigated using regression analysis. This work contributes to VE domains in which it is desirable to replicate an accurate perception of space, such as training and telepresence, where PR would be a useful tool for comparing subjective experience between a VE and the real world, or between different VEs.

The aim of the present research was to determine a range of principles for the development of virtual natural environments (VNEs), using low-cost commercial-off-the-shelf simulation technologies, for bedside and clinical healthcare applications. A series of studies have been conducted to systematically investigate different aspects of the VNEs on a wide variety of participants, ranging from undergraduate and postgraduate students, hospital patients and clinicians, to West Country villagers. The results of these studies suggest that naturalistic environmental spatial sounds can have a positive impact on user ratings of presence and stress levels. High visual fidelity and real-world-based VNEs can increase participants’ reported ratings of presence, quality and realism. The choice of input devices also has a significant impact on usability with these types of virtual environment (VE). Overall, the findings provide a strong set of principles supporting the future development of VNEs. Highly transferrable tools and techniques have also been developed in order to investigate the exploitation of new digital technology approaches in the generation of believable and engaging real-time, interactive virtual natural environments that can be modified and updated relatively easily, thereby delivering a system that can be regularly modified and updated to meet the needs of individual patients.

Intermolecular binding underlies various metabolic and regulatory processes of the cell, and the therapeutic and pharmacological properties of drugs. Molecular docking systems model and simulate these interactions in silico and allow the study of the binding process. In molecular docking, haptics enables the user to sense the interaction forces and intervene cognitively in the docking process. Haptics-assisted docking systems provide an immersive virtual docking environment where the user can interact with the molecules, feel the interaction forces using their sense of touch, identify visually the binding site, and guide the molecules to their binding pose. Despite a forty-year research e�ort however, the docking community has been slow to adopt this technology. Proprietary, unreleased software, expensive haptic hardware and limits on processing power are the main reasons for this. Another signi�cant factor is the size of the molecules simulated, limited to small molecules. The focus of the research described in this thesis is the development of an interactive haptics-assisted docking application that addresses the above issues, and enables the rigid docking of very large biomolecules and the study of the underlying interactions. Novel methods for computing the interaction forces of binding on the CPU and GPU, in real-time, have been developed. The force calculation methods proposed here overcome several computational limitations of previous approaches, such as precomputed force grids, and could potentially be used to model molecular exibility at haptic refresh rates. Methods for force scaling, multipoint collision response, and haptic navigation are also reported that address newfound issues, particular to the interactive docking of large systems, e.g. force stability at molecular collision. The i ii result is a haptics-assisted docking application, Haptimol RD, that runs on relatively inexpensive consumer level hardware, (i.e. there is no need for specialized/proprietary hardware).

Context: When considering the use of Augmented Reality to provide navigation cues in a completely unknown environment, the content must be delivered into the environment with a repeatable level of accuracy such that the navigation cues can be understood and interpreted correctly by the user. Aims: This thesis aims to investigate whether a still image based reconstruction of an Augmented Reality environment can be used to develop a content delivery system that providers a repeatable level of accuracy for content placement. It will also investigate whether manipulation of the properties of a Spatial Marker object is sufficient to reduce object selection ambiguity in an Augmented Reality environment. Methods: A series of experiments were conducted to test the separate aspects of these aims. Participants were required to use the developed Keyframe Tagging tool to introduce virtual navigation markers into an Augmented Reality environment, and also to identify objects within an Augmented Reality environment that was signposted using different Virtual Spatial Markers. This tested the accuracy and repeatability of content placement of the approach, while also testing participants’ ability to reliably interpret virtual signposts within an Augmented Reality environment. Finally the Keyframe Tagging tool was tested by an expert user against a pre-existing solution to evaluate the time savings offered by this approach against the overall accuracy of content placement. Results: The average accuracy score for content placement across 20 participants was 64%, categorised as “Good” when compared with an expert benchmark result, while no tags were considered “incorrect” and only 8 from 200 tags were considered to have “Poor” accuracy, supporting the Keyframe Tagging approach. In terms of object identification from virtual cues, some of the predicted cognitive links between virtual marker property and target object did not surface, though participants reliably identified the correct objects across several trials. Conclusions: This thesis has demonstrated that accurate content delivery can be achieved through the use of a still image based reconstruction of an Augmented Reality environment. By using the Keyframe Tagging approach, content can be placed quickly and with a sufficient level of accuracy to demonstrate its utility in the scenarios outlined within this thesis. There are some observable limitations to the approach, which are discussed with the proposals for further work in this area.

The education of the radiography profession is based within higher education establishments, yet a critical part of all radiography programmes is the clinical component where students learn the practical skills of the profession. Assessments therefore not only have to assess a student’s knowledge, but also their clinical competence and core skills in line with both Health and Care Professions Council and the Society and College of Radiographers requirements. This timely thesis examines the possibility of using the Virtual Environment for RadioTherapy (VERT) as an assessment tool to evaluate a student’s competence so giving the advantage of a standard assessment and relieving time pressures in the clinical department. A mixed methods approach was taken which can be described as a Quantitative Qualitative design with the emphasis being on the Quantitative element; a so called QUAN  qual design. The quantitative evaluation compared two simulations, one in the virtual reality environment and another in the department using a real treatment machine. Students were asked to perform two electron setups in each simulation; the order being randomly decided and so the study would be described as a randomised cross-over design. Following this, qualitative data was collected in student focus groups to explore student perspectives in more depth. Findings indicated that the performance between the two simulators was significantly different, p < 0∙001; the virtual simulation scoring significantly lower than the hospital based simulation overall and in virtually all parameters being assessed. Thematic analysis of the qualitative data supported this finding and identified 4 main themes; equipment use, a lack of reality, learning opportunities and assessment of competence. One other sub-theme identified for reality was that of the environment and senses.