Search

In a ten-week project with nine school classes across the North West of England we explored ocean health with IT-enabled solutions. We describe the activities carried out under headings of participation, learning, and design. Participation activities, which included recruitment, focused on setting the parameters for children’s inclusion and ensuring they understood how data might be used, and that handing in artefacts to the research team was their choice. Learning happened in an environment of contextual relevance that enabled children to develop data literacy whilst we could explore relevant research questions. Design was a journey from individual to whole-class design, while developing engineering thinking and social cohesion. We reflect on the journey showing that children learned from the activities and acquired a new enthusiasm for their local coastline. We reflect on how our inclusive approach can broaden HCI research to wider communities of children and encourage others to apply our model.

Social media sites have gained popularity in the past decade by enabling people to create and share content online easily. However, social media cannot engage users on a level beyond content prosumption to create immersive experiences. A promising technology to create such experiences is multimodal virtual reality. REVERIE is a multimodal framework with an integrated workflow for developing immersive experiences, including various forms of virtual representation of human users (e.g., replicas and avatars). To demonstrate the social and collaborative capabilities of the framework, the authors created the Virtual Hangout. It features a VE where people can meet online to socialise and collaborate. This paper presents a study designed to evaluate user immersion using participants with variable computer skills. The results showed that participants experienced engagement-based immersion but not presence. Barriers to immersion were the absence of pleasurable interactions and the participants' computer skills. The authors propose DGs for deeply immersive experiences in Virtual Hangout and similar systems.

The most common method of assessing the Quality of Musician’s Experience (QoME) in Network Music Performance (NMP) is to perform a subjective study, where the participants evaluate their experience via questionnaires. Translating experiences into metrics is not an exact science, though: in our recent study on the effects of audio delay and quality in the QoME of NMP, the responses had a high variance and were inconsistent. To strengthen our confidence in the results of the subjective study, we analyzed video recordings of the participants using machine learning. Specifically, we used Facial Expression Recognition (FER) to detect the emotions felt by the participants and then compare them with their questionnaire responses. In addition to pointing out interesting phenomena that were not apparent from the questionnaires, this multimodal analysis showed analogies between the emotions felt (as captured by FER) and the emotions expressed (as captured by the responses).

For Network Music Performance (NMP), low end-to-end delay is probably the most critical factor affecting the Quality of Musicians’ Experience (QoME), as inflated delays inhibit the synchronization between musicians. Even though customized NMP tools can greatly reduce application-layer delays, generally available Internet connections impose considerable network-layer delays to NMP sessions in wide-area settings. In order to empirically assess the delay tolerance of NMP, multiple pairs of musicians were asked to perform in a controlled network environment, while we varied the end-to-end delay. The musicians were then asked to evaluate a number of metrics. The statistical analysis of the results indicates that the tolerance of actual musicians to delay is higher than previously thought, thus expanding the applicability of NMP to longer distances.

One in every six persons in the UK suffers a hearing loss, either as a condition they have been born with, or they developed during their life. Nine hundred thousand people in the UK are severely or profoundly deaf. Based on a study by Action on Hearing Loss UK in 2013 only 17 percent of this population, can use the British Sign Language (BSL). That leaves a massive proportion of people with a hearing impediment who do not use sign language struggling in social interaction and suffering from emotional distress. It also leaves even a larger proportion of Hearing people who cannot communicate with those of the deaf community. This paper presents a Serious Game (SG) that aims to close the communication gap between able hearing people and people with hearing impairment by providing a tool that facilitates BSL learning targeting the adult population. The paper presents the theoretical framework supporting adult learning based on which a SG game using Virtual Reality (VR) technology has been developed. The paper explains the experimental framework of the study. It presents the creation of the research instruments to facilitate the study comprising of a SG that integrates video and conventional video-based educational material. It reports and analyses the study results that demonstrate the advantage of the SG in effectively supporting users learning a set of BSL signs. It also presents qualitative outcomes that inform the further development of the game to serve learning needs. The paper closes with conclusions, directions for further development of this educational resource, and future studies.

Several research projects in spatial cognition have suggested Virtual Environments (VEs) as an effective way of facilitating mental map development of a physical space. In the study reported in this paper, we evaluated the effectiveness of multimodal real-time interaction in distilling understanding of the VE after completing gamified educational tasks. We also measure the impact of these design elements on the user’s experience of educational tasks. The VE used reassemble an art gallery (namely REVERIE VG), and it was built using REVERIE (Real and Virtual Engagement In Realistic Immersive Environment), a framework designed to enable multimodal communication on the Web. We compared the impact of REVERIE VG with an educational platform called Edu-Simulation for the same gamified educational tasks. We found that the multimodal VE had no impact on the ability of students to retain a mental model of the virtual space. However, we also found that students thought that it was easier to build a mental map of the virtual space in REVERIE VG. This means that using a multimodal Collaborative Virtual Environment (CVE) in a gamified educational experience does not benefit spatial performance, but also it does not cause distraction. The paper ends with future work and conclusions and suggestions for improving mental map construction and user experience in multimodal CVEs.

In Network Music Performance (NMP), audio quality and audio delay are considered to be the most critical variables affecting the Quality of Musicians’ Experience (QoME). In order to quantify the extent to which these parameters affect QoME, we executed a pilot study where eight musicians performed music in pairs in a controlled NMP setting and were asked to evaluate eight variables related to perception, while the end-to-end delay and the quality of the exchanged audio were varied. We present the design and execution of this experiment and discuss its results and their implications for the tolerance of musicians to increased delay and degraded audio quality.

Considerable research has been conducted on network and system level metrics related to Network Music Performance (NMP). However, little empirical evidence is currently available to assess the actual Quality of Musician's Experience (QoME) over NMP. We propose a research framework that integrates both subjective and objective aspects of musicians' experience, by explicitly considering the psychological state and profile of each musician, the environmental acoustic variables, the music performance variables and the Quality of Service of the network as the key dimensions that impact QoME. We will use the proposed framework to drive empirical studies designed to explore the QoME of musicians performing over the Internet; this paper is a first step in this direction.

Many existing open source systems provide support for Network Music Performance (NMP), with each one catering to a specific system and usage scenario. As our research in evaluating the Quality of Experience (QoE) of NMP systems as perceived by musicians involves widely different scenarios and requires extensive instrumentation of the platform, we built a new NMP system, Aretousa. Our system offers a large number of configuration and monitoring options, without sacrificing latency, the most critical factor for NMP. To show that Aretousa provides flexibility while being competitive with the state of the art in terms of latency, we present measurements comparing it against JackTrip in multiple setups over a high speed research network.

This article presents the design and evaluation of a gamified mobile tour guide prototype for route-finding in historical attractions. The mobile tour guide prototype uses photographs of landmarks carefully chosen to guide visitors on routes of cultural interest in the castle. It also features a fully multimodal interface using an Embodied Conversational Agent (ECA). The authors compared the impact of the multimodal interface to a non-multimodal control (narrated text) for the same navigational tasks. Participants experienced routes of variable difficulty (simple vs complex) in the lab under simulated mobile conditions. The experimental results substantiated aspects of the ECA-based interface related to visitors' subjective experiences, but not objective task performance. Finally, the authors discuss future work which will focus on running the second round of evaluations in the field with an improved version of the prototype, demonstrating features such as user planned routes and route re-calculation.

This paper discusses the enactment and evaluation of Embodied Conversational Agents (ECA) capable of affective communication in Collaborative Virtual Environments (CVE) for learning. The CVE discussed is a reconstruction of the European Parliament in Brussels developed using the REVERIE (Real and Virtual Engagement In Realistic Immersive Environment) framework. REVERIE is a framework designed to support the creation of CVEs populated by ECAs capable of natural human-like behaviour, physical interaction and engagement. The ECA provides a tour of the virtual parliament and participates in the learning activity as an intervention mechanism to engage students. The ECA is capable of immediacy behaviour (verbal and non-verbal) and interactions to support a dialogic learning scenario. The design of the ECA is grounded on a theoretical framework that addresses the required characteristics of the ECA to successfully support collaborative learning. In this paper, we discuss the Heuristic Evaluation of the REVERIE ECA which revealed a wealth of usability problems that led to the development of a list of design recommendations to improve their usability, including its immediacy behaviours and interactions. An ECA capable of effectively creating rapport should result in more positive experiences for participants and better learning results for students in dialogic learning scenarios. Future work aims to evaluate this hypothesis in real-world scenarios with teachers and students participating in a shared virtual educational experience

One in every six persons in the UK suffers a hearing loss, either as a condition they have been born with or a disorder they acquired during their life. 900,000 people in the UK are severely or profoundly deaf and based on a study by Action On Hearing Loss UK in 2013 only 17 percent of this population, can use the British Sign Language (BSL). That leaves a massive proportion of people with a hearing impediment who do not use sign language struggling in social interaction and suffering from emotional distress, and an even larger proportion of Hearing people who cannot communicate with those of the deaf community. This paper presents a theoretical framework for the design of interactive games to support learning BSL supporting the entire learning cycle, instruction, practice and assessment. It then describes the proposed design of a game based on this framework aiming to close the communication gap between able hearing people and people with a hearing impediment, by providing a tool that facilitates BSL learning targeting adult population. The paper concludes with the planning of a large scale study and directions for further development of this educational resource.

Information-centric networking (ICN) has long been advocating for radical changes to the Internet, but the upgrade challenges that this entails have hindered its adoption. To break this loop, the POINT project proposed a hybrid, IP-over-ICN, architecture: IP networks are preserved at the edge, connected to each other over an ICN core. This exploits the key benefits of ICN, enabling individual network operators to improve the performance of their IP-based services, without changing the rest of the Internet. This paper first provides an overview of POINT and outlines how it can improve upon IP in terms of performance and resilience. It then describes a trial of the POINT prototype in a production network, where real users operated actual IPbased applications. As part of the trial, we carried out experiments to evaluate the Quality of Experience (QoE) for video services offered via either HLS or IPTV, using either IP or POINT as a substrate. The results from the trial verify that the IP-over-ICN approach of POINT offers enhanced QoE to the users of these video services, compared to traditional IP, especially under exceptional network conditions.

Online learning platforms are integrated systems designed to provide students and teachers with information, tools and resources to facilitate and enhance the delivery and management of learning. In recent years platform designers have introduced gamification and multimodal interaction as ways to make online courses more engaging and immersive. Current Web-based platforms provide a limited degree of immersion in learning experiences, thereby diminishing potential learning impact. To improve immersion, it is necessary to stimulate some or all the human senses by engaging users in an environment that perceptually surrounds them and allows intuitive and rich interaction with other users and its content. Learning in these collaborative virtual environments (CVEs) can be aided by increasing motivation and engagement through the gamification of the educational task. This rich interaction that combines multimodal stimulation and gamification of the learning experience has the potential to draw students into the learning experience and improve learning outcomes. This paper presents the results of an experimental study designed to evaluate the impact of multimodal real-time interaction on user experience and learning of gamified educational tasks completed in a CVE. Secondary school teachers and students participated in the study. The multimodal CVE is an accurate reconstruction of the European Parliament in Brussels, developed using the REVERIE (Real and Virtual Engagement In Realistic Immersive Environment) framework. In the study, we compared the impact of the VR parliament to a non-multimodal control (an educational platform called Edu-Simulation) for the same educational tasks. Our results show that the multimodal CVE improves student learning performance and aspects of subjective experience when compared to the non-multimodal control. More specifically it resulted in a more positive effect on the ability of the students to generate ideas compared to a non-multimodal co

The use of serious games in education and their pedagogical benefit is being widely recognized. However, effective integration of serious games in education depends on addressing two big challenges: the successful incorporation of motivation and engagement that can lead to learning; and the highly specialised skills associated with customised development to meet the required pedagogical objectives. This paper presents the Westminster Serious Games Platform (wmin-SGP) an authoring tool that allows educators/domain experts without games design and development technical skills to create bespoke roleplay simulations in three dimensional scenes featuring fully embodied virtual humans capable of verbal and non-verbal interaction with users fit for specific educational objectives. The paper presents the wmin-SGP system architecture and it evaluates its effectiveness in fulfilling its purpose via the implementation of two roleplay simulations, one for Politics and one for Law. In addition, it presents the results of two types of evaluation that address how successfully the wmin-SGP combines usability principles and game core drives based on the Octalysis gamification framework that lead to motivating games experiences. The evaluation results shows that the wmin-SGP: provides an intuitive environment and tools that support users without advanced technical skills to create in real-time bespoke roleplay simulations in advanced graphical interfaces; satisfies most of the usability principles; and provides balanced simulations based on the Octalysis framework core drives. The paper concludes with a discussion of future extension of this real time authoring tool and directions for further development of the Octalysis framework to address learning.

Mobile Guides are mobile applications that provide players with local and location-based services (LBS), such as navigation assistance, where and when they need them most. Advances in mobile technologies in recent years have enabled the gamification of these applications, opening up new opportunities to transfer education and culture through game play. However, adding traditional game elements such as PBLs (points, badges, and leaderboards) alone cannot ensure that the intended learning outcomes will be met, as the player’s cognitive resources are shared between the application and the surrounding environment. This distribution of resources prevents players from easily immersing themselves into the educational scenario. Adding artificial conversational characters (ECAs) that simulate the social norms found in real-life human-to-human guide scenarios has the potential to address this problem and improve the player’s experience and learning of cultural narratives [1]. Although significant progress has been made towards creating game-like mobile guides with ECAs ([2], [3]), there is still a lack of a unified framework that enables researchers and practitioners to investigate the potential effects of such applications to players and how to approach the concepts of player experience, cognitive accessibility and usability in this context. This paper presents a theoretically-well supported research framework consisted of four key components: differences in players, different features of the gamified task, aspects of how the ECA looks, sound or behaves and different mobile environments. Furthermore, it provides based on this framework a working definition of what player experience, cognitive accessibility and usability are in the context of game-like mobile guide applications. Finally, a synthesis of the results of six empirical studies conducted within this research framework is discussed and a series of design guidelines for the effective gamification of mobile guide appl

REVERIE (REal and Virtual Engagement in Realistic Immersive Environments) is a research project with the aim to build a safe, collaborative, online environment which brings together realistic inter-personal communication and interaction. The REVERIE platform integrates cutting-edge technologies and tools, such as social networking services, spatial audio adaptation techniques, tools for creating personalized lookalike avatars, and artificial intelligence (A.I) detection features of the user’s affective state into two distinct use cases. The first shows how REVERIE can be used in educational environments with an emphasis on social networking and learning. The second aims to emulate the look and feel of real physical presence and interaction for entertainment and collaborative purposes. This paper presents an expert evaluation of the first use case by potential users of REVERIE (teachers and students). Finally, the potential of REVERIE for game-based learning is discussed followed by an overview of the actionable recommendations that emerged as a result of the expert review.