Search

The light field (LF) imaging technique can capture 3D scene information in 4D by recording both 2D intensity and 2D direction of incoming light rays. Due to this capability, LF has shown a great interest in virtual reality (VR) and augmented reality (AR) for enhanced immersion, improved depth perception and reconstruction of realistic 3D environments. This paper presents a comprehensive review of LF imaging technology and other approaches used for VR content creation. The applications of LF technology beyond VR and AR are also discussed. The challenges and limitations of other approaches for VR content creation are examined. State‐of‐the‐art research has focused on how VR experiences benefit from LF technology and identified the challenges to creating comfortable, immersive and realistic VR content such as (1) image size and resolution, (2) processing speed, (3) precise calibration and (4) depth reconstruction. Recommendations that can be considered for creating immersive VR content are provided to enhance user experience. These recommendations aim to contribute to developing more comfortable and realistic VR content, extending the potential applications of LF imaging technology in diverse fields. [ABSTRACT FROM AUTHOR]

This study compares the influence of a gamified and a non-gamified virtual reality (VR) environment on wheelchair skills training. In specific, the study explores the integration of gamification elements and their influence on wheelchair driving performance in VR-based training. Twenty-two non-disabled participants volunteered for the study, of whom eleven undertook the gamified VR training, and eleven engaged in the non-gamified VR training. To measure the efficacy of the VR-based wheelchair skills training, we captured the heart rate (HR), number of joystick movements, completion time, and number of collisions. In addition, an adapted version of the Wheelchair Skills Training Program Questionnaire (WSTP-Q), the Igroup Presence Questionnaire (IPQ), and the Simulator Sickness Questionnaire (SSQ) questionnaires were administered after the VR training. The results showed no differences in wheelchair driving performance, the level of involvement, or the ratings of presence between the two environments. In contrast, the perceived cybersickness was statistically higher for the group of participants who trained in the non-gamified VR environment. Remarkably, heightened cybersickness symptoms aligned with increased HR, suggesting physiological connections. As such, while direct gamification effects on the efficacy of VR-based wheelchair skills training were not statistically significant, its potential to amplify user engagement and reduce cybersickness is evident. [ABSTRACT FROM AUTHOR]

Multisensory experiences have been increasingly applied in Human-Computer Interaction (HCI). In recent years, it is commonplace to notice the development of haptic, olfactory, and even gustatory displays to create more immersive experiences. Companies are proposing new additions to the multisensory world and are unveiling new products that promise to offer amazing experiences exploiting mulsemedia.multiple sensorial media. where users can perceive odors, tastes, and the sensation of wind blowing against their face. Whilst researchers, practitioners and users alike are faced with a wide range of such new devices, relatively little work has been undertaken to summarize efforts and initiatives in this area. The current article addresses this shortcoming in two ways: first, by presenting a survey of devices targeting senses beyond that of sight and hearing and, second, by describing an approach to guide newcomers and experienced practitioners alike to build their own mulsemedia environment, both in a desktop setting and in an immersive 360. environment. [ABSTRACT FROM AUTHOR]

Sensory studies emerged as a significant influence upon Human Computer Interaction and traditional multimedia. Mulsemedia is an area that extends multimedia addressing issues of multisensorial response through the combination of at least three media, typically a non-traditional media with traditional audio-visual content. In this paper, we explore the concepts of Quality of Experience and crossmodal correspondences through a case study of different types of mulsemedia setups. The content is designed following principles of crossmodal correspondence between different sensory dimensions and delivered through olfactory, auditory and vibrotactile displays. The Quality of Experience is evaluated through both subjective (questionnaire) and objective means (eye gaze and heart rate). Results show that the auditory experience has an influence on the olfactory sensorial responses and lessens the perception of lingering odor. Heat maps of the eye gazes suggest that the crossmodality between olfactory and visual content leads to an increased visual attention on the factors of the employed crossmodal correspondence (e.g., color, brightness, shape). [ABSTRACT FROM AUTHOR]

One of the main challenges in current multimedia networking environments is to find solutions to help accommodate the next generation of mobile application classes with stringent Quality of Service (QoS) requirements while enabling the Quality of Experience (QoE) provisioning for users. One such application class, featured in this article, is 360° mulsemedia—multiple sensorial media—which enriches 360° video by adding sensory effects that stimulate human senses beyond those of sight and hearing, such as the tactile and olfactory ones. In this article, we present a conceptual framework for 360° mulsemedia delivery and a 360° mulsemedia-based prototype that enables users to experience 360° mulsemedia content. User evaluations revealed that higher video resolutions do not necessarily lead to the highest QoE levels in our experimental setup. Therefore, bandwidth savings can be leveraged with no detrimental impact on QoE. [ABSTRACT FROM AUTHOR]

The advent and increased use of new technologies, such as innovative mulsemedia and multi-modal content distribution mechanisms, have brought new challenges and diverse opportunities for technology enhanced learning (TEL). NEWTON is a Horizon 2020 European project that revolutionizes the educational process through innovative TEL methodologies and tools, integrated in a pan-European STEM-related learning network platform. This article focuses on one of these novel TEL methodologies (i.e., mulsemedia) and presents how NEWTON enables mulsemedia- enhanced teaching and learning of STEM subjects, with a particular focus on telecommunication and networking related modules. The article also discusses the very promising results of NEWTON case studies carried out with engineering students across two different universities in Spain and Ireland, respectively. The case studies focused on analyzing the impact on the learning process of the mulsemedia-enhanced teaching in the context of telecommunication and networking modules. The main conclusion of the article is that mulsemedia-enhanced education significantly increases students' learning experience and improves their knowledge gain. [ABSTRACT FROM AUTHOR]

Mulsemedia--multiple sensorial media--makes possible the inclusion of layered sensory stimulation and interaction through multiple sensory channels. The recent upsurge in technology and wearables provides mulsemedia researchers a vehicle for potentially boundless choice. However, in order to build systems that integrate various senses, there are still some issues that need to be addressed. This review deals with mulsemedia topics that remain insufficiently explored by previous work, with a focus on the multi-multi (multiple media-multiple senses) perspective, where multiple types of media engage multiple senses. Moreover, it addresses the evolution of previously identified challenges in this area and formulates new exploration directions. [ABSTRACT FROM AUTHOR]

Different senses provide us with information of various levels of precision and enable us to construct a more precise representation of the world. Rich multisensory simulations are thus beneficial for comprehension, memory reinforcement, or retention of information. Crossmodal mappings refer to the systematic associations often made between different sensory modalities (e.g., high pitch is matched with angular shapes) and govern multisensory processing. A great deal of research effort has been put into exploring cross-modal correspondences in the field of cognitive science. However, the possibilities they open in the digital world have been relatively unexplored. Multiple sensorial media (mulsemedia) provides a highly immersive experience to the users and enhances their Quality of Experience (QoE) in the digital world. Thus, we consider that studying the plasticity and the effects of cross-modal correspondences in a mulsemedia setup can bring interesting insights about improving the human computer dialogue and experience. In our experiments, we exposed users to videos with certain visual dimensions (brightness, color, and shape), and we investigated whether the pairing with a cross-modal matching sound (high and low pitch) and the corresponding autogenerated vibrotactile effects (produced by a haptic vest) lead to an enhanced QoE. For this, we captured the eye gaze and the heart rate of users while experiencing mulsemedia, and we asked them to fill in a set of questions targeting their enjoyment and perception at the end of the experiment. Results showed differences in eye-gaze patterns and heart rate between the experimental and the control group, indicating changes in participants' engagement when videos were accompanied by matching cross-modal sounds (this effect was the strongest for the video displaying angular shapes and high-pitch audio) and transitively generated crossmodal vibrotactile effects. [ABSTRACT FROM AUTHOR]

Accurate distance perception and natural interactions are mandatory conditions when training precision aiming tasks in VR. However, many factors specific to virtual environments (VEs) lead to differences in the way users execute a motor task in VR versus the real world. To investigate these differences, the authors performed a study on basketball beginners' free-throw performance in VEs under different visual conditions. Although the success rate is not statistically different, some adaptations occurred in the way the users performed the task, depending on the visual conditions. In the third-person perspective visual condition, the release parameters indicate that the users more accurately estimated distance to target. Adding visual guidance information (gradual depth information showing the ideal ball trajectory) also led to more natural motor behavior. The final aim of this study was to develop a reliable basketball free-throw training system in VEs, so the authors compared beginners' performances in VR with experts' models of performance. Their results show that most of the performance variables tended to evolve closer to the experts' performance during the training in the VE. [ABSTRACT FROM PUBLISHER]

AbstractOne factor leading to compromised Quality of Life (QoL) for people with dementia in long-term care settings is the significant barriers they face in accessing experiences beyond their physical premises. Although, in recent years, research within HCI has investigated the use of Virtual Reality (VR) in providing enriched experiences for people with dementia, few studies have looked into the practical issues relating to integrating and deploying VR into care settings and how in-depth understanding of such issues can inspire VR design for dementia care. Considering the perspective of key stakeholders, we demonstrate VR’s potential to create a rich conversation space with family members, preserve the identity and personhood of people with dementia through VR, and consider organisational issues of VR deployment in care settings. As such, this paper contributes to the HCI research community with future design directions to enhance the deployment and adoption of VR in dementia care settings. [ABSTRACT FROM AUTHOR]

AbstractThe development of AI to function as communicators (i.e., conversational agents), has opened the opportunity to rethink AI’s place within people’s social worlds, and the process of sense-making between humans and machines, especially for people with autism who may stand to benefit from such interactions. The current study aims to explore the interactions of six autistic and six non-autistic adults with a conversational virtual human (CVH/conversational agent/chatbot) over 1–4 weeks. Using semi-structured interviews, conversational chatlogs and post-study online questionnaires, we present findings related to human-chatbot interaction, chatbot humanization/dehumanization and chatbot’s autistic/non-autistic traits through thematic analysis. Findings suggest that although autistic users are willing to converse with the chatbot, there are no indications of relationship development with the chatbot. Our analysis also highlighted autistic users’ expectations of empathy from the chatbot. In the case of the non-autistic users, they tried to stretch the conversational agent’s abilities by continuously testing the AI conversational/cognitive skills. Moreover, non-autistic users were content with Kuki’s basic conversational skills, while on the contrary, autistic participants expected more in-depth conversations, as they trusted Kuki more. The findings offer insights to a new human-chatbot interaction model specifically for users with autism with a view to supporting them via companionship and social connectedness. [ABSTRACT FROM AUTHOR]