Search

3D objects are important components in Mixed Reality (MR) environments as they allow users to inspect and interact with them in a six degrees of freedom (6DoF) system. Point clouds (PCs) and meshes are two common 3D object representations that can be compressed to reduce the delivered data at the cost of quality degradation. In addition, as the end users can move around in 6DoF applications, the viewing distance can vary. Quality assessment is necessary to evaluate the impact of the compressed representation and viewing distance on the Quality of Experience (QoE) of end users. This paper presents a demonstrator for subjective quality assessment of dynamic PC and mesh objects under different conditions in MR environments. Our platform allows conducting subjective tests to evaluate various QoE influence factors, including encoding parameters, quality switching, viewing distance, and content characteristics, with configurable settings for these factors.

Dynamic point cloud delivery can provide the required interactivity and realism to six degrees of freedom (6DoF) interactive applications. However, dynamic point cloud rendering imposes stringent requirements (e.g., frames per second (FPS) and quality) that current hardware cannot handle. A possible solution is to convert point cloud into meshes before rendering on the head-mounted display (HMD). However, this conversion can induce degradation in quality perception such as a change in depth, level of detail, or presence of artifacts. This paper, as one of the first, presents an extensive subjective study of the effects of converting point cloud to meshes with different quality representations. In addition, we provide a novel in-session content rating methodology, providing a more accurate assessment as well as avoiding post-study bias. Our study shows that both compression level and observation distance have their influence on subjective perception. However, the degree of influence is heavily entangled with the content and geometry at hand. Furthermore, we also noticed that while end users are clearly aware of quality switches, the influence on their quality perception is limited. As a result, this has the potential to open up possibilities in bringing the adaptive video streaming paradigm to the 6DoF environment.

Point Cloud (PC) streaming has recently attracted research attention as it has the potential to provide six degrees of freedom (6DoF), which is essential for truly immersive media. PCs require high-bandwidth connections, and adaptive streaming is a promising solution to cope with fluctuating bandwidth conditions. Thus, understanding the impact of different factors in adaptive streaming on the Quality of Experience (QoE) becomes fundamental. Mixed Reality (MR) is a novel technology and has recently become popular. However, quality evaluations of PCs in MR environments are still limited to static images. In this paper, we perform a subjective study on four impact factors on the QoE of PC video sequences in MR conditions, including quality switches, viewing distance, and content characteristics. The experimental results show that these factors significantly impact QoE. The QoE decreases if the sequence switches to lower quality and/or is viewed at a shorter distance, and vice versa. Additionally, the end user might not distinguish the quality differences between two quality levels at a specific viewing distance. Regarding content characteristics, objects with lower contrast seem to provide better quality scores.

In recent years, Virtual Reality (VR) has gained attention as a tool for a plethora of applications such as first-aid, firefighting and in the automotive industry. End-user immersion is a key factor in these applications to make the experience representative for its real-life counterpart. By enhancing the traditional audiovisual cues with additional sensory inputs in terms of haptic vibro-tactile and kinesthetic feedback, this immersion can be improved. But are current haptic implementations sufficient to provide the required added value? And how do they compare to other types of VR interaction? In this paper, we present a multi-modal VR training framework able to provide subjective and objective comparisons among three different interaction options: (i) haptic gloves, (ii) traditional VR controllers, and (iii) non-haptic handtracking. We performed a user test where the different interactivity flavours were compared in terms of their influence on both subjective perception and objective performance of the end-user by means of three VR training scenarios. The subjective results show an aversion towards non-haptic handtracking for constrained, cognitively light tasks while a preference towards controllers exist for more cognitively heavy multi-tasking. This is however not reflected in objective results, where differences between interaction methods are far less pronounced.

Accurate and fast estimations or predictions of the (near) future location of the users of head-mounted devices within the virtual omnidirectional environment open a plethora of opportunities in application domains such as interactive immersive gaming and tele-surgery. Therefore, the past years have seen growing attention to models for viewport prediction in 360֯ environments. Among the approaches, content-agnostic, trajectory-based methods have the potential to provide very fast solutions, as they do not require complex analysis of the videos to provide a prediction. However, accurate trajectory-based viewport prediction is rather difficult due to the intrinsic variability in user behaviour. Furthermore, even when making use of machine learning, current approaches tend to be brute-force and heavily tailored to specific datasets with little comparison to existing benchmarks or publicly available studies. This article presents a generic, content-agnostic viewport prediction method consisting of a window-based approach combined with a preprocessing system to classify behavioural patterns in terms of user clustering and trajectory correlation. Moreover, as the state of the art does not provide a comparative analysis of different approaches, this work contributes to this. Based on the obtained results, a combined prediction model is proposed and evaluated. Our method shows a 36.8% to 53.9% improvement when compared to the static prediction baseline for a prediction horizon of 8 seconds. In addition, a 11.5% to 24.0% improvement to a brute-force machine learning prediction approach is obtained. As such, this work contributes towards the creation of more generic and structured solutions for content-agnostic viewport prediction in terms of data representation, preprocessing and modelling.

Multimodal, immersive systems are the latest development within the field of multimedia. These systems emulate the senses by means of omnidirectional visuals, 360° sound, motion tracking and touch simulation to create a feeling of presence in the virtual environment. They have the potential to substitute physical interactions in application domains such as training (Industry 4.0) or e-health (tele-surgery). However, the COVID-19 pandemic has shown that they are not ready, as they still have room for improvement in terms of network streaming quality, usability and the users’ feeling of presence. In addition, these systems can induce feelings of dizziness, nausea etc. (i.e. cybersickness). These factors therefore have an important impact on the user’s total immersion. In this work, we therefore propose that immersiveness can be devised from measuring four aspects, namely: presence (i.e. the feeling of being "in" the environment), cybersickness, network related Quality-of-Experience (QoE) and the usability of the application. Therefore, a two-dimensional user-centric approach on the assessment and modelling of immersiveness is proposed. These dimensions include (i) subjective and objective assessment of presence, cybersickness, usability and QoE and (ii) real-time modelling of immersiveness. Furthermore, a proof-of-concept is envisioned including two use cases. As such, we believe that this position paper will significantly advance the state of the art on immersive systems and multimedia in general.

The advent of softwarized networks has enabled the deployment of service chains of virtual network components on computational resources from the cloud up to the edge. The next generation of use cases (e.g. Virtual Reality (VR) content delivery services) puts even more stringent requirements on the infrastructure, calling for considerable advancements towards fully cloud-native architectures. This paper identifies important challenges for next-generation architectures to support low latency applications throughout their execution life cycle. A flexible architecture named SRFog is presented for the support of VR content delivery in next-generation networks. The approach combines Fog Computing (FC) concepts, an extension of cloud computing, with Segment Routing (SR), which leverages the source routing paradigm. Service Function Chaining (SFC) is also discussed as a major functionality for the proper orchestration of emerging use cases. Early implementations show promising results when deploying container-based VR chains in a flexible FC architecture.