Search

This paper introduces the Quaternion Transition Generator (QTG), a new network architecture tailored to animation transition generation for virtual characters. The QTG is simpler than the current state of the art, making it lightweight and easier to implement. It uses approximately 80% fewer arithmetic operations compared to other transition networks. Additionally, this architecture is capable of generating visually accurate rotation-based animations transitions and results in a lower Mean Absolute Error than transition generation techniques that are commonly used for animation blending.

In this paper, we describe a user study in which pairs of users watch a video trailer and interact with each other, using two social Virtual Reality (sVR) systems, as well as in a face-to-face condition. The sVR systems are: Facebook Spaces, based on puppet-like customized avatars, and a video-based sVR system using photo-realistic virtual user representations. We collect subjective and objective data to analyze users’ Quality of Experience (QoE) and compare their interaction in VR to that observed during the real-life scenario. Our results show that the experience delivered by the video-based sVR system is comparable with real-life settings, while the puppet-based avatars limit the perceived quality of the interaction. Our protocol for QoE assessment is fully documented to allow replication in similar experiments.

This study explores the extent to which individuals embodied in Virtual Reality tend to self-attribute the movements of their avatar. More specifically, we tested subjects performing goal-directed movements and distorted the mapping between user and avatar movements by decreasing or increasing the amplitude of the avatar hand movement required to reach for a target, while maintaining the apparent amplitude – visual distance – fixed. In two experiments, we asked subjects to report whether the movement that they have seen matched the movement that they have performed, or asked them to classify whether a distortion was making the task easier or harder to complete. Our results show that subjects perform poorly in detecting discrepancies when the nature of the distortion is not made explicit and that subjects are biased to self-attributing distorted movements that make the task easier. These findings, in line with previous accounts on the sense of agency, demonstrate the flexibility of avatar embodiment and open new perspectives for the design of guided interactions in Virtual Reality.

We present a visualization tool for human motion analysis in augmented reality. Our tool builds upon our previous work on joint biomechanical modelling for kinematic analysis, based on optical motion capture and personalized anatomical reconstruction of joint structures from medical imaging. It provides healthcare professionals with the in situ visualization of joint movements, where bones are accurately rendered as a holographic overlay on the subject-like if the user has an 'X-ray vision'-and in real-time as the subject performs the movement. The tool also provides a recording mechanism for the examination and acquisition of movements and range of motion information. Recorded information can be for instance retrieved at a later moment to assess patient's progress in terms of kinematics during the rehabilitation phase. We also propose an intuitive non-sequential mean of navigating through recordings. It consists of pointing at movement trajectories for easy and intuitive retrieval of the meaningful portions of a movement. This tool allows for the post hoc replay and analysis of fast movements, such as from athletes movements in sports injury evaluation. Currently, hip and knee joints are supported.

We present a visualization tool for human motion analysis in augmented reality. Our tool builds upon our previous work on joint biomechanical modelling for kinematic analysis, based on optical motion capture and personalized anatomical reconstruction of joint structures from medical imaging. It provides healthcare professionals with the in situ visualization of joint movements, where bones are accurately rendered as a holographic overlay on the subject-like if the user has an 'X-ray vision'-and in real-Time as the subject performs the movement. Currently, hip and knee joints are supported.

We present the design of a handheld-based interface for collaborative manipulations of 3D objects in mobile augmented reality. Our approach combines touch gestures and device movements for fast and precise control of 7-DOF transformations. Moreover, the interface creates a shared medium where several users can interact through their point-of-view and simultaneously manipulate 3D virtual augmentations. We evaluated our collaborative solution in two parts. First, we assessed our interface in single user mode, comparing the user task performance in three conditions: Touch gestures, device movements and hybrid. Then, we conducted a study with 30 participants to understand and classify the strategies that arise while working in pairs, when partners are free to make their task organization. Furthermore, we investigated the effectiveness of simultaneous manipulations compared with the individual approach.

The relative location of human body parts often materializes the semantics of on-going actions, intentions and even emotions expressed, or performed, by a human being. However, traditional methods of performance animation fail to correctly and automatically map the semantics of performer postures involving self-body contacts onto characters with different sizes and proportions. Our method proposes an egocentric normalization of the body-part relative distances to preserve the consistency of self contacts for a large variety of human-like target characters. Egocentric coordinates are character independent and encode the whole posture space, i.e., it ensures the continuity of the motion with and without self-contacts. We can transfer classes of complex postures involving multiple interacting limb segments by preserving their spatial order without depending on temporal coherence. The mapping process exploits a low-cost constraint relaxation technique relying on analytic inverse kinematics; thus, we can achieve online performance animation. We demonstrate our approach on a variety of characters and compare it with the state of the art in online retargeting with a user study. Overall, our method performs better than the state of the art, especially when the proportions of the animated character deviate from those of the performer.

With the broad range of motion capture devices available on the market, it is now commonplace to directly control the limb movement of an avatar during immersion in a virtual environment. Here, we study how the subjective experience of embodying a full-body controlled avatar is influenced by motor alteration and self-contact mismatches. Self-contact is in particular a strong source of passive haptic feedback and we assume it to bring a clear benefit in terms of embodiment. For evaluating this hypothesis, we experimentally manipulate self-contacts and the virtual hand displacement relatively to the body. We introduce these body posture transformations to experimentally reproduce the imperfect or incorrect mapping between real and virtual bodies, with the goal of quantifying the limits of acceptance for distorted mapping on the reported body ownership and agency. We first describe how we exploit egocentric coordinate representations to perform a motion capture ensuring that real and virtual hands coincide whenever the real hand is in contact with the body. Then, we present a pilot study that focuses on quantifying our sensitivity to visuo-Tactile mismatches. The results are then used to design our main study with two factors, offset (for self-contact) and amplitude (for movement amplification). Our main result shows that subjects' embodiment remains important, even when an artificially amplified movement of the hand was performed, but provided that correct self-contacts are ensured.

In the context of the 3DUI Contest promoted by the IEEE VR 2018, we propose 3D interaction techniques that address three distinct tasks in a virtual environment setup: climbing a ladder, controlling a quadcopter in a first-person view flight, and building a tower by stacking a series of objects. The interaction techniques were developed so the player, our 3D-Athlete, has control over the events in each task, following metaphors that facilitate the use of the interface, and having status and spatial awareness supported by clear feedback cues. Thus, the player should be able to execute the tasks with precision and agility.

In this paper, we propose a new approach for the classification of reaching targets before movement onset, during visually-guided reaching in 3D space. Our approach combines the discriminant power of two-dimensional Electroencephalography (EEG) signals (i.e., EEG images) built from short epochs, with the feature extraction and classification capabilities of deep learning (DL) techniques, such as the Convolutional Neural Networks (CNN). In this work, reaching motions are performed into four directions: left, right, up and down. To allow more natural reaching movements, we explore the use of Virtual Reality (VR) to build an experimental setup that allows the subject to perform self-paced reaching in 3D space while standing. Our results reported an increase both in classification performance and early detection in the majority of our experiments. To our knowledge this is the first time that EEG images and CNN are combined for the classification of reaching targets before movement onset.

We introduce a novel procedure that extends the time feasibility for classification of early human actions. Its major characteristic is to use epoch training data from a wider time duration before action onset (i.e., within the intention period) instead of data from localized sliding windows. This is the case of time-specific and selected fixed classifiers. Our approach models human actions from EEG signals and leverages on amplitudes and power frequencies to construct fifteen groups of action vectors, which were subjected to a set of classifiers. Regarding early classification our approach did it earlier than both time-specific and selected fixed classifiers. Moreover, our results reported an increase in classification performance.

Interaction in augmented reality environments may be very complex, depending on the degrees of freedom (DOFs) required for the task. In this work we present a 3D user interface for collaborative manipulation of virtual objects in augmented reality (AR) environments. It maps position - acquired with a camera and fiducial markers - and touchscreen input of a handheld device into gestures to select, move, rotate and scale virtual objects. As these transformations require the control of multiple DOFs, collaboration is proposed as a solution to coordinate the modification of each and all the available DOFs. Users are free to decide their own manipulation roles. All virtual elements are displayed directly on the mobile device as an overlay of the camera capture, providing an individual point of view of the AR environment to each user.

We present a 3D user interface for collaborative manipulation of three-dimensional objects in virtual environments. It maps inertial sensors, touch screen and physical buttons of a mobile phone into well-known gestures to alter the position, rotation and scale of virtual objects. As these transformations require the control of multiple degrees of freedom (DOFs), collaboration is proposed as a solution to coordinate the modification of each and all the available DOFs. Users are free to decide their own manipulation roles. All virtual elements are displayed in a single shared screen, which is handy to aggregate multiple users in the same physical space.

Purpose Recent technology advances in Augmented Reality (AR) is creating an opportunity for a paradigm shift in the field of Computer Assisted Surgery (CAS). Despite having the potential of being a promising technology in clinical applications, its accuracy, robustness, and performances under specific environmental conditions have not been fully investigated and evaluated. In an attempt to address the limitations and uncertainties that arise from using computer vision, we propose a Hand-Eye (HE) based approach to determine an invariant mapping between the HoloLens headset (i.e., rigid body defined by a unique set of passive markers) and the OHMDs virtual camera. Methods In order to achieve the required precision for using the Microsoft HoloLens in a surgical application, an invariant mapping was estimated. First, a set of non-collinear retroreflective markers were rigidly attached to the Microsoft HoloLens headset. Consequently, the OHMDs's intrinsic parameters, as well as their radial and tangential lenses distortion coefficients, were determined by solving simultaneously a set of homogeneous system of equations by using nonlinear optimization techniques. The accuracy of the proposed HE based method has been evaluated by using a retro-projection error estimation method. Results Initially, a surgical frame has been anchored to the organ of interest. After loading the patient-specific dataset, a hologram of the organ of interest appears in front of the user's initial position. Correspondences between the hologram and the fiducial markers can then be established by using the tracked surgical pointer and then be mapped with respect to the surgical frame. After finishing this process, an orthonormal basis is created in the physical space, which has its counterpart in the image space and, therefore, a transformation can be determined to map the hologram with respect to the surgical frame. Conclusions OHMDs may be able to create an opportunity for a paradigm shift in the field of CAS, since all this detailed information could easily be displayed on virtual and interactive screens in operating rooms on the same time and in real time. A very important aspect of using such OHMDs is that it can be very intuitive, especially among inexperienced surgeon, since surgical instruments and maneuvers motions are represented with respect to the surgeon's visual system and, therefore, attention disruptions are more likely to be reduced in terms of the surgeon's gaze, considering that the surgeon would not need to be searching for a conventional display located at static location in the operating room.

While medical images are fundamental in the surgery planning procedure, the process of analysis of such images slice-by-slice is still tedious and inefficient. In this work we introduce a system for exploration of the internal anatomy structures directly on the surface of the real body using a mobile display device as a window to the interior of the patient's body. The method is based on volume visualization of standard computed tomography datasets and augmented reality for interactive visualization of the generated volume. It supports our liver surgery planner method in the analysis of the segmented liver and in the color classification of the vessels. We present a set of experiments showing the system's ability to operate on mobile devices. Quantitative performance results are detailed, and applications in teaching anatomy and doctor-patient communication are discussed.

Part 1: Long and Short Papers (Continued); International audience; We present the disambiguation canvas, a technique developed for easy, accurate and fast selection of small objects and objects inside cluttered virtual environments. Disambiguation canvas rely on selection by progressive refinement, it uses a mobile device and consists of two steps. During the first, the user defines a subset of objects by means of the orientation sensors of the device and a volume casting pointing technique. The subsequent step consists of the disambiguation of the desired target among the previously defined subset of objects, and is accomplished using the mobile device touchscreen. By relying on the touchscreen for the last step, the user can disambiguate among hundreds of objects at once. User tests show that our technique performs faster than ray-casting for targets with approximately 0.53 degrees of angular size, and is also much more accurate for all the tested target sizes.

Hepatectomies are resections in which segments of the liver are extracted. While medical images are fundamental in the surgery planning procedure, the process of analysis of such images slice-by-slice is still tedious and inefficient. In this work we propose a strategy to efficiently and semi-automatically segment and classify patient-specific liver models in 3D through a mobile display device. The method is based on volume visualization of standard CT datasets and allows accurate estimation of functional remaining liver volume. Experiments showing effectiveness of the method are presented, and quantitative and qualitative results are discussed.

We present levels of precision (LOP) cursor, a metaphor for high precision pointing and simultaneous cursor controlling using commodity mobile devices. The LOP-cursor uses a two levels of precision representation that can be combined to access low and high resolution of input. It provides a constrained area of high resolution input and a broader area of lower input resolution, offering the possibility of working with a two legs cursor using only one hand. LOP-cursor is designed for interaction with large high resolution displays, e.g. display walls, and distributed screens/computers scenarios. This paper presents the design of the cursor, the implementation of a prototype, and user evaluation experiments showing that our method allows both, the acquisition of small targets, and fast interaction while using simultaneous cursors in a comfortable manner. Targets smaller than 0.3 cm can be selected by users at distances over 1.5 m from the screen with minimum effort.

This paper presents a 3D user interface to solve a three-dimensional wooden blocks puzzle. Such interface aims at reproducing the real scenario of puzzle solving using involving devices and techniques for interaction and visualization which include a mobile device, haptics and enhanced stereo vision. The paper describes our interaction approach, the system implementation and user experiments.

Object manipulation in 3D virtual environments demands a combined coordination of rotations, translations and scales, as well as the camera control to change the user's viewpoint. Then, for many manipulation tasks, it would be advantageous to share the interaction complexity among team members. In this paper we propose a novel 3D manipulation interface based on a collaborative action coordination approach. Our technique explores a smartphone - the touchscreen and inertial sensors - as input interface, enabling several users to collab-oratively manipulate the same virtual object with their own devices. We first assessed our interface design on a docking and an obstacle crossing tasks with teams of two users. Then, we conducted a study with 60 users to understand the influence of group size in collaborative 3D manipulation. We evaluated teams in combinations of one, two, three and four participants. Experimental results show that teamwork increases accuracy when compared with a single user. The accuracy increase is correlated with the number of individuals in the team and their work division strategy.

In this paper we evaluate the use of non-planar projections as a means to increase the Field of View (FoV) in embodied Virtual Reality (VR). Our main goal is to bring the virtual body into the user's FoV and to understand how this affects the virtual body/environment relation and quality of interaction. Subjects wore a Head Mounted Display (HMD) and were instructed to perform a selection and docking task while using either Perspective (≈ 106° vertical FoV), Hammer or Equirectangular (≈ 180° vertical FoV for both) projection. The increased FoV allowed for a shorter search time as well as less head movements. However, quality of interaction was generally inferior, requiring more time to dock, increasing docking error and producing more body/environment collisions. We also assessed cybersickness and the sense of embodiment toward the virtual body through questionnaires, for which the difference between projections seemed to be less pronounced.

This project investigates the impact of a virtual zero gravity experience on the human gravity model. In the planned experiment, subjects are immersed with HMD and full body motion capture in a virtual world exhibiting either normal gravity or the apparent absence of gravity (i.e. body and objects floating in space). The study evaluates changes in the subjects' gravity model by observing changes on motor planning of actions dependent on gravity. Our goal is to demonstrate that a virtual reality exposure can induce some modifications to the humans internal gravity model, analogous to those resulting from real exposure (e.g. parabolic flights), even if users remain under normal gravity condition in reality.