Search

Contextual illusions, such as the Ebbinghaus Illusion, can be potentially used to improve or hinder reach-to-grasp interaction in a virtual environment by affecting the perception of object size and the action. However, the illusion effect has only been evaluated using 2D objects like discs or annuluses, and limited research has been conducted in a virtual environment. Moreover, it remains unknown how the sudden, or dynamic, change of surrounding features will impact the perception and then the action towards the object. In this paper, we conducted a series of experiments to evaluate the effects of 3D Ebbinghaus illusion with dynamic surrounding features on the task of reaching to grasp a 3D object in an immersive virtual environment. An innovative 3D perceptual judgment task revealed that the static 3D illusion affected the perceived size of the 3D object. Then, we experimentally manipulated the visual gain and loss of the 3D contextual inducers, the participant's virtual hand, and the entire 3D contextual object. Results revealed that the depth error (error in depth of reaching action) was influenced by a dynamic change in the size of the inducers, the fine adjustment of grasp was dependent on the visual presence of the virtual hand and vision of 3D contextual object was required for the reaching and grasping movements. These results will benefit the understanding of reach-to-grasp interactions in immersive virtual environments and can improve interaction design.

Kinetic Skin is a thin, adhesive patch that employs resistive sensing via a meandering carbon ink trace, where the circuit designs are printed on temporary tattoo material, similar to ones worn by children for decorative play. Each tattoo sensor is worn on your body across a joint (ie. finger, wrist, and elbow joints). In this paper, we present implementation details of prototyping Kinetic Skin worn on the finger and wrist joints for relative tracking to control 6-DoF manipulation. We demonstrate translation, rotation, and scale of virtual objects, including camera navigation. In this paper, we describe four interaction techniques that map finger joint input to 6-DOF control. The potential of this technology could be used for long-term light weight tracking of body movements to inform rehabilitation; for dance, music, or other performing arts; for gaming; and for other 3D User Interaction in Virtual and Augmented Reality Applications.

Low cost sensors and materials are increasingly of interest to designers for developing new ways to gather 3-Dimensional input. Silicone is a low cost material with capabilities of a variety of forms and sizes, thereby facilitating flexible construction. Given these properties, users can construct unique input solutions for a variety of applications. However, aside from other existing methods of measuring volume deformation, molded silicone (without added components inside the silicone and without added external cameras) for volumetric input has not been largely explored. In this paper we present an evaluation that investigated the parameters of silicone as volumetric input. The silicone volume has no added components inside making it easy to construct and use, however some external but small, flexible, and portable low-cost components are used for deformation measurement. We present the 3-dimensional input results as a function of the physical pressure on the silicone by the volume of silicone. Researchers can use these input metrics to design a silicone-based device with desired size and thickness to achieve the desired sensitivity and resolution of input for their application.

As Virtual and Augmented Reality tasks become increasingly complex, there is a need to organize virtual information within the virtual environment better. Users may need to organize their virtual objects or tools by hiding them until usage is needed or carry with them to another location. For these scenarios, there may be benefits to associate or store virtual objects in relation to one’s avatar (Egocentric) or in relation to the other virtual objects in the environment (Exocentric). This research presents these techniques and results of a user study comparing the effects of Egocentric and Exocentric Virtual Object Storage Techniques on cognition in a virtual environment.

As tasks in Virtual and Augmented Reality applications become increasingly complex, there is a need to better organize information or virtual objects within the virtual environment. For example, 3D Widget or menus may need to hide information from the user until it is needed. Users may need to organize their own virtual objects or tools by storing them until usage is needed or carry with them to another location. For these scenarios and others, there may be benefits to associating or storing virtual objects in relation to one’s own avatar (Egocentric) or in relation to the other virtual objects in the environment (Exocentric). This paper presents the results of a user study comparing the effects of Egocentric and Exocentric Virtual Object Storage Techniques on cognition in a virtual environment. Participants performed a task that enabled them to store virtual objects using different Virtual Object Storage Techniques. As participants completed their tasks, we measured task and cognition performance. We found that the egocentric technique improves task performance, increases accuracy, and reduces cognitive workload during memory task for virtual object storage in comparison to the exocentric technique in a virtual environment.

This paper presents a novel interaction technique to combine multiple inputs and output modalities with a 3D printed open disc which is tracked and augmented with a virtual information. Props have been used in virtual and augmented environments to provide tangible associations to virtual elements. When used in a larger immersive environment or where a user is walking around, users may not have the capability to set these objects/props down or are limited to the number of hands, so users are reduced to the capability of using only two props or controllers. In our technique, users can instead use their body to store and organize the virtual objects associated with tangible props by physically hanging them on or attaching them to the body. This type of interaction is well-suited for both immersive visualizations and immersive virtual environments. In this paper, we present the results of an experimental evaluation of our Body Prop Interaction technique for each type of immersive environment and corresponding task type, demonstrating more effective usage in each scenario than a joystick or gesture-based interaction.

In this paper, we present our design of a high school summer course which uses our Visual Design Problem-based Learning Pedagogy using Virtual Environments as a strategy to teach computer science. Students solved visual design problems by creating 3D sculptures in an online virtual environment. These creations were further explored and refined in immersive display systems fostering embodied learning and remote peer presence and support. To achieve the desired design, students use programming and computing concepts, such as loops, to solve those visual design centered problems, i.e. solving for composition, positive/negative space, balance, as opposed to computational problems first, i.e. create a loop, a fractal, randomized lines, etc. We present results from a study conducted on three high school summer courses. We compared the use of our Visual Design Problem-based teaching strategy (students wrote code to solve challenges based on art and design principles) to a traditional strategy (students wrote code to demonstrate comprehension of computer science concepts). Our results showed that test scores were higher for students in our Visual Design Problem-based courses. This work may have a positive impact on computer science education by increasing engagement, knowledge acquisition, and self-directed learning.

Prior work has shown that visual cues are more dominant than tactile cues for static objects. When texture and material properties are added, the reverse is true. However, it is not well understood which cues play a role in perceiving these textures when objects are moving. In this paper, we present an evaluation investigating speed and angular direction mismatch between visual and tactile information while objects are moving. Our results show that when objects are stationary or moving slowly, the sense of touch dominates visual cues; but as the speed increases, visual cues dominate tactile cues. When tactile and visual cues were not matching for moving virtual objects, visual cues were more dominant when the virtual objects moved at angles less than 30 degrees different than the tactile feedback in either direction. Therefore, potentially a 1D directional haptic device with physical texture substitutions may be able to convey sufficient realism when touching virtual objects that are moving faster than 4-8cm/s and with a difference in angular direction of 30 degrees or less. In this paper, we also present the limitations of this work and outline a research agenda for future investigation of visual and tactile cue dominance for perceiving textures of virtual objects moving in a virtual environment.

In the real world, prior research in the field of perception and action has shown that individuals visually perceive objects differently based on their actual performance of an action on those objects, especially for sporting activities. For example, a golfer who performs well on putting a ball into the hole, will perceive that ball as larger (easier to hit) and the hole as larger (easier to put the ball in). We asked the following research question, can manipulation of visual perception of objects influence actual performance in the real world? Virtual objects are easily manipulated in Virtual Reality environments, therefore we investigated the use of Virtual Reality training where the properties of objects, such as size, were manipulated to influence perception on a golf putting task. In this paper, we present the results of our experimental user study. Putting performance increased after virtual reality training exposure when virtual objects were larger (perceived as easier to hit) and decreased when virtual objects were smaller (more difficult to hit). Our research has the potential to broaden the study of how virtual reality training can be used to improve sports training in a unique way.

Real-time data collection in immersive virtual reality can be difficult due to the visual obstruction to the real-world. Furthermore, subjective data collection is conducted post-task, reducing the information of detailed user responses during a task therefore potentially missing critical events, subtle difference, or issues not obvious to a user. In this paper we present, KnobCollector, a custom device controller for dynamic real-time collection of subjective user data during an experimental. We conducted a user study with the KnobCollector to demonstrate the feasibility of use during a virtual reality task. Our results provide more rich data sets of user feedback during the virtual reality experience and could be used for future user studies in virtual reality environments.

This paper presents the details of a novel selection technique, Physically-Based Volumetric Selection, to combat challenges in selecting volumes of data using immersive display spaces. We present results on a physical experiment that we used to inform design and discuss the details of our proposed technique. We also present the results of a virtual experiment to evaluate feasibility, utility, and usability. A description of user strategies and design is included. Our results will have implications for applications that use alternative rendering techniques as well as volume selection to assist with selection tasks in a physically-based manner. Our technique can be used in combination with other selection and manipulation techniques to improve the overall desired performance of volumetric selection.

Immersive Visualizations of 3-dimensional scientific data pose unique interaction challenges. One challenge is to be able to interact with the visual volumetric representations of data using direct manipulation in a way that facilitates exploration and discovery, yet maintains data relationships. In this paper, we present Body-Prop Interaction, a novel tangible multimodal interface for immersive visualizations. Our interaction technique combines multiple input and output modalities with 3D printed open discs that are tracked and augmented with virtual information. To demonstrate our technique, we implemented interactive tasks for a volume visualization of graphite billet data. While demonstrated for this type of data visualization, our novel interaction technique has the potential to be used to interact with other types of augmented and virtual reality.

In this poster, we present a novel 3-dimensional (3D) interaction technique, Altered Muscle Mapping (AMM), to re-map muscle movements of hands/arms to fingers/wrists. We implemented an initial design of AMM as a 3-Dimensional (3D) selection technique, where finger movements translate a virtual cursor (in 3-degrees-of-freedom) for selection. Direct Manipulation performance benefits may be preserved yet reduce physical fatigue. We designed an initial set of mapping variations. Our results from an initial pilot study provide initial performance insights of mapping configurations. AMM has potential for direct hand interaction in virtual and augmented reality and for users with a limited range of motion.

Collaboration among research scientists across multiple types of visualizations and platforms is useful to enhance scientific workflow and lead to unique analysis and discovery. However, current analytic tools and visualization infrastructure lack sufficient capabilities to fully support collaboration across multiple types of visualizations, display/interactive systems, and geographically distributed researchers. We have combined, adapted, and enhanced several emerging immersive and visualization technologies into a novel collaboration system that will provide scientists with the ability to connect with other scientists to work together across multiple visualization platforms (i.e. stereoscopic versus monoscopic), multiple datasets (i.e. 3-Dimensional versus 2-Dimensional data), and multiple visualization techniques (i.e. volumetric rendering versus 2D plots). We have demonstrated several use cases of this system in materials science, manufacturing, planning, and others. In one such use case, our collaboration system imports material science data (i.e., graphite billet) and enable multiple scientists to analyze and explore the density change of graphite across immersive and non-immersive systems, which will help to understand the potential structural problem in it. We recruited scientists that work with the datasets we demonstrate in three use case scenarios and conducted an experimental user study to evaluate our novel collaboration system on scientific visualization workflow effectiveness. In this paper, we present the results on task completion time, task performance, user experience, and feedback among multiple and feedback among multiple geographically distributed collaborators using different multiple-platform for collaboration.

There still lacks a collaborative framework that integrates existing and new virtual reality systems supporting remote collaboration for data visualization. The collaborative framework currently available might limit the collaborators for collaboration because it is not developed to support interaction with diverse 3D applications while using heterogeneous virtual reality (VR) systems remotely in a synchronized way. We will present our collaborative framework that supports remote collaboration across heterogeneous VR systems to interact with multiple 3D applications. This system is independent of any VR systems and can be used to interact collaboratively in real-time on a shared task. Additionally, we conducted a pilot study to gain perspective on the usability of the system for collaborative work across heterogeneous VR systems. We will present the results on the usability of the system, and the results obtained from the users are motivating and encouraging. Our system has the potential to increase task effectiveness and workflow because it enables shared and real-time interaction while remotely collaborating with diverse 3D applications across heterogeneous VR systems.

There are benefits to visualizing and exploring scientific data in immersive display systems. Interactive data analytics further exploit these benefits. Direct interaction techniques enable scientists to leverage their visual-cognitive skills for identifying and selecting visually rendered data. However, fatigue, data density, rendering, and visual encoding issues may result in reduced usage of immersive scientific visualizations. In this paper, we present algorithmic details of a novel volume selection interaction, implementation details, and discuss use case scenario. Our technique seeks to improve usage of immersive visualizations for scientific discovery and insight.

Data visualization and interaction is an important part of understanding and analyzing complex data. Immersive display systems provide benefits to exploration. In this paper, we present a user study to analyze co-located tasks and behaviors during a collaboration task. Results from this study helped us to identify patterns in co-located collaborative interaction to be able to better design remote collaborative environments. We discuss the challenges while interacting with data in an immersive data visualization environment and our design of a remote interactive collaboration framework for analysis and workflow of data visualizations. The goal of this framework is to preserve the benefits of physically co-located collaboration actions and add the benefits of virtual components that do not conform to real world restrictions.

3D applications appear in every corner of life in the current technology era. There is a need for an ubiquitous 3D input device that works with many different platforms, from head-mounted displays (HMDs) to mobile touch devices, 3DTVs, and even the Cave Automatic Virtual Environments. We present 3DTouch, a novel wearable 3D input device worn on the fingertip for 3D manipulation tasks. 3DTouch is designed to fill the missing gap of a 3D input device that is self-contained, mobile, and universally works across various 3D platforms. This paper presents a low-cost solution to designing and implementing such a device. Our approach relies on a relative positioning technique using an optical laser sensor and a 9-DOF inertial measurement unit. The device employs touch input for the benefits of passive haptic feedback, and movement stability. On the other hand, with touch interaction, 3DTouch is conceptually less fatiguing to use over many hours than 3D spatial input devices. We propose a set of 3D interaction techniques including selection, translation, and rotation using 3DTouch. An evaluation also demonstrates the device's tracking accuracy of 1.10 mm and 2.33 degrees for subtle touch interaction in 3D space. We envision that modular solutions like 3DTouch opens up a whole new design space for interaction techniques to further develop on. With 3DTouch, we attempt to bring 3D applications a step closer to users.

Benefits of immersive three-dimensional (3D) applications are enhanced by effective 3D interaction techniques. While gesture-based interaction provides benefits in these types of environments users commonly report higher fatigue than with other interaction solutions. Typically, fatigue is measured subjectively but may lack precision, consistency and depth. Our research proposes a novel, more consistent and predictable measure of fatigue. This paper presents the details of our technique based on human motor coordination, the results of an experimental study on gesture-based interaction, identifies attributing causes for fatigue and outlines design guidelines to reduce fatigue for gesture-based interaction techniques. These results have implications for gesture-based or mid-air interaction techniques for 3D environments, such as virtual environments and immersive visualizations.

Selection is a fundamental task in volume visualization as it is often the first step for manipulation and analysis tasks. The presented work describes and investigates a novel 3-Dimensional (3D) selection technique for dense clouds of points. This technique solves issues with current selection techniques employed in such applications by allowing users to select similar regions of datasets without requiring prior knowledge about the structures within the data, thus bypassing occlusion and high density. We designed a prototype and experimented on large dense volumetric datasets. The preliminary results of our performance evaluation and the user-simulated test show encouraging results and indicate in which environments this technique could have high potential.

In this paper, we present an investigation on the performance effects of analytical tools through visual and non-visual interaction in an immersive visualization. We explored two types of touch-based input device (with a display screen as direct and without a display screen as indirect), and compared these two touch-based input devices with a 6-degrees of freedom (DOF) tracked input device and a 2DOF input device, where a user could interact in 6DOF spatial context but the degrees of freedom were constrained. The results revealed that for visually demanding tasks, touch input is comparable to 6DOF, however it is important to use physical means to constrain degrees of freedom to retain performance levels using analytical tools involving selection. Furthermore results revealed that precision can be negatively affected by the design of the direct touch interface. Our results will have implications on touch-based interface design as well as design considerations when reducing degrees of freedom control.

Visualization enables scientists to transform data in its raw form to a visual form that will facilitate discoveries and insights. Although there are advantages for displaying inherently 3-dimensional (3D) data in immersive environments, those advantages are hampered by the challenges involved in selecting volumes of that data for exploration or analysis. Selection involves the user identifying a set of points for a specific task. This paper preliminary data collection on natural user actions for volume selection. This paper also presents a research agenda outlining an extension for volume selection classification, as well as challenges, for designing components for a direct selection of volumes of data points.

Our research is focused on the initial exploration of training a Brain Computer Interface (BCI) by using audio cues to navigate a virtual environment, instead of motor imagery. We have designed our BCI training and navigation to use audio cues that adhere to the dichotic listening (DL) mechanism so that users have an active choice for interaction or giving commands. We have implemented our Dichotic Listening BCI in a Virtual Environment so that it can be used to train users with disabilities to apply those skills for a BCI to control a real-world assisted locomotive device or to simply navigate within the virtual environment. We hypothesized that the lateralization of the brain's response to music and speech will enhance the classification of a BCI. Unlike previous attempts in using the oddball paradigm, our results show that audio cues can be used simultaneously to elicit distinct EEG signals for BCI while still enabling an active choice for the user. We evaluated users' performance to actively input navigation tasks. Dichotic Listening BCI performs slightly better than Motor Imagery based BCI. In addition navigation paths reveal participants having more navigation control when using Dichotic Listening.

Our research is focused on the initial exploration of training a Brain Computer Interface (BCI) by using audio cues to navigate a virtual environment, instead of motor imagery. We have designed our BCI training and navigation to use audio cues that adhere to the dichotic listening (DL) mechanism so that users have an active choice for interaction or giving commands. We have implemented our Dichotic Listening BCI in a Virtual Environment so that it can be used to train users to apply those skills for a BCI to control a real-world assisted locomotive device or to simply navigate within the virtual environment. We hypothesized that the lateralization of the brain's response to music and speech will enhance the classification of a BCI. Unlike previous attempts in using the oddball paradigm, our results show that audio cues can be used simultaneously to elicit distinct EEG signals for BCI while still enabling an active choice for the user. We evaluated users' performance to actively input navigation tasks. Dichotic Listening BCI performs slightly better than Motor Imagery based BCI.

Multimodal systems have been previously used as an aid to improve quality and safety inspection in various domains, though few studies have evaluated these systems for accuracy and user comfort. Our research aims to combine our software interface designed for high usability with multimodal hardware configurations and to evaluate these systems to determine their user performance benefits and user acceptance data. We present two multimodal systems for using a novel system-directed interface to aid in inspecting vehicles along the assembly line: (1) wearable monocular display with speech input and audio output and (2) large screen display with speech input and audio output. We conducted two evaluations: (a) an experimental evaluation with novice users, resulting in accuracy, timing, user preferences, and other performance results and (b) an expert-based usability evaluation conducted on and off the assembly line providing insight on user acceptance, preferences, and performance potential in the production environment. We also compared these systems to current technology used in the production environment: a handheld display without speech input/output. Our results show that for visual and tactile tasks, benefits of system-directed interfaces are best realized when used with multimodal systems that reduce visual and tactile interaction per item and instead deliver system-directed information on the audio channel. Interface designers that combine system-directed interfaces with multimodal systems can expect faster and more efficient user performance when the delivery channel is different from channels necessary for task completion.

In this paper we present initial research of the investigation in the design collaborative interaction modalities for classroom-based immersive visualizations of 3D spatial data, with an initial implementation for geo-spatial applications. Additionally we allowed some pilot testing to gain a sense of our design decisions and where user error might occur. Valuable feedback will allow us to redesign and refine implementation for a much more formal long-term evaluation of the system. Initial results give indications that our interaction modalities may facilitate teaching and learning, but the use of devices should be different for user type.

There are benefits for social and behavioral researchers to conduct studies in online virtual worlds. However, typically learning scripting takes additional time or money to hire a consultant. We propose a prototype Virtual World Toolkit for to help researchers design, set up, and run experiments in Virtual Worlds, with little coding or scripting experience needed. We explored three types of prototype designs, focused on a traditional interface with pilot results. We also present results of initial expert user study of our toolkit to determine the learnability, usability, and feasibility of our toolkit to conduct experiments. Results suggest that our toolkit requires little training and sufficient capabilities for a basic experiment. The toolkit received a great feedback from a number of expert users who thought that it is a promising first version that lays the foundation to more future improvements. This toolkit prototype contributes to enhancing researchers’ capabilities in conducting social/behavioral studies in virtual worlds and hopefully will empower social and behavioral researchers by proving a toolkit prototype that requires less time, efforts and costs to setup stimulus responses types of human subject studies in virtual worlds.