Search

This paper presents a set of adaptive reset techniques for use with haptic retargeting systems focusing on interaction with hybrid virtual reality interfaces that align with a physical interface. Haptic retargeting between changing physical and virtual targets requires a reset where the physical and virtual hand positions are re-aligned. We present a modified Point technique to guide the user in the direction of their next interaction such that the remaining distance to the target is minimized upon completion of the reset. This, along with techniques drawn from existing work are further modified to consider the angular and translational gain of each redirection and identify the optimal position for the reset to take place. When the angular and translational gain is within an acceptable range, the reset can be entirely omitted. This enables continuous retargeting between targets removing interruptions from a sequence of retargeted interactions. These techniques were evaluated in a user study which showed that adaptive reset techniques can provide a significant decrease in task completion time, travel distance, and the number of user errors Refereed/Peer-reviewed

Augmented reality (AR) technologies create an immersive environment by augmenting the real world with rendered virtual objects. One of the key requirements of an AR system is to understand how users perceive the depth of an AR object. Perceived distances to AR objects can be based on various depth cues such as rendering method used for the virtual object. The existing researches on the shading model and opacity mainly focus on the shape perception or the position relationship of several virtual objects in the virtual environment. We predicted that shading models and opacity would impact a user's estimation of a virtual object's depth in AR. We conducted two experiments to investigate the impact of color, size (Experiment 1), shading model, and opacity (Experiment 2) on the depth perception in an optical see-through head-mounted display (HMD). We found that the virtual object's color and size impact estimation efficacy. An interaction effect between color and size was also found. The results showed that shading models with specular highlights could help to improve depth perception in AR. Additionally, users had the lowest matching error when the opacity of a virtual object was 0.8. Refereed/Peer-reviewed

Immersive analytics turns the very space surrounding the user into a canvas for data analysis, supporting human cognitive abilities in myriad ways. We present the results of a design study, contextual inquiry, and longitudinal evaluation involving professional economists using a Virtual Reality (VR) system for multidimensional visualization to explore actual economic data. Results from our preregistered evaluation highlight the varied use of space depending on context (exploration vs. presentation), the organization of space to support work, and the impact of immersion on navigation and orientation in the 3D analysis space. Refereed/Peer-reviewed

This paper investigates the use of immersive virtual reconstructions as an aid for jurors during a courtroom trial. The findings of a between-participant user study on memory and decision-making are presented in the context of viewing a simulated hit-run-death scenario. Participants listened to the opening statement of a prosecutor and a defence attorney before viewing the crime scene in Virtual Reality (VR) or as still images. We compare the effects on cognition and usability of using VR over images presented on a screen. We found several significant improvements, including that VR led to more consistent decision-making among participants.This shows that VR could provide a promising solution for the court to present crime scenes when site visitations are not possible. Refereed/Peer-reviewed

Remote assistance represents an important use case for mixed reality. With the rise of handheld and wearable devices, remote assistance has become practical in the wild. However, spontaneous provisioning of remote assistance requires an easy, fast and robust approach for capturing and sharing of unprepared environments. In this work, we make a case for utilizing interactive light fields for remote assistance. We demonstrate the advantages of object representation using light fields over conventional geometric reconstruction. Moreover, we introduce an interaction method for quickly annotating light fields in 3D space without requiring surface geometry to anchor annotations. We present results from a user study demonstrating the effectiveness of our interaction techniques, and we provide feedback on the usability of our overall system. Refereed/Peer-reviewed

Three-dimensional scatterplots suffer from well-known perception and usability problems. In particular, overplotting and occlusion, mainly due to density and noise, prevent users from properly perceiving the data. Thanks to accurate head and hand tracking, immersive Virtual Reality (VR) setups provide new ways to interact and navigate with 3D scatterplots. VR also supports additional sensory modalities such as haptic feedback. Inspired by methods commonly used in Scientific Visualisation to visually explore volumes, we propose two techniques that leverage the immersive aspects of VR: first, a density-based haptic vibration technique (Scaptics) which provides feedback through the controller; and second, an adaptation of a cutting plane for 3D scatterplots (Highlight-Plane). We evaluated both techniques in a controlled study with two tasks involving density (finding high- and low-density areas). Overall, Scaptics was the most time-efficient and accurate technique, however, in some conditions, it was outperformed by Highlight-Plane Refereed/Peer-reviewed

International audience; We introduce IATK, the Immersive Analytics Toolkit, a software package for Unity that allows interactive authoring and exploration of data visualisation in immersive environments. The design of IATK was informed by interdisciplinary expert-collaborations as well as visual analytics applications and iterative refinement over several years. IATK allows for easy assembly of visualisations through a grammar of graphics that a user can configure in a GUI— in addition to a dedicated visualisation API that supports the creation of novel immersive visualisation designs and interactions. IATK is designed with scalability in mind, allowing visualisation and fluid responsive interactions in the order of several million points at a usable frame rate. This paper outlines our design requirements, IATK’s framework design and technical features, its user interface, as well as application examples.

This paper presents the use of a projector-based spatial augmented reality system in an industrial quality assurance setting to highlight spot-weld locations on vehicle panels for manual welding operators. The aim of this work is to improve the precision and accuracy of manual spot-weld placements with the aid of visual cues as a proactive step by the automotive manufacturer to enhance product quality. The prototype system was deployed at General Motors (GM) Holden plant in Elizabeth, Australia on the production line building Holden Cruze vehicles. Production trials were conducted and techniques developed to analyse and validate the precision and accuracy of spot-welds both with and without the visual cues. A reduction of 52 % of the standard deviation of manual spot-weld placement was observed when using augmented reality visual cues. The average standard deviation with-AR assistance (19 panels and 114 spot-welds) was calculated at 1.94 mm compared to without-AR (45 panels and 270 spot-welds) at 4.08 mm. All welds were within the required specification and panels evaluated in this study were used as the final product made available to consumers. The visual cues enabled operators to spot-weld at a higher degree of precision and accuracy. Refereed/Peer-reviewed

This paper presents the use of a projector-based spatial augmented reality system in an industrial quality assurance setting to highlight spot-weld locations on vehicle panels for manual welding operators. The aim of this work is to improve the precision and accuracy of manual spot-weld placements with the aid of visual cues as a proactive step by the automotive manufacturer to enhance product quality. The prototype system was deployed at General Motors (GM) Holden plant in Elizabeth, Australia on the production line building Holden Cruze vehicles. Production trials were conducted and techniques developed to analyse and validate the precision and accuracy of spot-welds both with and without the visual cues. A reduction of 52 % of the standard deviation of manual spot-weld placement was observed when using augmented reality visual cues. The average standard deviation with-AR assistance (19 panels and 114 spot-welds) was calculated at 1.94 mm compared to without-AR (45 panels and 270 spot-welds) at 4.08 mm. All welds were within the required specification and panels evaluated in this study were used as the final product made available to consumers. The visual cues enabled operators to spot-weld at a higher degree of precision and accuracy. Refereed/Peer-reviewed

Research into how virtual reality (VR) can be a beneficial technology for new and emerging large, complex data visualizations for data scientists is ongoing. In this paper, we evaluate three-dimensional VR navigation technique for data visualizations and test their effectiveness with a large graph visualization. We evaluate two prominent navigation techniques employed in VR (Teleportation and One-Handed Flying) against two less common methods (Two-Handed Flying and Worlds In Miniature) and evaluate their performance and effectiveness through a series of tasks. We found Steering Patterns (One Handed Flying and Two-Handed Flying) to be faster and preferred by participants for completing searching tasks in comparision to Teleportation. Worlds-In-Miniature was the least physically demanding of the navigations, and was preferred by participants for tasks that required an overview of the graph such as triangle counting Refereed/Peer-reviewed

Recent advances in virtual reality technologies enable high-fidelity exploration of data in immersive environments. This is advantageous for professional applications of high-dimensional datasets (such as geo-temporal narratives), as we can leverage all three spatial axes while immersing the user in the information itself. Geo-temporal narratives tell a story of entities, their movements, and as a result, their potential relationships, thereby defining the who, what, where, and when that define a story; everything except the why. This paper describes an immersive virtual reality system we have developed to convey these narratives, specifically focusing on the law enforcement domain. The system lets users not only view who was where and when, but also view explicit and implicit relationships between entities, repeated visits to recurring locations, as well as the crucial descriptive information supporting the why. We present the results of an expert review of the system from federal law enforcement and defence agencies that validate our approach. Refereed/Peer-reviewed

Functional brain networks (FBNs) are gaining increasing attention in computational neuroscience due to their ability to reveal dynamic interdependencies between brain regions. The dynamics of such networks during cognitive activity between stimulus and response using multi-channel electroencephalogram (EEG), recorded from 16 healthy human participants are explored in this research. Successive EEG segments of 500[Formula: see text]ms duration starting from the onset of cognitive stimulation have been used to analyze and understand the cognitive dynamics. The approach employs a combination of signal processing techniques, nonlinear statistical measures and graph-theoretical analysis. The efficacy of this approach in detecting and tracking cognitive load induced changes in EEG data is clearly demonstrated using graph metrics. It is revealed that most cognitive activity occurs within approximately 500[Formula: see text]ms of the stimulus presentation in addition to temporal variability in the FBNs. It is shown that mutual information (MI), a nonlinear measure, produces good correlations between the EEG channels thus enabling the construction of FBNs which are sensitive to cognitive load induced changes in EEG. Analyses of the dynamics of FBNs and the visualization approach reveal hard to detect subtle changes in cognitive function and hence may lead to a better understanding of cognitive processing in the brain. The techniques exploited have the potential to detect human cognitive dysfunction (impairments).

This paper presents a context-aware design pattern for situated analytics called Blended Model View Controller. Our approach is an event-driven design, allowing a seamless transition between the physical space and information space during use. The Blended Model View Controller allows common user interface controls to work in tandem with printed information on a physical object by adapting the operation and presentation based on a semantic matrix. Also presented is an authoring tool that has been developed to assign the parameters of the semantic matrix. We demonstrate the use of the design pattern with a set of augmented reality interactions including; pinch zoom, menus, and details-on-demand. We analyse each control to highlight how the physical and virtual information spaces work in tandem to provide a rich interaction environment in augmented reality. Refereed/Peer-reviewed

We present HoloBee, a system for visual analytics of bee drift data using a head-mounted augmented reality interface (Microsoft HoloLens). HoloBee facilitates interactive manipulation and exploration of the bee activity data on 3D geo- spatial terrains to gain insights into the drifting behaviour of honey bees in their natural habitat. We conducted a user study to compare the task performance and user experience of HoloBee with a conventional desktop application. The results show that HoloBee allows for solving tasks in a similar time and accuracy compared to the desktop interface, despite participants' having significantly more experience interacting with a desktop interface. We received positive feedback from a subset of participants on the natural and intuitive interactions that HoloBee provides, while others preferred using the desktop application. We discuss the implications and limitations of these results and provide guidance for future work in this direction. Refereed/Peer-reviewed

Ecologists often need to extract data from forests to determine the flora and fauna conditions. Tree climbing is one technique used for this. Climbers go to the top of the tree and use hand held equipment to collect and make notes while they are descending. This process demands expert climbing skills and biology expertise, which are two characteristics not commonly found. This paper describes the design of mobile system for 3D data collection and reconstruction of the field research environment for remote evaluation. The prototype can be used to reduce the time spent up trees and collected data that can be used for crowd-sourced evaluation. Refereed/Peer-reviewed

This paper presents our new Virtual Reality (VR) interactive visualization techniques to assist users querying large image sets. The VR system allows users to query a set of images on four different filters such as locations and keywords. The goal is to investigate if a VR platform is preferred over a non-VR platform for viewing and querying large image sets. We employed an HTC Vive and a traditional desktop screen to represent VR and non-VR platforms. We found users preferred the VR platform over the traditional desktop screen.

High-quality immersive display technologies are becoming mainstream with the release of head-mounted displays (HMDs) such as the Oculus Rift. These devices potentially represent an affordable alternative to the more traditional, centralised CAVE-style immersive environments. One driver for the development of CAVE-style immersive environments has been collaborative sense-making. Despite this, there has been little research on the effectiveness of collaborative visualisation in CAVE-style facilities, especially with respect to abstract data visualisation tasks. Indeed, very few studies have focused on the use of these displays to explore and analyse abstract data such as networks and there have been no formal user studies investigating collaborative visualisation of abstract data in immersive environments. In this paper we present the results of the first such study. It explores the relative merits of HMD and CAVE-style immersive environments for collaborative analysis of network connectivity, a common and important task involving abstract data. We find significant differences between the two conditions in task completion time and the physical movements of the participants within the space: participants using the HMD were faster while the CAVE2 condition introduced an asymmetry in movement between collaborators. Otherwise, affordances for collaborative data analysis offered by the low-cost HMD condition were not found to be different for accuracy and communication with the CAVE2. These results are notable, given that the latest HMDs will soon be accessible (in terms of cost and potentially ubiquity) to a massive audience. Refereed/Peer-reviewed

This paper presents a novel technique, called Horus Eye, for augmented reality information visualization. 'Horus', the famous deity in ancient Egyptian mythology, inspires this visualization technique, which is designed to simulate bird and snake vision to highlight data of interest. The contribution of this approach is the merging of information with the real scene, leveraging the real world context to interpret the data. Our technique is a context-based interactive visualization, controlled by users' queries. This paper presents a work in progress with use cases and two adaptations of Horus Eye. Refereed/Peer-reviewed

This paper presents design guidelines and recommendations for developing cursor manipulation interaction devices to be employed in a wearable context. The work presented in this paper is the culmination three usability studies designed to understand commercially available pointing (cursor manipulation) devices suitable for use in a wearable context. The set of guidelines and recommendations presented are grounded on experimental and qualitative evidence derived from three usability studies and are intended to be used in order to inform the design of future wearable input devices. In addition to guiding the design process, the guidelines and recommendations may also be used to inform users of wearable computing devices by guiding toward the selection of a suitable wearable input device. The synthesis of results derived from a series of usability studies provide insights pertaining to the choice and usability of the devices in a wearable context. That is, the guidelines form a checklist that may be utilized as a point of comparison when choosing between the different input devices available for wearable interaction. Refereed/Peer-reviewed

Research on 3D interaction has explored the application of multi-touch technologies to 3D stereoscopic displays. However, the ability to perceive 3D objects at different depths (in front or behind the screen surface) conflicts with the necessity of expressing inputs on the screen surface. Touching the screen increases the risk of causing the vergence-accommodation conflict which can lead to the loss of the stereoscopic effect or cause discomfort. In this work, we present two studies evaluating a novel approach based on the concept of indirect touch interaction via an external multi-touch device. We compare indirect touch techniques to two state-of-the-art 3D interaction techniques: DS3 and the Triangle Cursor. The first study offers a quantitative and qualitative study of direct and indirect interaction on a 4 DOF docking task. The second presents a follow-up experiment focusing on a 6 DOF docking task. Results show that indirect touch interaction techniques provide a more comfortable viewing experience than both techniques. It also shows that there are no drawbacks when switching to indirect touch, as their performances in terms of net manipulation times are comparable. ispartof: pages:13-22 ispartof: 2016 IEEE Symposium on 3D User Interfaces (3DUI) pages:13-22 ispartof: IEEE Symposium on 3D User Interfaces (3DUI) location:Greenville: SC date:19 Mar - 20 Mar 2016 status: published

Typical personal fabrication using a laser cutter allows objects to be created from raw material and the engraving of existing objects. Current methods to precisely align an object with the laser is a difficult process due to indirect manipulations. In this paper, we propose a marker-based system as a novel paradigm for direct interactive laser cutting on existing objects. Our system, MARCut, performs the laser cutting based on tangible markers that are applied directly onto the object to express the design. Two types of markers are available; hand constructed Shape Markers that represent the desired geometry, and Command Markers that indicate the operational parameters such as cut, engrave or material. Refereed/Peer-reviewed

Given the diverse set of pervasive tracking technologies available, temporal-geospatial data is being collected at an unprecedented rate. However, the effective visualization and interpretation of this data remains elusive. Visualizations have focused on showing an object's location, however more complex inter-entity queries also need to be supported, e.g. "did X and Y meet, and if so, where and when?". We present Cooperative Visual Analysis, a combination of two novel visualizations, the Parallel Schedule View and the Braille Plot, working in synergy with a traditional 2D map. The Parallel Schedule View focuses on showing colocation (simultaneous or time separated), with the Braille Plot used to resolve position ambiguity and identify patterns and trends within a data trace (in addition to colocation). We present descriptions of each, and a user study showing support for these approaches. The study compared Cooperative Visual Analysis with a current approach, the Space Time Cube, and found the Cooperative Visual Analysis is an effective means for visualizing temporal-geospatial relationships in a data set, performing at or above the Space Time Cube, whilst being preferred by users. Refereed/Peer-reviewed

Augmented reality systems commonly employ overt cueing to direct a user's attention and guide their actions. Spatial augmented reality based supraliminal annotations have been shown to improve user performance compared to LCD screens. This paper explores subliminal cues: annotations that exist below the threshold of consciousness. We investigate whether subliminal cueing is technically possible with standard data projectors, and if subliminal cues can further improve users' reaction time in procedural tasks. This paper describes a new technique for temporal subliminal cues in spatial augmented reality, and presents the results of three user studies evaluating the effectiveness of this technique in a button-pressing task. The results show that the presentation of annotations is indeed subliminal, in that the visual stimulus was not perceivable to users. We found a statistically significant improvement in task performance when using subliminal cues and supraliminal annotations, compared to supraliminal annotations alone, with mean trial times improving from 633.49ms to 604.64ms. Refereed/Peer-reviewed

Integrating novel feedback mechanisms into wearable devices could support medical rehabilitation applications and enhance existing interaction methods. One such feedback mechanism is layer jamming, which can be used to alter material stiffness in real time under computer control. Jamming can be integrated into smart garments, allowing clothing to adapt to environmental conditions or support a particular user task. For example, a person with a knee injury that requires restricted movement for their treatment would normally wear a knee brace. Layer jamming-enabled smart pants can be integrated as part of the treatment to restrict both speed and angular movement of the affected joint during recovery. This article explores the use of smart materials that combine low-profile jamming with clothing items for computer-controlled material stiffness. The authors present a current prototype system that employs jamming technology for this variable stiffness function and discuss future applications.

This paper presents a new mitten incorporating vacuum layer jamming technology to provide haptic feedback to a user. We demonstrate that layer jamming technology can be successfully applied to a mitten, and discuss advantages layer jamming provides as a wearable technology through its low profile form factor. Jamming differs from traditional wearable haptic systems by restricting a user's movement, rather than applying an actuation force on the user's body. Restricting the user's movement is achieved by varying the stiffness of wearable items, such as gloves. We performed a pilot study where the qualitative results showed users found the haptic sensation of the jamming mitten similar to grasping the physical counterpart. Refereed/Peer-reviewed

Providing high resolution tangible user interface components without batteries such as dials and sliders that support dy- namic user interface arrangement is challenging. Previous work uses RFID to support limited resolution custom-built components. We demonstrate improved techniques using commercial off the shelf input controls incorporated into pas- sive RFID tags using an on-off key subcarrier to encode state information into the RFID signal. Our method supports high resolution components that do not require power cables or batteries. We provide exemplars demonstrating how the technique supports a range of user interface components in- cluding buttons, dials, sliders, flex and light sensors. Com- pared to previous work, we obtain a higher resolution, only limited by sample time, for all components and demonstrate 115 discrete dial positions. Our technique allows the TUI components to be freely placed and rearranged without hard- wiring or batteries. Refereed/Peer-reviewed

Mobile devices such as some recent phones are now fitted with projection capabilities that support Spatial Augmented Reality (SAR) and require investigation to uncover new interaction possibilities. This paper presents a study measuring user performance in a search and select task using a tracked handheld projector and data tags, a 3D physical cue. This physical cue is used to mark the location of hidden SAR information. The experiment required participants to search for virtual symbols presented on two 5ft, multi-sided control panels. Two methods of presenting AR information were employed, SAR alone and SAR with the inclusion of physical cues to indicate the location of the information. The results showed that attaching data tags, compared to virtual content alone lowered the overall task completion time and reduced handheld projector movement. Subjectively, participants also preferred the combination of virtual data with data tags across both task variations. Refereed/Peer-reviewed

This paper presents distance-based interactions for wearable augmented reality systems enabled by an Ultrasonic Glove input device. The ultrasonic glove contains a tilt sensor and a pair of ultrasonic transducers in the palms. The transducers are distance-ranging sensors that emit ultrasonic waves from the palm of the hand. We developed distance-based interactions including modeling by measurement, numeric entry, and affine transformation interaction techniques. The interactions are based on natural gestures such as facing the palms towards each other or other surfaces. Virtual models of physical objects are created by the user specifying the dimensions with hand gestures. Distance data from the ultrasonic transducers is combined with orientation data to create dimensional vectors and construct models. We conducted an evaluation for the techniques and input device, including a pilot experiment, a user study, and an expert study session. The results indicated that for the task of modeling physical objects, the ultrasonic glove reduced completion time and, in many cases, task error. Our techniques can be generalized to different sensor technologies. Refereed/Peer-reviewed

This paper explores enhancing mobile immersive augmented reality manipulations by providing a sense of computer-captured touch through the use of a passive deformable haptic glove that responds to objects in the physical environment. The glove extends our existing pinch glove design with a Digital Foam sensor that is placed under the palm of the hand. The novel glove input device supports a range of touch-activated, precise, direct manipulation modeling techniques with tactile feedback including hole cutting, trench cutting, and chamfer creation. A user evaluation study comparing an image plane approach to our passive deformable haptic glove showed that the glove improves a user's task performance time, decreases error rate and erroneous hand movements, and reduces fatigue. Refereed/Peer-reviewed

The ability to visualize occluded objects or people offers tremendous potential to users of augmented reality (AR). This is especially true in mobile applications for urban environments, in which navigation and other operations are hindered by the urban infrastructure. This “X-ray vision” feature has intrigued and challenged AR system designers for many years, with only modest progress in demonstrating a useful and usable capability. The most obvious challenge is to the human visual system, which is being asked to interpret a very unnatural perceptual metaphor. We review the perceptual background to understand how the visual system infers depth and how these assumptions are or are not met by augmented reality displays. In response to these challenges, several visualization metaphors have been proposed; we survey these in light of the perceptual background. Because augmented reality systems are user-centered, it is important to evaluate how well these visualization metaphors enable users to perform tasks that benefit from X-ray vision. We summarize studies reported in the literature. Drawing upon these analyses, we offer suggestions for future research on this tantalizing capability.

Ray tracing is an elegant and intuitive image generation method. The introduction of GPU-accelerated ray tracing and corresponding software frameworks makes this rendering technique a viable option for Augmented Reality applications. Spatial Augmented Reality employs projectors to illuminate physical models and is used in fields that require photorealism, such as design and prototyping. Ray tracing can be used to great effect in this Augmented Reality environment to create scenes of high visual fidelity. However, the peculiarities of SAR systems require that core ray tracing algorithms be adapted to this new rendering environment. This paper highlights the problems involved in using ray tracing in a SAR environment and provides solutions to overcome them. In particular, the following issues are addressed: ray generation, hybrid rendering and view-dependent rendering. Refereed/Peer-reviewed

We present a set of distance-based interaction techniques for modeling and manipulation, enabled by a new input device called the ultrasonic gloves. The ultrasonic gloves are built upon the original design of the pinch glove device for virtual reality systems with a tilt sensor and a pair of ultrasonic transducers in the palms of the gloves. The transducers are distance-ranging sensors that allow the user to specify a range of distances by natural gestures such as facing the palms towards each other or towards other surfaces. The user is able to create virtual models of physical objects by specifying their dimensions with hand gestures. We combine the reported distance with the tilt orientation data to construct virtual models. We also map the distance data to create a set of affine transformation techniques, including relative and fixed scaling, translation, and rotation. Our techniques can be generalized to different sensor technologies. Refereed/Peer-reviewed

This paper provides a provocative view of wearable computer research over the years, starting with the first IEEE International Symposium on Wearable Computers in 1997. The goal of this paper is to reflect on the original research challenges from the first few years. With this goal in mind, two questions can be examined: 1) have we achieved the goals we set out? and 2) how has the direction of research changed in the past fifteen years? This is not a survey paper, but a platform to stimulate discussion. Refereed/Peer-reviewed

This paper explores how passive colored markers used for blob-tracking can be advanced to improve their operation in spatial augmented reality environments. We have created an adaptive marker that captures the color of environmental light, and then changes its own appearance in order to optimize tracking performance. We have provided implementation details and proof of concept for an active marker that supports interactive operations and visual feedback. We compared the performance of both the passive and adaptive color marker in varying lighting conditions. Our results show a significant improvement in four scenarios where a passive marker is known to have poor tracking quality. Refereed/Peer-reviewed

Spatial Augmented Reality allows the appearance of physical objects to be transformed using projected light. Computer controlled light projection systems have become readily available and cost effective for both commercial and personal use. This chapter explores how large Spatial Augmented Reality systems can be applied to enhanced design mock-ups. Unlike traditional appearance altering techniques such as paints and inks, computer controlled projected light can change the color of a prototype at the touch of a button allowing many different appearances to be evaluated. We present the customized physical-virtual tools we have developed such as our hand held virtual spray painting tool that allows designers to create many customized appearances using only projected light. Finally, we also discuss design parameters of building dedicated projection environments including room layout, hardware selection and interaction considerations.

In this chapter we present our research directions on the problem of action at a distance in outdoor augmented reality using wearable computers. Our most recent work presents the augmented viewport technique to enable interactions with distant virtual objects in augmented reality. Our technique utilizes physical cameras to provide real world information from the distant location. We examine a number of factors required to achieve an efficient and effective solution for precise manipulation at a distance for outdoor augmented reality using wearable computers. These include the improved usage of physical cameras, collaboration, viewpoint visualization, alignment of virtual objects, and improved input devices. We particularly focus on the collaboration aspect of the technique, with the utilization of remote cameras from multiple users of wearable computer systems and mobile devices. Such collaboration supports precise manipulation tasks by allowing viewing from different perspectives, directions, and angles, as well as collaborative ­precise triangulation of virtual objects in the augmented environment.

Effective designs rarely emerge from good structural design or aesthetics alone. It is more often the result of the end product's overall design integrity. Added to this, design is inherently an interdisciplinary collaborative activity. With this in mind, today's tools are not powerful enough to design complex physical environments, such as command control centers or hospital operating theaters. In this demo a process for designing small appliances using spatial augmented reality is presented. The goal of this demo is to show how using a SAR environment can improve the design process. Refereed/Peer-reviewed

Spatial Augmented Reality (SAR) superimposes computer generated virtual objects directly on physical objects' surfaces. This enables user to interact with real world objects in a natural manner. This chapter investigates SAR techniques and summarizes advantages with the difficulties of SAR in presenting digital information to users. The chapter then presents a concept of portable collaborative SAR. The concept utilizes both projector-based SAR and Head-Mounted-Display (HMD) based Augmented Reality (AR) in a single environment to assist collaborations within multiple users. The concept combines advantages of both projector-based SAR for collaboration and HMD-based AR display in personalization to improve the efficiency of collaborative tasks. The presented concept is explored in a case study of industrial quality assurance scenario to show its effectiveness.

Developing usable and robust mixed reality systems requires unique human-computer interaction techniques and customized hardware systems. The design of the hardware is directed by the requirements of the rich 3D interactions that can be performed using immersive mobile MR systems. Geometry modeling and capture, navigational annotations, visualizations, and training simulations are all enhanced using augmented computer graphics. We present the design guidelines that have led us through 10 years of evolving mobile outdoor MR hardware systems.

This paper presents the e-SUIT, a wearable computer incorporated in a traditional business suit. A key feature of the system is an array of input/output devices integrated into the garment. These devices are connected to a network bus incorporated into the suit jacket. The network connects the jackets sensors and I/O with a Compaq iPAQ/sup TM/ operating Windows CE. The iPAQ is connected to the users personal information systems via a wireless LAN. Demonstrated within is an application allowing the e-SUIT to control the factory installed Microsoft Pocket Outlook/sup /spl reg//. Pocket Outlook is a functioning business class application that allows the proof of concept testing to be performed on an integrated product used in the marketplace.

Projection-based Augmented Reality commonly employs a rigid substrate as the projection surface and does not support scenarios where the substrate can be reshaped. This investigation presents a projection-based AR system that supports deformable substrates that can be bent, twisted or folded. We demonstrate a new invisible marker embedded into a deformable substrate and an algorithm that identifies deformations to project geometrically correct textures onto the deformable object. The geometrically correct projection-based texture mapping onto a deformable marker is conducted using the measurement of the 3D shape through the detection of the retro-reflective marker on the surface. In order to achieve accurate texture mapping, we propose a marker pattern that can be partially recognized and can be registered to an object's surface. The outcome of this work addresses a fundamental vision recognition challenge that allows the underlying material to change shape and be recognized by the system. Our evaluation demonstrated the system achieved geometrically correct projection under extreme deformation conditions. We envisage the techniques presented are useful for domains including prototype development, design, entertainment and information based AR systems. Refereed/Peer-reviewed