Search

This study investigated the effects of Augmented Reality (AR) on learning, cognitive load and spatial abilities. More specifically, it measured learning gains, perceived cognitive load, and the role spatial abilities play with students engaged in an astronomy lesson about lunar phases. Research participants were 182 students from a public university in southeastern United States, and were recruited from psychology research pool. Participants were randomly assigned to two groups: (a) Augmented Reality and Text Astronomy Treatment (ARTAT); and (b) Images and Text Astronomy Treatment (ITAT). Upon entering the experimental classroom, participants were given (a) Paper Folding Test to measure their spatial abilities; (b) the Lunar Phases Concept Inventory (LPCI) pre-test; (c) lesson on Lunar Phases; (d) NASA-TLX to measure participants’ cognitive load; and (e) LPCI post-test. Statistical analysis found (a) no statistical difference for learning gains between the ARTAT and ITAT groups; (b) statistically significant difference for cognitive load; and (c) no significant difference for spatial abilities scores.

This dissertation investigates the role of annotative locative media in mediating experiences of place. The overarching impetus motivating this research is the need to bring to bear the theoretical and substantive concerns of cultural landscape studies on the development of a methodological framework for interrogating the ways in which annotative locative media reconfigure experiences of urban landscapes. I take as my empirical cases i) Google Maps with its associated Street View and locational placemark interface, and ii) Layar, an augmented reality platform combining digital mapping and real-time locational augmentation. In the spirit of landscape studies’ longstanding and renewed interest in what may be termed “ordinary” residential landscapes, and reflecting the increasing imbrication of locative media technologies in everyday lives, the empirical research is based in Kenwick, a middleclass, urban residential neighborhood in Lexington, Kentucky. Overall, I present an argument about the need to consider the digital, code (i.e. software), and specifically locative media, in the intellectual context of critical geographies in general and cultural landscape studies in particular.

Under the title “Emerging Topologies”, this thesis draws together theories conceptualising space (Michel de Certeau's tactical and strategic) and theories spatialisating concepts (Gilles Deleuze and Felix Guattari's machinic assemblages) in order to examine our contemporary experiences of the city 'augmented' by digital technology. This thesis explores how developments in- and public adoption of- mapping, navigation, and documentation technology are leading to the emergence of new topological connections between people, technology, and environment, resulting in an everyday spatial practice that increasingly inhabits digital, virtual space. Virtual space has been characterised variously as oppressive, emancipating, or degrees of both (Grozs, Shields, Haraway). This thesis argues that virtual space is a 'real' space in its own right (politically, socially, phenomenologically) and that despite its rigid organisation, propositional logic, and digital representation, it provides a strategic territory just as open to tactical inhabitation as the conventional built environment. Building on the author's training as a computer scientist and artist, software was developed to map, scan, and visualise physical space, and virtual environments were created and explored as potential sites for both poetic, creative expression and rationalisation, digitising, and ordering of space. The emerging topologies of body-technology-environment, as formations of new machinic assemblages, can dramatically disrupt the landscape around them but – as this thesis demonstrates – they may not remain 'in flight' for long. Tours solidify into maps. Early experiments into mapping tactical spatial practices offered valuable insight into the ease with which virtual space can be reterritorialised: my first attempt at an engagement with local graffiti practice (authoring a public, real-time map of graffiti using Google Maps) became mobilised against it, facilitating the logistical work of council clean-up crews. However, occasionally practices can remain temporarily autonomous from strategic control, tracing alternative, speculative spaces. I demonstrate that while the tactical practices of everyday life are being increasingly mapped, it is always possible to make poetic journeys across the mapped territory, whether city, virtual environment, or text.

Augmented reality technology involves the layering of dynamic, context-aware computer-generated input over the user s field of vision. The adoption of augmented reality in game design has resulted in the creation of playful experiences that explicitly link the game world to everyday reality. These experiences have been under theorised. Instead, researchers and designers have focused on the technical aspects of augmented reality, which has created a spatial dualism where augmented realty is conceptualised within a binary of the real or the virtual. This thesis examines the spatial experiences associated with mobile augmented reality games; conceptualising augmented reality as a process of spatial layering. Engaging with mobile augmented reality games both theoretically and empirically, this thesis extends scholarship and implications of future design. This thesis re-evaluates the spatial metaphors used to understand augmented reality as a multi-layered spatial experience that directly engages the player. Over-emphasis on the technical aspects of mobile augmented reality leads to conceptualisations of it as primarily a see through screen allowing for easy adoption of the screen-as-a-window metaphor. Similarly, an over emphasis on the boundary that separates play from everyday reality sidetracks player agency in negotiating and establishing such contractual boundaries. The result is a construction of a spatial dualism, where play is conducted either inside or outside of a fixed boundary. Such metaphors are re-evaluated as conceptual tools for the analysis of mobile augmented reality, arguing for a conceptualisation of mobile augmented reality games as multi-spatial layers. Case studies of a selection of mobile augmented reality games are analysed utilising mixed methodologies to create a triangulation of data. Four mobile augmented reality games are analysed as a series of elements, focusing on boundary maintenance, prescription of roles, and the integration of the everyday. The fluidity of the roles is re-examined via the direct observation of research participants, who expressed their experience in interviews post-play. The result is a holistic understanding of a select type of mobile augmented reality games. What emerges is a view of augmented reality not just as a process of layering computer-generated input over the real environment, but as a series of spatial negotiations performed by the player.

Abstract: Today, with gaming technology advancing by leaps-and-bounds, we are witnessing the proliferation of games for entertainment and education. Users can easily record their real-world experiences, through affordable smart phones, equipped with accelerometers, GPS receivers, compasses, and cameras. Commodity virtual worlds enable users to socialize, in realistic environments where they simulate real-world activities, or in imaginary worlds where they can play fantasy games. Finally, augmented reality browsers enable users to ‘annotate’ the real world with digital content through their mobile-devices displays. This synergy of technological advances and perceived service opportunities make the design and implementation of Mobile Augmented Alternate Reality Games (MAARGs) a compelling research problem. In this thesis, we present fAARS, an innovative platform that fully exploits these technologies to support the design, development and deployment of a variety of mobile games, including two systematically evaluated games in collaboration with the Psychology Department and the Faculty of Medicine.

Construction projects require large amounts of cyber-information, such as 3D models, in order to achieve success. Unfortunately, this information is typically difficult for construction field personnel to access and use on-site, due to the highly mobile nature of the job and hazardous work environments. Field personnel rely on carrying around large stacks of construction drawings, diagrams, and specifications, or traveling to a trailer to look up information electronically, reducing potential project efficiency. This thesis details my work on Hybrid 4-Dimensional Augmented Reality, known as HD4AR, a mobile augmented reality system for construction projects that provides high-precision visualization of semantically-rich 3D cyber-information over real-world imagery. The thesis examines the challenges related to augmenting reality on a construction site, describes how HD4AR overcomes these challenges, and empirically evaluates the capabilities of HD4AR.

21st Century civilization allows a glimpse into the progressive art paradigm that encompasses our constantly transforming technological advancements. These ideas explore the global network of our modern times and the interrelated possibilities within it. My collaborative and interdisciplinary methods give a glimpse into the future of digital/print media, augmented reality, and interactive environments. My artwork explores the hybrid, identity, culture, and an evolving reality within an age of Digitalism.

In Civil Infrastructure System (CIS) applications, the requirement of blending synthetic and physical objects distinguishes Augmented Reality (AR) from other visualization technologies in three aspects: 1) it reinforces the connections between people and objects, and promotes engineers’ appreciation about their working context; 2) It allows engineers to perform field tasks with the awareness of both the physical and synthetic environment; 3) It offsets the significant cost of 3D Model Engineering by including the real world background. The research has successfully overcome several long-standing technical obstacles in AR and investigated technical approaches to address fundamental challenges that prevent the technology from being usefully deployed in CIS applications, such as the alignment of virtual objects with the real environment continuously across time and space; blending of virtual entities with their real background faithfully to create a sustained illusion of co- existence; integrating these methods to a scalable and extensible computing AR framework that is openly accessible to the teaching and research community, and can be readily reused and extended by other researchers and engineers. The research findings have been evaluated in several challenging CIS applications where the potential of having a significant economic and social impact is high. Examples of validation test beds implemented include an AR visual excavator-utility collision avoidance system that enables spotters to ”see” buried utilities hidden under the ground surface, thus helping prevent accidental utility strikes; an AR post-disaster reconnaissance framework that enables building inspectors to rapidly evaluate and quantify structural damage sustained by buildings in seismic events such as earthquakes or blasts; and a tabletop collaborative AR visualization framework that allows multiple users to observe and interact with visual simulations of engineering processes.

In Mixed Reality scenarios, background replacement is a common way to immerse a user in a synthetic environment. Properly identifying the background pixels in an image or video is a dif- ficult problem known as matting. Proper alpha mattes usually come from human guidance, special hardware setups, or color dependent algorithms. This is a consequence of the under-constrained nature of the per pixel alpha blending equation. In constant color matting, research identifies and replaces a background that is a single color, known as the chroma key color. Unfortunately, the algorithms force a controlled physical environment and favor constant, uniform lighting. More generic approaches, such as natural image matting, have made progress finding alpha matte solutions in environments with naturally occurring backgrounds. However, even for the quicker algorithms, the generation of trimaps, indicating regions of known foreground and background pixels, normally requires human interaction or offline computation. This research addresses ways to automatically solve an alpha matte for an image in realtime, and by extension a video, using a consumer level GPU. It does so even in the context of noisy environments that result in less reliable constraints than found in controlled settings. To attack these challenges, we are particularly interested in automatically generating trimaps from depth buffers for dynamic scenes so that algorithms requiring more dense constraints may be used. The resulting computation is parallelizable so that it may run on a GPU and should work for natural images as well as chroma key backgrounds. Extra input may be required, but when this occurs, commodity hardware available in most Mixed Reality setups should be able to provide the input. This allows us to provide real-time alpha mattes for Mixed Reality scenarios that take place in relatively controlled environments. As a consequence, while monochromatic backdrops (such as green screens or retro-reflective material) aid the algorithm’s accuracy, they are not an explicit requirement. iii Finally we explore a sub-image based approach to parallelize an existing hierarchical approach on high resolution imagery. We show that locality can be exploited to significantly reduce the memory and compute requirements of previously necessary when computing alpha mattes of high resolution images. We achieve this using a parallelizable scheme that is both independent of the matting algorithm and image features. Combined, these research topics provide a basis for Mixed Reality scenarios using real-time natural image matting on high definition video sources.

iiAbstractAn Augmented Reality (AR) application combines the virtual world and reality to enhance the user‟s perception of the real world. Earlier, augmented reality technology had not found much commercial success because it required bulky gadgets and considerable processing power. But now AR technologies are available and computationally feasible on commodity mobile devices, such as smartphones.Pose estimation AR applications make use of inbuilt hardware features like the accelerometer, magnetometer and Global Positioning System (GPS) for overlaying images onto the real world. However, the dependence of the working of AR on hardware features gives rise to new challenges, which must be addressed in order to provide augmentation in as seamless a manner as possible. These challenges include real time estimation of the user‟s viewport, geo-magnetic sensor noise, the delay in receiving GPS data, and the synchronization of timer threads. Other challenges include those of debugging and testing because realistic GPS feeds and other sensory data are unavailable during development.Next, most AR applications have the need for common framework-based services, such as tracking, image registration and interface handling. This thesis also explores the feasibility of such a framework, where a reusable platform is coupled with a customizable application specific component to create the AR application.iiiFinally, this thesis also proposes an architecture for an AR application that lives and evolves inside a cyber-infrastructure, in order to support a wider range of users through the development of a wider range of AR applications.

A consumer study was administered in order to test the effects of utilizing augmented reality in online shopping for an apparel item. A theoretical model was developed as a framework for the analysis and included telepresence, attitude, and product knowledge as independent variables, and purchase intent as the dependent variable. The results of the analysis suggested that only attitude predicted purchase intent. Attitude was affected directly by the stimulus, telepresence, and product knowledge. This implied that augmented reality has the potential to positively impact purchase intent, if and only if it can positively impact attitude.

Computer technology helps improve human productivity and performance. We believe that if the computer technology lies, most people are unlikely to notice the deception. We explored whether Augmented Reality can improve a user’s performance, whether an Augmented Reality system will gain human’s trust easily, and whether, when their trust is betrayed, the user is unlikely to notice. To evaluate our three hypotheses, we designed a card sorting experiment involving human subjects with augmented reality (AR) technology. Our experimental results support our hypotheses, with high levels of statistical significance.

Conventional head-mounted projection displays (HMPDs) contain of a pair of miniature projection lenses, beamsplitters, and miniature displays mounted on the helmet, as well as a retro-reflective screen placed strategically in the environment. We have extened the HMPD technology integrating the screen into a fully mobile embodiment. Some initial efforts of demonstrating this technology has been captured followed by an investigation of the diffraction effects versus image degradation caused by integrating the retro-reflective screen within the HMPD. The key contribution of this research is the conception and development of a mobileHMPD (M-HMPD). We have included an extensive analysis of macro- and microscopic properties that encompass the retro-reflective screen. Furthermore, an evaluation of the overall performance of the optics will be assessed in both object space for the optical designer and visual space for the possible users of this technology. This research effort will also be focused on conceiving a mobile M-HMPD aimed for dual indoor/outdoor applications. The M-HMPD shares the known advantage such as ultralightweight optics (i.e. 8g per eye), unperceptible distortion (i.e. ≤ 2.5%), and lightweight headset (i.e. ≤ 2.5 lbs) compared with eyepiece type head-mounted displays (HMDs) of equal eye relief and field of view. In addition, the M-HMPD also presents an advantage over the preexisting HMPD in that it does not require a retro-reflective screen placed strategically in the environment. This newly developed M-HMPD has the ability to project clear images at three different locations within near- or far-field observation depths without loss of image quality. This particular M-HMPD embodiment was targeted to mixed reality, augmented reality, and wearable display applications.

New computer graphics tools are developed for use in the automotive paint design and paint application industries. These tools are designed to aid in every aspect of automotive painting from initial paint design all the way to viewing a final spray paint job. This thesis also provides new computer graphics techniques by leveraging industrial appearance standards and measurement instruments to yield new interaction, rendering and display algorithms. First, a system is introduced for the simulation of spray painting. Head mounted display goggles are combined with a tracking system to allow users to paint a virtual surface with a spray gun. Ray tracing is used to simulate droplets landing on the surface of the object, allowing arbitrary shapes and spray gun patterns to be used. This system is combined with previous research on spray gun characteristics to provide a realistic simulation of the spray paint including the effects of viscosity, air pressure, and paint pressure. Experiments were performed to validate the system for use as a training tool. Next, a virtual airbrush tool is introduced. The basic particle simulation used in the spray paint system is modified to include the finer control needed for airbrushing. Paint mixing between colors applied to the surface is modeled using Kubelka-Munk theory. Computerized stencils, including semi-permeable stencils, can be manually positioned by the artist or projected onto the object’s surface. The resulting airbrush system can be used by airbrush artists to both practice their craft as well as to decorate virtual models. The dissertation then utilizes an industrial measurement instrument to simulate the surface roughness in automotive paint finishes. This simulation is integrated with a design interface to enable industry professionals to devise new paints that have detailed surface effects. The industrial measurement device can also be used to rapidly measure and render real world materials without the need for elaborate full BRDF acquisition tools. Finally, the surface model developed in this research can be used to study human detection of small scale surface roughness. Lastly, new projection systems are introduced to display the paints generated by the previous algorithms. A desired paint is projected onto the surface of an object, making that object appear to be painted with that material. This allows painters to evaluate how the final painted surface will look in a very natural way. A novel projection system is also described in which the user can hold and evaluate a flexible sample of virtual paint. This provides a more compelling display of the paint than a standard 2D monitor can generate. These display methods can also be used to display materials other than paint, potentially benefiting a large range of industries. The combination of these tools is intended to allow spray painters to design paints, paint them onto an automobile or other object, and verify exactly what the result will look like before ever having to manufacture the paint itself. The use of these tools could therefore reduce wasted paint, speed up training times, and allow for precise design of automobile appearance: all before ever manufacturing any real paint. In addition to aiding paint industries, these tools enhance the current state of the art in computer graphics. The airbrush tool provides a new texture creation system that existing airbrush artists can use with little to no training; the surface roughness simulation provides enhanced rendering of automotive paint that could be used in movies and games; and the projection display system improves the state of the art in augmented reality systems.

In the coming years, augmented reality, mobile computing, and related technologies have the potential to completely redefine how we interact with and use computers. No longer will we be bound to desktops and laptops, nor will we be bound to monitors, two-dimensional (2D) screens, and graphical user interface (GUI) backgrounds. Instead we will employ wearable systems to move about and augmented reality displays to overlay 2D and three-dimensional (3D) graphics onto the real world. When the computer graphics and user interface communities evolved from text-based user interfaces to 2D GUIs, many in the field noted the need for “new analyses and metrics“ [Shneiderman et al., 1995]; the same is equally true today as we shift from 2D GUI-based user interfaces and environments, to 3D, stereoscopic virtual (VR) and augmented reality (AR) environments. As we rush to advance the state of technology of AR and its capabilities, we need to advance the processes by which these environments are designed, built, and evaluated. Along these lines, this dissertation provides insight into the processes and products of AR usability evaluation. Despite the fact that this technology fundamentally changes the way we visualize, use, and interact with information, very little HCI work in general, and user-centered design and evaluation in particular, have been done to date specifically in AR [Swan & Gabbard, 2005]. While traditional HCI methods can be successfully applied in AR to determine what information should be presented to the user [Gabbard, 2002], these approaches do not tell us, and what, to date, has not been researched, is how information should be presented to the user. A difficulty in producing effective AR user interfaces (UIs) in outdoor AR settings lies in the wide range of environmental conditions that may be present, and specifically large-scale fluctuations in natural lighting and wide variations in likely backgrounds or objects in the scene. In many cases, a carefully designed AR user interface may be easily legible under some lighting and background conditions, and minutes later be totally illegible in others. Since lighting and background conditions may vary from minute to minute in dynamic AR usage contexts, there is a need for basic research to understand the relationship between real-world backgrounds and objects and associated augmenting text drawing styles. This research identifies characteristics of AR text drawing styles that affect legibility on common real-world backgrounds. We present the concept of active text drawing styles that adapt in real-time to changes in the real-world backgrounds. We also present lessons learned on applying traditional usability engineering techniques to outdoor AR application development and propose a modified usability engineering process to support user interface design of novel technologies such as AR. Results of this research provide the following scientific contributions to the field of AR: Empirical evidence regarding effectiveness of various text drawing styles in affording legibility to outdoor AR users. Empirical evidence that real-world backgrounds have an effect on the legibility of text drawing styles. Guidelines to aid AR user interface designers in choosing among various text drawing styles and characteristics of drawing styles produced by the pilot and user-based studies described in this dissertation. Candidate drawing style algorithms to support an active, real-time, AR display system, where sensors interpret real-world backgrounds to determine appropriate values for display drawing style characteristics. Lessons learned on applying traditional usability engineering processes to outdoor AR. A modified usability engineering process to assist developers in identifying effective UI designs vis-à-vis user-based studies.

Construction processes can be conceived as systems of discrete, interdependent activities. Discrete Event Simulation (DES) has thus evolved as an effective tool to model operations that compete over available resources (personnel, material, and equipment). A DES model has to be verified and validated to ensure that it reflects a modeler’s intentions, and faithfully represents a real operation. 3D visualization is an effective means of achieving this, and facilitating the process of communicating and accrediting simulation results. Visualization of simulated operations has traditionally been achieved in Virtual Reality (VR). In order to create convincing VR animations, detailed information about an operation and the environment has to be obtained. The data must describe the simulated processes, and provide 3D CAD models of project resources, the facility under construction, and the surrounding terrain (Model Engineering). As the size and complexity of an operation increase, such data collection becomes an arduous, impractical, and often impossible task. This directly translates into loss of financial and human resources that could otherwise be productively used. In an effort to remedy this situation, this dissertation proposes an alternate approach of visualizing simulated operations using Augmented Reality (AR) to create mixed views of real existing jobsite facilities and virtual CAD models of construction resources. The application of AR in animating simulated operations has significant potential in reducing the aforementioned Model Engineering and data collection tasks, and at the same time can help in creating visually convincing output that can be effectively communicated. This dissertation presents the design, methodology, and development of ARVISCOPE, a general purpose AR animation authoring language, and ROVER, a mobile computing hardware framework. When used together, ARVISCOPE and ROVER can create three-dimensional AR animations of any length and complexity from the results of running DES models of engineering operations. ARVISCOPE takes advantage of advanced Global Positioning System (GPS) and orientation tracking technologies to accurately track a user’s spatial context, and georeferences superimposed 3D graphics in an augmented environment. In achieving the research objectives, major technical challenges such as accurate registration, automated occlusion handling, and dynamic scene construction and manipulation have been successfully identified and addressed.

Virtual Reality (VR) and Augmented Reality (AR) Head Mounted Display (HMD) or Head Worn Display (HWD) technology represents low-cost, wide Field of Regard (FOR), deployable systems when compared to traditional simulation facilities. However, given current technological limitations, HWD flight simulator implementations provide a limited effective Field of View (eFOV) far narrower than the normal human 200[degrees] horizontal and 135[degrees] vertical FOV. Developing a HWD with such a wide FOV is expensive but can increase the aviator's visual stimulus, perception, sense of presence and overall training effectiveness. This research and experimentation test this proposition by manipulating the eFOV of experienced pilots in a flight simulator while measuring their reflexive motor response and task performance. Reflexive motor responses are categorized as information, importance and effort behaviors. Performance metrics taken include runway alignment error (RAE) and vertical track error (VTE). Results indicated a significant and systematic change in visual scan pattern, head movement and flight control performance as the eFOV was sequentially decreased. As FOV decreased, the average visual scan pattern changed to focus less on out-the-window (OTW) and more on the instruments inside the cockpit. The head range of movement significantly increased below 80[degrees] horizontal x 54[degrees] vertical eFOV as well as significantly decreasing runway alignment and vertical track performance, which occurred below 120[degrees] horizontal x 81[degrees] vertical eFOV.

For successful collaboration to occur, a workspace must support inter-referential awareness - or the ability for one participant to refer to a set of artifacts in the environment, and for that reference to be correctly interpreted by others. While referring to objects in our everyday environment is a straight-forward task, the non-tangible nature of digital artifacts presents us with new interaction challenges. Augmented reality (AR) is inextricably linked to the physical world, and it is natural to believe that the re-integration of physical artifacts into the workspace makes referencing tasks easier; however, we find that these environments combine the referencing challenges from several computing disciplines, which compound across scenarios. This dissertation presents our studies of this form of awareness in collaborative AR environments. It stems from our research in developing mixed reality environments for molecular modeling, where we explored spatial and multi-modal referencing techniques. To encapsulate the myriad of factors found in collaborative AR, we present a generic, theoretical framework and apply it to analyze this domain. Because referencing is a very human-centric activity, we present the results of an exploratory study which examines the behaviors of participants and how they generate references to physical and virtual content in co-located and remote scenarios; we found that participants refer to content using physical and virtual techniques, and that shared video is highly effective in disambiguating references in remote environments. By implementing user feedback from this study, a follow-up study explores how the environment can passively support referencing, where we discovered the role that virtual referencing plays during collaboration. A third study was conducted in order to better understand the effectiveness of giving and interpreting references using a virtual pointer; the results suggest the need for participants to be parallel with the arrow vector (strengthening the argument for shared viewpoints), as well as the importance of shadows in non-stereoscopic environments. Our contributions include a framework for analyzing the domain of inter-referential awareness, the development of novel referencing techniques, the presentation and analysis of our findings from multiple user studies, and a set of guidelines to help designers support this form of awareness.

Robots have reached a level of sophistication such that it is now productive to include robot-assisted search teams in urban search and rescue scenarios. The perceptual interface to the human operator is frequently a video display that relays images from a camera on the robot. Research into issues surrounding human-robot interaction has suggested that if more intelligence were placed on the robot, it could lessen the load on the human operator and improve performance in the search task. In this thesis we examine the relative performance in a search task where the robot is controlled by a remote human operator under different simulated environmental conditions. Three distinct environmental conditions are represented by: a nominal display from the robot's camera, an artificially degraded display (representing signal interference) of the same information, and an augmented display of the degraded information, in which object recognition algorithms enable the application of a visual overlay to identify a search target. Results show that reduced performance under the degraded condition can be improved to near the nominal level by this augmentation method.

Advances in simulation technology have brought about many improvements to the way we train tasks, as well as how we perform tasks in the operational field. Augmented reality (AR) is an example of how to enhance the user's experience in the real world with computer generated information and graphics. Visual search tasks are known to be capacity demanding and therefore may be improved by training in an AR environment. During the experimental task, participants searched for enemies (while cued from visual, auditory, tactile, combinations of two, or all three modality cues) and tried to shoot them while avoiding shooting the civilians (fratricide) for two 2-minute low-workload scenarios, and two 2-minute high-workload scenarios. The results showed significant benefits of attentional cuing on visual search task performance as revealed by benefits in reaction time and accuracy from the presence of the haptic cues and auditory cues when displayed alone and the combination of the visual and haptic cues together. Fratricide occurrence was shown to be amplified by the presence of the audio cues. The two levels of workload produced differences within individual's task performance for accuracy and reaction time. Accuracy and reaction time were significantly better with the medium cues than all the others and the control condition during low workload and marginally better during high workload. Cue specificity resulted in a non-linear function in terms of performance in the low workload condition. These results are in support of Posner's (1978) theory that, in general, cueing can benefit locating targets in the environment by aligning the attentional system with the visual input pathways. The cue modality does not have to match the target modality. This research is relevant to potential applications of AR technology. Furthermore, the results identify and describe perceptual and/or cognitive issues with the use of displaying computer generated augmented objects and information overlaid upon the real world. The results also serve as a basis for providing a variety of training and design recommendations to direct attention during military operations. Such recommendations include cueing the Soldier to the location of hazards, and mitigating the effects of stress and workload.

The need for real-time video analysis is rapidly increasing in today's world. The decreasing cost of powerful processors and the proliferation of affordable cameras, combined with needs for security, methods for searching the growing collection of video data, and an appetite for high-tech entertainment, have produced an environment where video processing is utilized for a wide variety of applications. Tracking is an element in many of these applications, for purposes like detecting anomalous behavior, classifying video clips, and measuring athletic performance. In this dissertation we focus on augmented reality, but the methods and conclusions are applicable to a wide variety of other areas. In particular, our work deals with achieving real-time performance while tracking with augmented reality systems using a minimum set of commercial hardware. We have built prototypes that use both existing technologies and new algorithms we have developed. While performance improvements would be possible with additional hardware, such as multiple cameras or parallel processors, we have concentrated on getting the most performance with the least equipment. Tracking is a broad research area, but an essential component of an augmented reality system. Tracking of some sort is needed to determine the location of scene augmentation. First, we investigated the effects of illumination on the pixel values recorded by a color video camera. We used the results to track a simple solid-colored object in our first augmented reality application. Our second augmented reality application tracks complex non-rigid objects, namely human faces. In the color experiment, we studied the effects of illumination on the color values recorded by a real camera. Human perception is important for many applications, but our focus is on the RGB values available to tracking algorithms. Since the lighting in most environments where video monitoring is done is close to white, (e.g., fluorescent lights in an office, incandescent lights in a home, or direct and indirect sunlight outside,) we looked at the response to "white" light sources as the intensity varied. The red, green, and blue values recorded by the camera can be converted to a number of other color spaces which have been shown to be invariant to various lighting conditions, including view angle, light angle, light intensity, or light color, using models of the physical properties of reflection. Our experiments show how well these derived quantities actually remained constant with real materials, real lights, and real cameras, while still retaining the ability to discriminate between different colors. This color experiment enabled us to find color spaces that were more invariant to changes in illumination intensity than the ones traditionally used. The first augmented reality application tracks a solid colored rectangle and replaces the rectangle with an image, so it appears that the subject is holding a picture instead. Tracking this simple shape is both easy and hard; easy because of the single color and the shape that can be represented by four points or four lines, and hard because there are fewer features available and the color is affected by illumination changes. Many algorithms for tracking fixed shapes do not run in real time or require rich feature sets. We have created a tracking method for simple solid colored objects that uses color and edge information and is fast enough for real-time operation. We also demonstrate a fast deinterlacing method to avoid "tearing" of fast moving edges when recorded by an interlaced camera, and optimization techniques that usually achieved a speedup of about 10 from an implementation that already used optimized image processing library routines. Human faces are complex objects that differ between individuals and undergo non-rigid transformations. Our second augmented reality application detects faces, determines their initial pose, and then tracks changes in real time. The results are displayed as virtual objects overlaid on the real video image. We used existing algorithms for motion detection and face detection. We present a novel method for determining the initial face pose in real time using symmetry. Our face tracking uses existing point tracking methods as well as extensions to Active Appearance Models (AAMs). We also give a new method for integrating detection and tracking data and leveraging the temporal coherence in video data to mitigate the false positive detections. While many face tracking applications assume exactly one face is in the image, our techniques can handle any number of faces. The color experiment along with the two augmented reality applications provide improvements in understanding the effects of illumination intensity changes on recorded colors, as well as better real-time methods for detection and tracking of solid shapes and human faces for augmented reality. These techniques can be applied to other real-time video analysis tasks, such as surveillance and video analysis.

Augmented reality "AR" is a promising paradigm that can provide users with real-time, high-quality visualization of a wide variety of information. In AR, virtual objects are added to the real-world view in a real time. Using the AR technology can offer a very realistic environment for driving enhancement as well as driving performance testing under different scenarios. This can be achieved by adding virtual objects (people, vehicles, hazards, and other objects) to the normal view while driving in a safe controlled environment. In this dissertation, the feasibility of adapting the AR technology into traffic engineering was investigated. Two AR systems; AR Vehicle "ARV" system and Offline AR Simulator "OARSim" system were built. The systems' outcomes as well as the on-the-road driving under the AR were evaluated. In evaluating systems' outcomes, systems were successfully able to duplicate real scenes and generate new scenes without any visual inconsistency. In evaluating on-the-road driving under the AR, drivers' distance judgment, speed judgment, and level of comfort while driving were evaluated. In addition, our systems were used to conduct two traffic engineering studies; left-turn maneuver at un-signalized intersection, and horizontal visibility blockage when following a light truck vehicle. The results from this work supported the validity of our AR systems to be used as a surrogate to the field-testing for transportation research.

Medical simulation has facilitated the understanding of complex biological phenomenon through its inherent explanatory power. It is a critical component for planning clinical interventions and analyzing its effect on a human subject. The success of medical simulation is evidenced by the fact that over one third of all medical schools in the United States augment their teaching curricula using patient simulators. Medical simulators present combat medics and emergency providers with video-based descriptions of patient symptoms along with step-by-step instructions on clinical procedures that alleviate the patient's condition. Recent advances in clinical imaging technology have led to an effective medical visualization by coupling medical simulations with patient-specific anatomical models and their physically and physiologically realistic organ deformation. 3D physically-based deformable lung models obtained from a human subject are tools for representing regional lung structure and function analysis. Static imaging techniques such as Magnetic Resonance Imaging (MRI), Chest x-rays, and Computed Tomography (CT) are conventionally used to estimate the extent of pulmonary disease and to establish available courses for clinical intervention. The predictive accuracy and evaluative strength of the static imaging techniques may be augmented by improved computer technologies and graphical rendering techniques that can transform these static images into dynamic representations of subject specific organ deformations. By creating physically based 3D simulation and visualization, 3D deformable models obtained from subject-specific lung images will better represent lung structure and function. Variations in overall lung deformations may indicate tissue pathologies, thus 3D visualization of functioning lungs may also provide a visual tool to current diagnostic methods. The feasibility of medical visualization using static 3D lungs as an effective tool for endotracheal intubation was previously shown using Augmented Reality (AR) based techniques in one of the several research efforts at the Optical Diagnostics and Applications Laboratory (ODALAB). This research effort also shed light on the potential usage of coupling such medical visualization with dynamic 3D lungs. The purpose of this dissertation is to develop 3D deformable lung models, which are developed from subject-specific high resolution CT data and can be visualized using the AR based environment. A review of the literature illustrates that the techniques for modeling real-time 3D lung dynamics can be roughly grouped into two categories: Geometrically-based and Physically-based. Additional classifications would include considering a 3D lung model as either a volumetric or surface model, modeling the lungs as either a single-compartment or a multi-compartment, modeling either the air-blood interaction or the air-blood-tissue interaction, and considering either a normal or pathophysical behavior of lungs. Validating the simulated lung dynamics is a complex problem and has been previously approached by tracking a set of landmarks on the CT images. An area that needs to be explored is the relationship between the choice of the deformation method for the 3D lung dynamics and its visualization framework. Constraints on the choice of the deformation method and the 3D model resolution arise from the visualization framework. Such constraints of our interest are the real-time requirement and the level of interaction required with the 3D lung models. The work presented here discusses a framework that facilitates a physics-based and physiology-based deformation of a single-compartment surface lung model that maintains the frame-rate requirements of the visualization system. The framework presented here is part of several research efforts at ODALab for developing an AR based medical visualization framework. The framework consists of 3 components, (i) modeling the Pressure-Volume (PV) relation, (ii) modeling the lung deformation using a Green's function based deformation operator, and (iii) optimizing the deformation using state-of-art Graphics Processing Units (GPU). The validation of the results obtained in the first two modeling steps is also discussed for normal human subjects. Disease states such as Pneumothorax and lung tumors are modeled using the proposed deformation method. Additionally, a method to synchronize the instantiations of the deformation across a network is also discussed.

The Army needs a distributed training environment that can be accessed whenever and wherever required for training and mission rehearsal. This paper describes an exploratory experiment designed to investigate the effectiveness of a prototype of such a system in training a navigation task. A wearable computer, acoustic tracking system, and see-through head mounted display (HMD) were used to wirelessly track users' head position and orientation while presenting a graphic representation of their virtual surroundings, through which the user walked using natural movement. As previous studies have shown that virtual environments can be used to train navigation, the ability to add natural movement to a type of virtual environment may enhance that training, based on the proprioceptive feedback gained by walking through the environment. Sixty participants were randomly assigned to one of three conditions: route drawing on printed floor plan, rehearsal in the actual facility, and rehearsal in a mixed reality (MR) environment. Participants, divided equally between male and female in each group, studied verbal directions of route, then performed three rehearsals of the route, with those in the map condition drawing it onto three separate printed floor plans, those in the practice condition walking through the actual facility, and participants in the MR condition walking through a three dimensional virtual environment, with landmarks, waypoints and virtual footprints. A scaling factor was used, with each step in the MR environment equal to three steps in the real environment, with the MR environment also broken into "tiles", like pages in an atlas, through which participant progressed, entering each tile in succession until they completed the entire route. Transfer of training testing that consisted of a timed traversal of the route through the actual facility showed a significant difference in route knowledge based on the total time to complete the route, and the number of errors committed while doing so, with "walkers" performing better than participants in the paper map or MR condition, although the effect was weak. Survey knowledge showed little difference among the three rehearsal conditions. Three standardized tests of spatial abilities did not correlate with route traversal time, or errors, or with 3 of the 4 orientation localization tasks. Within the MR rehearsal condition there was a clear performance improvement over the three rehearsal trials as measured by the time required to complete the route in the MR environment which was accepted as an indication that learning occurred. As measured using the Simulator Sickness Questionnaire, there were no incidents of simulator sickness in the MR environment. Rehearsal in the actual facility was the most effective training condition; however, it is often not an acceptable form of rehearsal given an inaccessible or hostile environment. Performance between participants in the other two conditions were indistinguishable, pointing toward continued experimentation that should include the combined effect of paper map rehearsal with mixed reality, especially as it is likely to be the more realistic case for mission rehearsal, since there is no indication that maps should be eliminated. To walk through the environment beforehand can enhance the Soldiers' understanding of their surroundings, as was evident through the comments from participants as they moved from MR to the actual space: "This looks like I was just here", and "There's that pole I kept having trouble with". Such comments lead one to believe that this is a tool to continue to explore and apply. While additional research on the scaling and tiling factors is likely warranted, to determine if the effect can be applied to other environments or tasks, it should be pointed out that this is not a new task for most adults who have interacted with maps, where a scaling factor of 1 to 15,000 is common in orienteering maps, and 1 to 25,000 in military maps. Rehearsal time spent in the MR condition varied widely, some of which could be blamed on an issue referred to as "avatar excursions", a system anomaly that should be addressed in future research. The proprioceptive feedback in MR was expected to positively impact performance scores. It is very likely that proprioceptive feedback is what led to the lack of simulator sickness among these participants. The design of the HMD may have aided in the minimal reported symptoms as it allowed participants some peripheral vision that provided orientation cues as to their body position and movement. Future research might include a direct comparison between this MR, and a virtual environment system through which users move by manipulating an input device such as a mouse or joystick, while physically remaining stationary. The exploration and confirmation of the training capabilities of MR as is an important step in the development and application of the system to the U.S. Army training mission. This experiment was designed to examine one potential training area in a small controlled environment, which can be used as the foundation for experimentation with more complex tasks such as wayfinding through an urban environment, and or in direct comparison to more established virtual environments to determine strengths, as well as areas for improvement, to make MR as an effective addition to the Army training mission.

This thesis presents a new 3-dimensional (3D) user interface system for a Cave Automated Virtual Environment (CAVE) application, based on Virtual Reality (VR), Augmented Reality (AR), and Tangible User Interface (TUI). We explore fundamental 3D interaction tasks with our user interface for the CAVE system. User interface (UI) is comprised of a specific set of components, including input/output devices and interaction techniques. Our approach is based on TUIs using ARToolKit, which is currently the most popular toolkit for use in AR projects. Physical objects (props) are used as input devices instead of any tethered electromagnetic trackers. An off-the-shelf webcam is used to get tracking input data. A unique pattern marker is attached to the prop, which is easily and simply tracked by ARToolKit. Our interface system is developed on CAVE infrastructure, which is a semi-immersive environment. All virtual objects are directly manipulated with props, each of which corresponds to a certain virtual object. To navigate, the user can move the background itself, while virtual objects remain in place. The user can actually feel the prop's movement through the virtual space. Thus, fundamental 3D interaction tasks such as object selection, object manipulation, and navigation are performed with our interface. To feel immersion, the user is allowed to wear stereoscopic glasses with a head tracker. This is the only tethered device for our work. Since our interface is based on tangible input tools, seamless transition between one and two-handed operation is provided. We went through three design phases to achieve better task performance. In the first phase, we conducted the pilot study, focusing on the question whether or not this approach is applicable to 3D immersive environments. After the pilot study, we redesigned props and developed ARBox. ARBox is used for as interaction space while the CAVE system is only used for display space. In this phase, we also developed interaction techniques for fundamental 3D interaction tasks. Our summative user evaluation was conducted with ARDesk, which is redesigned after our formative user evaluation. Two user studies aim to get user feedback and to improve interaction techniques as well as interface tools' design. The results from our user studies show that our interface can be intuitively and naturally applied to 3D immersive environments even though there are still some issues with our system design. This thesis shows that effective interactions in a CAVE system can be generated using AR technique and tangible objects.

Augmented reality provides its user with additional information not available through the natural real-world environment. This additional information displayed to the user potentially poses a risk of perceptual and cognitive load and vision-based difficulties. The presence of real-world objects together with virtual augmenting information requires the user to repeatedly switch eye focus between the two in order to extract information from both environments. Switching eye focus may result in additional time on user tasks and lower task accuracy. Thus, one of the goals of this research was to understand the impact of switching eye focus between real-world and virtual information on user task performance. Secondly, focus depth, which is an important parameter and a depth cue, may affect the user's view of the augmented world. If focus depth is not adjusted properly, it may result in vision-based difficulties and reduce speed, accuracy, and comfort while using an augmented reality display. Thus, the second goal of this thesis was to study the effect of focus depth on task performance in augmented reality systems. In augmented reality environments, real-world and virtual information are found at different distances from the user. To focus at different depths, the user's eye needs to accommodate and converge, which may strain the eye and degrade performance on tasks. However, no research in augmented reality has explored this issue. Hence, the third goal of this thesis was to determine if distance of virtual information from the user impacts task performance. To accomplish these goals, a 3x3x3 within subjects design was used. The experimental task for the study required the user to repeatedly switch eye focus between the virtual text and real-world text. A monocular see-through head- mounted display was used for this research. Results of this study revealed that switching between real-world and virtual information in augmented reality is extremely difficult when information is displayed at optical infinity. Virtual information displayed at optical infinity may be unsuitable for tasks of the nature used in this research. There was no impact of focus depth on user task performance and hence it is preliminarily recommended that manufacturers of head-mounted displays may only need to make fixed focus depth displays; this clearly merits additional intensive research. Further, user task performance was better when focus depth, virtual information, and real-world information were all at the same distance from the user as compared to conditions when they were mismatched. Based on this result we recommend presenting virtual information at the same distance as real-world information of interest.

Advances in computer networks and rendering systems facilitate the creation of distributed collaborative environments in which the distribution of information at remote locations allows efficient communication. Particularly challenging are distributed interactive Virtual Environments (VE) that allow knowledge sharing through 3D information. In a distributed interactive VE the dynamic shared state represents the changing information that multiple machines must maintain about the shared virtual components. One of the challenges in such environments is maintaining a consistent view of the dynamic shared state in the presence of inevitable network latency and jitter. A consistent view of the shared scene will significantly increase the sense of presence among participants and facilitate their interactive collaboration. The purpose of this work is to address the problem of latency in distributed interactive VE and to develop a conceptual model for consistency maintenance in these environments based on the participant interaction model. A review of the literature illustrates that the techniques for consistency maintenance in distributed Virtual Reality (VR) environments can be roughly grouped into three categories: centralized information management, prediction through dead reckoning algorithms, and frequent state regeneration. Additional resource management methods can be applied across these techniques for shared state consistency improvement. Some of these techniques are related to the systems infrastructure, others are related to the human nature of the participants (e.g., human perceptual limitations, area of interest management, and visual and temporal perception). An area that needs to be explored is the relationship between the dynamic shared state and the interaction with the virtual entities present in the shared scene. Mixed Reality (MR) and VR environments must bring the human participant interaction into the loop through a wide range of electronic motion sensors, and haptic devices. Part of the work presented here defines a novel criterion for categorization of distributed interactive VE and introduces, as well as analyzes, an adaptive synchronization algorithm for consistency maintenance in such environments. As part of the work, a distributed interactive Augmented Reality (AR) testbed and the algorithm implementation details are presented. Currently the testbed is part of several research efforts at the Optical Diagnostics and Applications Laboratory including 3D visualization applications using custom built head-mounted displays (HMDs) with optical motion tracking and a medical training prototype for endotracheal intubation and medical prognostics. An objective method using quaternion calculus is applied for the algorithm assessment. In spite of significant network latency, results show that the dynamic shared state can be maintained consistent at multiple remotely located sites. In further consideration of the latency problems and in the light of the current trends in interactive distributed VE applications, we propose a hybrid distributed system architecture for sensor-based distributed VE that has the potential to improve the system real-time behavior and scalability.

A virtual environment is a set of surroundings that appears to exist to a user through sensory stimuli provided by a computer. By virtual environment, we mean to include environments supporting the full range from VR to pure reality. A necessity for virtual environments is knowledge of the location of objects in the environment. This is referred to as the tracking problem, which points to the need for accurate and precise tracking in virtual environments. Marker-based tracking is a technique which employs fiduciary marks to determine the pose of a tracked object. A collection of markers arranged in a rigid configuration is called a tracking probe. The performance of marker-based tracking systems depends upon the fidelity of the pose estimates provided by tracking probes. The realization that tracking performance is linked to probe performance necessitates investigation into the design of tracking probes for proponents of marker-based tracking. The challenges involved with probe design include prediction of the accuracy and precision of a tracking probe, the creation of arbitrarily-shaped tracking probes, and the assessment of the newly created probes. To address these issues, we present a pioneer framework for designing conformal tracking probes. Conformal in this work means to adapt to the shape of the tracked objects and to the environmental constraints. As part of the framework, the accuracy in position and orientation of a given probe may be predicted given the system noise. The framework is a methodology for designing tracking probes based upon performance goals and environmental constraints. After presenting the conformal tracking framework, the elements used for completing the steps of the framework are discussed. We start with the application of optimization methods for determining the probe geometry. Two overall methods for mapping markers on tracking probes are presented, the Intermediary Algorithm and the Viewpoints Algorithm. Next, we examine the method used for pose estimation and present a mathematical model of error propagation used for predicting probe performance in pose estimation. The model uses a first-order error propagation, perturbing the simulated marker locations with Gaussian noise. The marker locations with error are then traced through the pose estimation process and the effects of the noise are analyzed. Moreover, the effects of changing the probe size or the number of markers are discussed. Finally, the conformal tracking framework is validated experimentally. The assessment methods are divided into simulation and post-fabrication methods. Under simulation, we discuss testing of the performance of each probe design. Then, post-fabrication assessment is performed, including accuracy measurements in orientation and position. The framework is validated with four tracking probes. The first probe is a six-marker planar probe. The predicted accuracy of the probe was 0.06 deg and the measured accuracy was 0.083 plus/minus 0.015 deg. The second probe was a pair of concentric, planar tracking probes mounted together. The smaller probe had a predicted accuracy of 0.206 deg and a measured accuracy of 0.282 plus/minus 0.03 deg. The larger probe had a predicted accuracy of 0.039 deg and a measured accuracy of 0.017 plus/minus 0.02 deg. The third tracking probe was a semi-spherical head tracking probe. The predicted accuracy in orientation and position was 0.54 plus/minus 0.24 deg and 0.24 plus/minus 0.1 mm, respectively. The experimental accuracy in orientation and position was 0.60 plus/minus 0.03 deg and 0.225 plus/minus 0.05 mm, respectively. The last probe was an integrated, head-mounted display probe, created using the conformal design process. The predicted accuracy of this probe was 0.032 plus/minus 0.02 degrees in orientation and 0.14 plus/minus 0.08 mm in position. The measured accuracy of the probe was 0.028 plus/minus 0.01 degrees in orientation and 0.11 plus/minus 0.01 mm in position. These results constitute an order of magnitude improvement over current marker-based tracking probes in orientation, indicating the benefits of a conformal tracking approach. Also, this result translates to a predicted positional overlay error of a virtual object presented at 1m of less than 0.5 mm, which is well above reported overlay performance in virtual environments.

The use of systems that span many knowledge domains is becoming more common as technology advances, requiring expert-performance in a domain from users who are usually not experts in that domain. This study examined a means of communicating expertise (in system operation and fault diagnosis) to non-experts and furthering the understanding of expert mental models. It has been suggested that conceptions of abstract models of system-functions distinguish expert performance from non-expert performance (Hanisch, Kramer, and Hulin, 1991). This study examined the effects on performance of augmenting a simple control panel device with a model of the functions of the device, interacting with the model, and augmenting the device with graphically superimposed procedural indicators (directions). The five augmented display conditions studied were: Device Only, Device + Model, Device + Procedural Indicators, Interactive Model, and Interactive Model + Procedural Indicators. The device and displays were presented on a PC workstation. Performance measures (speed and accuracy) and subjective measures (questionnaires, NASA TLX, and structured interviews) were collected. It was expected that participants who interact with the device + procedural indicators would exhibit the shortest performance time and least errors; however, those who interacted with the simplest display (device only) were fastest and exhibited the least errors. Results of this study are discussed in terms of building a mental model and identifying situations that require a mental model.

Ensuring inspection performance is not a trivial design problem, because inspection is a complex and difficult task that tends to be error-prone, whether performed by human or by automated machines. Due to economical or technological reasons, human inspectors are responsible for inspection functions in many cases. Humans, however, are rarely perfect. A system of manual inspection was found to be approximately 80-90% effective, thus allowing non-confirming parts to be processed (Harris & Chaney, 1969; Drury, 1975). As the attributes of interest or the variety of products increases, the complexity of an inspection task increases. The inspection system becomes less effective because of the sensory and cognitive limitations of human inspectors. Any means that can support or aid the human inspectors is necessary to compensate for inspection difficulty. Augmented reality offers a new approach in designing an inspection system as a means to augment the cognitive capability of inspectors. To realize the potential benefits of AR, however the design of AR-aided inspection requires a through understanding of the inspection process as well as AR technology. The cognitive demands of inspection and the capabilities of AR to aid inspectors need to be evaluated to decide when and how to use AR for a dimensional inspection. The objectives of this study are to improve the performance of a dimensional inspection task by using AR and to develop guidelines in designing an AR-aided inspection system. The performance of four inspection methods (i.e., manual, 2D-aided, 3D-aided, and AR-aided inspections) was compared in terms of inspection time and measurement accuracy. The results suggest that AR might be an effective tool that reduces inspection time. However, the measuring accuracy was basically the same across all inspection methods. The questionnaire results showed that the AR and 3D-aided inspection conditions are preferred over the manual and 2D-aided inspection. Based on the results, four design guidelines were formed in using AR technology for a dimensional inspection.

The focus of this research was to examine how effectively augmented reality (AR) displays, generated with a wearable computer, could be used for aiding an operator performing a manufacturing assembly task. The research concentrated on comparing two technologies for generating augmented reality displays (opaque vs. see-through), with two current types of assembly instructions (a traditional assembly instruction manual vs. computer aided instruction). The study was used to evaluate the effectiveness of the wearable based augmented reality compared to traditional instruction methods, and was also used to compare two types of AR displays in the context of an assembly task. For the experiment, 15 subjects were asked to assemble a computer motherboard using the four types of instruction: paper manual, computer aided, an opaque AR display, and a see-through AR display. The study was run as a within subjects design, where subjects were randomly assigned the order of instruction media. For the AR conditions, the augmented environments were generated with a wearable computer, and viewed through two types of monocular, head-mounted displays (HMD). The first type of HMD was a monocular opaque HMD, and the second was a monocular see-though HMD. Prior to the experiment, all subjects performed a brief training session teaching them how to insert the various components of the motherboard in their respective slots. The time of assembly and assembly errors were measured for each type of media, and a questionnaire was administered to each subject at the end of each condition, and at the end of the experiment to determine the usability of the four instructional media. The results of the experiment indicated that both augmented reality conditions were more effective instructional aids for the assembly task than either the paper instruction manual or the computer aided instruction. The see-through HMD resulted in the fastest assembly times followed by the opaque HMD, the computer aided instruction, and the paper instructions respectively. In addition, subjects made fewer errors using the AR conditions compared to the other two types of instructional media. However, while the two AR conditions were a more effective instructional media when time was the response measure, there were still some important usability issues associated with the AR technology that were not present in the non-AR conditions. Many of the subjects indicated that both types of HMDs were uncomfortable, and over half expressed concerns about poor image contrast with the see-through HMDs. Finally, this thesis discusses the results of this study as well as implications for the design and use of AR and wearable computers for manufacturing assembly tasks.

Augmented reality, the superimposing of graphics or text onto an actual visual scene, is intended to enhance a user's understanding of the real world. This research examines the perceptual, cognitive, and human factors implications of combining integrally designed computer-generated imagery with real world scenes. Three experiments were conducted to test the theoretical and practical consequences of integral perception in augmented reality. The first experiment was a psychophysical study that had participants subjectively assess the integrality of 32 scenes comprising four different augmented reality object environments (computer, brain, text, and liquid dynamic model), projected at two transparency levels (opaque, and semi-transparent), and presented with four different graphic textures (color, grayscale, white, and wireframe). The second experiment expanded the psychophysical integrality assessment of augmented scenes to 32 different images composed of four new environments (housing development, computer lab, planetary photo, and trees in countryside), with multiple computer-generated graphics (two, four, six, and eight), at two levels of integrality as defined by experiment one (high, low). The third experiment was an applied study that had two phases: 1) learning tasks using three augmented environments; and, 2) assembly tasks using eight augmented video instructions. The computer-generated graphics for each phase of experiment three were presented at two levels of integrality (high, low) as defined by experiment one. The primary results of the three experiments show that augmented reality scenes with computer-generated imagery presented transparently and in color were perceived most integrally; increasing the number of graphics from two to eight decreased integral perception; and, high integral graphics aided performance in learning and real assembly tasks. From the statistical results and experimenter observation of the three experiments, guidelines for designing integrally perceived graphics in augmented environments were compiled based on principles of human factors, perception, and graphic design. The key themes of the design guidelines were: 1) maintaining true shape information in the computer-generated graphics 2) using highly realistic graphics for naturalistic augmented settings; 3) considering the hardware limitations of the augmented system, particularly the display; and, 4) designing appropriately for the task (simple, complex, hands-on, cognitive, dynamic, static, etc.).