Search

Augmented Reality (AR) is a key technology in the transition to Industry 4.0 and smart manufacturing, gaining reputation in a wide range of industrial fields One promising application is in another field of Industry 4.0, the data monitoring, where AR can be used to visualize and interact with complex and big data in real time, potentially improving the efficiency and accuracy of decision-making In this work, we propose a Pervasive AR tool for data monitoring, developed in collaboration with industry partners. An web application of the same data monitoring function is also created for comparison purposes A Human-Centered Design (HCD) methodology was used to identify the needs and requirements of industrial analysts, which led to the development of these systems Preliminary user studies, with 17 participants having distinct levels of expertise in industry, data analysis and computer science, were conducted to collect opinions and suggestions for improvements, as well as, quantitative data regarding the technologies considered A succeeding user study was then prepared, in which 12 participants used two conditions (C1 — Hands-free Pervasive AR tool for HMDs and C2 — Web tool for tablet devices) to fulfill a set of data monitoring tasks in an industry assembly line The result of these studies confirm the potential of Pervasive AR for data monitoring, as it engages users, promote environmental awareness, contextualizes data and allows fast self-localization. [Display omitted] • Two user studies with more than 10 participants each, industrial settings tasks. • Understand which method (Web or AR) stands out according to the task characteristic. • The findings reveal that certain methods were more effective for types of task. • Display methods significantly influenced participants across analysis. • These studies confirm the potential of Pervasive AR for data monitoring. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) is an emerging technology in neurosurgery with the potential to become a strategic tool in the delivery of care and education for trainees. Advances in technology have demonstrated promising use for improving visualization and spatial awareness of critical neuroanatomic structures. In this report, we employ a novel AR registration system for the visualization and targeting of skull landmarks. A markerless AR system was used to register 3-dimensional reconstructions of suture lines onto the head via a head-mounted display. Participants were required to identify craniometric points with and without AR assistance. Targeting error was measured as the Euclidian distance between the user-defined location and the true craniometric point on the subjects' heads. All participants successfully registered 3-dimensional reconstructions onto the subjects' heads. Targeting accuracy was significantly improved with AR (3.59 ± 1.29 mm). Across all target points, AR increased accuracy by an average of 19.96 ± 3.80 mm. Posttest surveys revealed that participants felt the technology increased their confidence in identifying landmarks (4.6/5) and that the technology will be useful for clinical care (4.2/5). While several areas of improvement and innovation can further enhance the use of AR in neurosurgery, this report demonstrates the feasibility of a markerless headset-based AR system for visualizing craniometric points on the skull. As the technology continues to advance, AR is expected to play an increasingly significant role in neurosurgery, transforming how surgeries are performed and improving patient care. [ABSTRACT FROM AUTHOR]

Most cultural promotion and dissemination are nowadays performed through the digitization of heritage sites and museums, a necessary requirement to meet the new needs of the public. Augmented Reality (AR), Mixed Reality (MR), and Virtual Reality (VR) have the potential to improve the experience quality and educational effect of these sites by stimulating users' senses in a more natural and vivid way. In this respect, head-mounted display (HMD) devices allow visitors to enhance the experience of cultural sites by digitizing information and integrating additional virtual cues about cultural artifacts, resulting in a more immersive experience that engages the visitor both physically and emotionally. This study contributes to the development and incorporation of AR, MR, and VR applications in the cultural heritage domain by providing an overview of relevant studies utilizing fully immersive systems, such as headsets and CAVE systems, emphasizing the advantages that they bring when compared to handheld devices. We propose a framework study to identify the key features of headset-based Extended Reality (XR) technologies used in the cultural heritage domain that boost immersion, sense of presence, and agency. Furthermore, we highlight core characteristics that favor the adoption of these systems over more traditional solutions (e.g., handheld devices), as well as unsolved issues that must be addressed to improve the guests' experience and the appreciation of the cultural heritage. An extensive search of Google Scholar, Scopus, IEEE Xplore, ACM Digital Library, and Wiley Online Library databases was conducted, including papers published from January 2018 to September 2022. To improve review reporting, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were used. Sixty-five papers met the inclusion criteria and were classified depending on the study's purpose: education, entertainment, edutainment, touristic guidance systems, accessibility, visitor profiling, and management. Immersive cultural heritage systems allow visitors to feel completely immersed and present in the virtual environment, providing a stimulating and educational cultural experience that can improve the quality and learning purposes of cultural visits. Nonetheless, the analyzed studies revealed some limitations that must be faced to give a further impulse to the adoption of these technologies in the cultural heritage domain. [ABSTRACT FROM AUTHOR]

This pilot study was undertaken to evaluate the safety, performance, and usability of the Xvision-Spine (XVS) System (Augmedics, Arlington Heights, IL) during open spinal fixation procedures in patients requiring pedicle screw placement in the lumbosacral spine. The XVS System is an augmented reality head-mounted display (HMD) based on a computer navigation system designed to assist surgeons in accurately placing pedicle screws. It uses an HMD-mounted tracking camera to provide optical tracking technology, and provides the surgeon a translucent direct near-eye display of the navigated surgical instrument's location relative to the computed tomographic image. We report the preliminary results of a prospective series of all consecutive patients who underwent augmented reality-assisted pedicle screw placement in the lumbosacral vertebrae at 3 institutions. Clinical accuracy for each pedicle screw was graded with Gertzbein-Robbins scores by 2 independent and blinded neuroradiologists. The 19 study participants included 8 men and 11 women with a mean age of 59.13 ± 12.09 and 59.91 ± 12.89 years, respectively. Seventeen procedures were successfully completed via the XVS System. Two procedures were not completed due to technical issues with the system's intraoperative scanner. A total of 86 screws were inserted. The accuracy of the XVS System was 97.7%. The XVS System's performance in accurate placement of pedicle screws in the lumbosacral vertebrae had an overall accuracy of 97.7%. These preliminary results were comparable to the accuracy of other manual computer-assisted navigation systems reported in the literature. [ABSTRACT FROM AUTHOR]

One of the main targets of criticism of head-mounted displays (HMDs) is the field of view (FOV) size, whether in virtual or augmented reality. This limitation is prominent with optical see-through head-mounted displays (OST-HMD), as those with narrow overlay FOV (OFOV) sizes only provide a small window to view virtual objects. We investigated if restricting this OFOV negatively affects a user's ability to memorize spatial locations in a simulation of a work environment, and consequently, long-term memory transfer to an equivalent scenario in the real world two days later. To find empirical evidence, we conducted a within-subjects experiment with 18 participants performing in three phases with an OST-HMD, simulated on an immersive HMD. For each phase, they viewed the training scenario with a different OFOV size of the augmentable area (30°, 70°, 110° diagonal). Results from recall tests showed that smaller OFOV size did not significantly affect user's performance on both short-term and transfer tests, but HMD data revealed that users rotated their heads less with a 110° OFOV. We also found that proximity of objects to memorize had an interaction effect with smaller OFOV sizes. Our findings could have implications on the design and HMD choices of augmented training. [Display omitted] • We investigate the effects of three overlay FOV (OFOV) sizes on spatial memorization. • We highlight the impact of OFOV and object proximity on memorization. • OFOV size did not affect spatial memorization, but wide OFOV reduced head rotation. • Our results imply that a larger OFOV may not necessarily benefit training outcomes. [ABSTRACT FROM AUTHOR]

• We propose an approach to compensate the visual defects caused by metamorphopsia • Our approach enables interactive measurement of distortion in user's visual field • We compensate the warped visual field through a real-time processing of video streams • We conducted an experiment on 17 patients affected by metamorphopsia • The results show the proposed system is able to reduce visual field distortion Advances in Augmented Reality technologies and, particularly, the availability of video see-through enabled head mounted displays (HMD), are allowing to devise new strategies to help individuals with visual impairments in daily life. In this work, an approach is proposed to compensate a serious visual impairment, known as metamorphopsia, a vision disorder characterized by deformed images. The goal is to provide patients with a digitally restored visual field, through real-time processing of video see-through streams captured from the HMD. To this regard, we present two contributions, respectively, an interactive discrete modeling of patient's eye-specific vision distortion and a compensation of the latter by means of corresponding real-time counter-distortion of incoming frames. Our approach, indeed, maps each of the video streams acquired by the stereoscopic video see-through cameras aboard the headset on a 2D polygonal mesh which is then counter-warped by moving its vertices based on the previously built distortion model and then displayed, restored, on the HMD's screen. First user evaluations report promising results along with usability issues related to HMD technology. [ABSTRACT FROM AUTHOR]

Augmented reality neuronavigation (ARN) systems can overlay three-dimensional anatomy and disease without the need for a two-dimensional external monitor. Accuracy is crucial for their clinical applicability. We performed a systematic review regarding the reported accuracy of ARN systems and compared them with the accuracy of conventional infrared neuronavigation (CIN). PubMed and Embase were searched for ARN and CIN systems. For ARN, type of system, method of patient-to-image registration, accuracy method, and accuracy of the system were noted. For CIN, navigation accuracy, expressed as target registration error (TRE), was noted. A meta-analysis was performed comparing the TRE of ARN and CIN systems. Thirty-five studies were included, 12 for ARN and 23 for CIN. ARN systems could be divided into head-mounted display and heads-up display. In ARN, 4 methods were encountered for patient-to-image registration, of which point-pair matching was the one most frequently used. Five methods for assessing accuracy were described. Ninety-four TRE measurements of ARN systems were compared with 9058 TRE measurements of CIN systems. Mean TRE was 2.5 mm (95% confidence interval, 0.7–4.4) for ARN systems and 2.6 mm (95% confidence interval, 2.1–3.1) for CIN systems. In ARN, there seems to be lack of agreement regarding the best method to assess accuracy. Nevertheless, ARN systems seem able to achieve an accuracy comparable to CIN systems. Future studies should be prospective and compare TREs, which should be measured in a standardized fashion. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) technology is a promising tool for field service applications such as assembly. AR can show computer-aided design (CAD) models of assembly parts at installation locations. This provides a means for delivering assembly instructions to an operator. Furthermore, evaluating the assembly process can be beneficial for preventing assembly errors. Field service engineers who perform assembly operations may choose to use a head-mounted display (HMD) for its good portability and hands-free operation. In this study, we propose a method for efficiently evaluating the misalignment between real and virtual objects, which is a mismatch between the two objects' poses, in an HMD-based AR assembly assistance system. By using a depth camera in the HMD, we evaluate the misalignment by comparing the depth maps of the real and virtual worlds. With our preliminary prototype, the system can evaluate misalignment with an accuracy of ±1 cm at a rate of 30 fps in real time. Moreover, we also propose a coordinate calibration method and establish a basic HMD-based AR assembly assistance system to demonstrate the methods. The efficient evaluation of misalignment could help to monitor operations to prevent assembly errors and decrease operation time. [ABSTRACT FROM AUTHOR]

• A new method of zoom projector design applying liquid lens is revealed. • An optical structure of zoom projector with excellent performance is proposed. • A complete prototype of adaptive micro-projection system is manufactured. Projectors play an important role in the fields of education, entertainment, etc. The conventional projectors are generally bulky, lack of portability and adaptability. In this paper, an adaptive micro-projection system based on liquid lens is proposed. The main part of the system consists of a sequential three-color light source module, a light-homogenized module, a piece of liquid–crystal-on-silicon (LCoS) with high refresh rate, a polarization beam splitter (PBS), a zoom projection lens based on liquid lenses, a liquid lens controller and a display driver board. By adjusting the driving voltages on the liquid lenses, the adaptive micro-projection system can work with the distance at least from 150 mm to 310 mm continuously. Moreover, by synchronously controlling the refresh of the LCoS and the sequential lighting of the three-color light source through the display driver board, a 60 Hz color video display can be realized with only one piece of LCoS. We introduce the design method of the liquid lens based zoom projection lens in detail and provide an optical structure of the liquid lens based zoom projection lens with excellent performance. The control software and hardware of the system are carefully developed. Simulations and experiments demonstrate the feasibility of the system. The proposed system has the characteristics of fast response, portability, integration and high adaptability and is expected to be used in virtual reality (VR), augmented reality (AR), on-board display, head-mounted display and so on. [ABSTRACT FROM AUTHOR]

Implementing augmented reality (AR) technology in the construction industry, particularly during the assembly stage of projects, presents a significant opportunity for accelerating the sector's digitalization process and enhancing labour productivity. Research, as the established literature shows, grows every year, however, further research is needed in order to achieve widespread adoption. This study aims to provide researchers with a systematic review of all the studies in the literature related to augmented reality technology's role in facilitating assembly adaptation within the construction industry. Utilizing the PRISMA method, a systematic review was conducted. Through thorough examinations, 42 articles directly relevant to the topic were identified, and the full texts consequently meticulously examined. The research suggests that a strong relationship between Building Information Modelling (BIM) technology and augmented reality concerning construction management issues exists. Additionally, the studies indicated that Head-Mounted Displays (HMDs) are frequently used as visualization tools, and the Unity game engine is commonly employed as software. • One of our aims in our study is to encourage scientists who can conduct research on the adaptation of augmented reality technology to the assembly phase of the construction industry. We aim to provide them with comprehensive information on the methods, software applications, and visualization tools utilized in previous studies. • Head Mountain Devices have been commonly used as visualization tools. Researchers predominantly choose the Unity game engine. Additionally, it is noteworthy that 25% of researchers in this field prefer to publish their work in the Automation in Construction journal. • In our study, the full texts of 40 articles were thoroughly examined using the PRISMA checklist. It was found that the number of studies in this field is increasing every year. [ABSTRACT FROM AUTHOR]

• Understanding the quality of VX with several VM based on different XR technologies. • Exploring relations between presence, VX and attitude toward experience. • Comparing VR, AR and MR technologies to determine best solution for virtual museum. Virtual museum systems, based on different X-reality technologies, has begun to spread, as they represent decisive tools to promote exhibitions and reaching out to audiences. Although budgetary considerations have so far limited the choice of technologies a wide range of possible technological options are available today at low cost. This paper provides the results of an empirical study, with the aim to determine the most appropriate technologies to satisfy the visitors' expectation and maximise their likelihood to repeat and recommend the experience. The study focuses on the comparison of the performance of five VM systems for visualise digital reproduction of archaeological finds, based on different technologies (i.e., PC desktop, holographic display, 3D stereoscopic projection, head mounted display and mobile Augmented Reality). The results provide useful insight for the development of VM systems, in order to maximize the visitor experience in terms of presence and ability to activate an experience economy perspective. [ABSTRACT FROM AUTHOR]

• Dark shadows can be represented even on optical see-through head-mounted displays. • Users actually perceive virtual shadows darker than without augmentations. • Virtual shadows can improve the photo-reality of even transparent objects. In this paper, we present two effects of illusion-based virtual shadows on a real surface for optical see-through head-mounted displays without devices that selectively reduce or intercept light rays from the real scene. The virtual shadows are based on the simultaneous brightness-contrast illusion in which a central region surrounded by a lighter one is perceived as darker than one without such a lighter region. To display a virtual shadow, a virtual object with a real image texture is placed around the shadow region as the lighter region to provide a sense of a darker area. We conducted two kinds of experiments to validate the feasibility of our method. The first experiment showed that users tend to perceive the virtual shadow part as darker than the surrounding area with the same physical luminance. The second experiment provided evidence that there are cases in which the virtual shadows can improve the reality of virtual objects. [ABSTRACT FROM AUTHOR]

• A gaze-based method that enables pure hands-free object positioning in a virtual world for head-mounted displays. • Our method can function with both pure head gaze and pure eye gaze. • Experimental results show that eye and head gaze-based control with our method can achieve 78% and 96% success rates, respectively, when compared to a hand-held controller. In virtual and augmented reality, gaze-based methods have been explored for decades as effective user interfaces for hands-free interaction. Though several well-known gaze-based methods exist for simple interactions such as selection, no solutions exist for 3D manipulation tasks requiring a higher degree of freedom (DoF). In this paper, we introduce OrthoGaze, a novel user interface that allows users to intuitively manipulate the three-dimensional position of a virtual object using only their eye or head gaze. Our approach makes use of three selectable, orthogonal planes, where each plane not only helps guide the user's gaze in an arbitrary virtual space, but also allows for 2-DoF manipulations of object position. To evaluate our method, we conducted two user studies involving aiming and docking tasks in virtual reality to evaluate the fundamental characteristics of sustained gaze aiming and to determine which type of gaze-based control performs best when combined with OrthoGaze. Results showed that eye gaze was more accurate than head gaze for sustained aiming. Additionally, eye and head gaze-based control for 3D manipulations achieved 78% and 96% performance, respectively, in comparison with a hand-held controller. Subjective results also suggest that gaze-based manipulation can comprehensively cause more fatigue than controller-based. From the experimental results, we expect OrthoGaze to become an effective method for pure hands-free object manipulation in head-mounted displays. [ABSTRACT FROM AUTHOR]

Augmented Reality Head-Mounted Displays (AR HMD) are considered a promising technology for assisting on-site construction tasks such as assembly. However, concerns about creating information overload and becoming a distraction to workers outweigh the potential benefits to productivity and task performance. Previous research investigated workers' mental workloads as impacted by AR-based displays for assembly tasks. Within that context, this paper proposes a Long Short-Term Memory (LSTM) based approach to predict mental workload when using AR HMD for construction assembly, to forecast users' cognitive status under such complex working condition. Thirty participants were recruited to finish a wood frame assembly using an AR HMD projecting a 3D conformal model on the workspace. The proposed method provided a reliable prediction of real-time electroencephalogram (EEG) signal and mental workload. The outcomes validated the feasibility of LSTM model for EEG-based mental workload prediction and specifically provided a usable method for evaluating AR HMD use in construction tasks. • Discussed the benefits of predicting mental workload while using AR-HMD in construction assembly tasks. • Conducted an AR-based assembly test with 30 subjects to explore mental workload oscillations and feasibility of prediction. • A deep learning approach was deployed to forecast the raw EEG data and predict subjects' mental workload. • The performance of proposed LSTM model was compared with other commonly used RNNs. [ABSTRACT FROM AUTHOR]

• Developed an augmented reality-based pedestrian simulation framework: AR-PED. • AR-PED enables actual pedestrian-in-the-loop simulation in 1:1-scale environment. • Extended capability of microscopic traffic simulation by involving real pedestrians. • Supported different types of users for realistically experiencing traffic simulation. Pedestrian simulators are in high demand for many research and industry applications, such as evaluating pedestrian safety and testing of autonomous vehicles. A recent trend on the development of pedestrian simulators is to use virtual reality (VR) head-mounted display (HMD) devices, since they offer a highly immersive and realistic view of the virtual environment (VE). However, existing pedestrian simulators that make use of this technology have a set of limitations that make them unsuitable for the simulation of a large virtual scenario. Specifically, the boundaries of the virtual scene set by these simulators encompass a relatively small area (e.g., a single crosswalk). In light of these issues, this paper aims to develop an augmented reality (AR) enabled pedestrian-in-the-loop simulation (AR-PED) framework that enables the simulation of pedestrians in a large virtual scenario (e.g., a city) with an actual scale. The proposed AR-PED simulation framework is empowered by the latest advances in cloud computing and AR to allow multiple users to access the simulation at the same time. A client-server architecture allows users to act as pedestrians, drive a car or simply visualize the scene simultaneously. Additionally, other vehicles can be artificially generated by a microscopic traffic simulation (MTS) package that acts as the server. A prototype of the proposed framework is implemented and demonstrated by simulating a model of a city. The promising capabilities and performance of the AR-PED framework are attested with the deployable experiments and potential improvements over the present study to further excel the current framework are also discussed. [ABSTRACT FROM AUTHOR]

To investigate how embodied sensorimotor interactions shape subjective visual experience, we developed a novel combination of Virtual Reality (VR) and Augmented Reality (AR) within an adapted breaking continuous flash suppression (bCFS) paradigm. In a first experiment, participants manipulated novel virtual 3D objects, viewed through a head-mounted display, using three interlocking cogs. This setup allowed us to manipulate the sensorimotor contingencies governing interactions with virtual objects, while characterising the effects on subjective visual experience by measuring breakthrough times from bCFS. We contrasted the effects of the congruency (veridical versus reversed sensorimotor coupling) and contingency (live versus replayed interactions) using a motion discrimination task. The results showed that the contingency but not congruency of sensorimotor coupling affected breakthrough times, with live interactions displaying faster breakthrough times. In a second experiment, we investigated how the contingency of sensorimotor interactions affected object category discrimination within a more naturalistic setting, using a motion tracker that allowed object interactions with increased degrees of freedom. We again found that breakthrough times were faster for live compared to replayed interactions (contingency effect). Together, these data demonstrate that bCFS breakthrough times for unfamiliar 3D virtual objects are modulated by the contingency of the dynamic causal coupling between actions and their visual consequences, in line with theories of perception that emphasise the influence of sensorimotor contingencies on visual experience. The combination of VR/AR and motion tracking technologies with bCFS provides a novel methodology extending the use of binocular suppression paradigms into more dynamic and realistic sensorimotor environments. [ABSTRACT FROM AUTHOR]

A novel solution for the large view angle holographic head-mounted display (HHMD) is presented. Divergent light is used for the hologram illumination to construct a large size three-dimensional object outside the display in a short distance. A designed project-type lens with large numerical aperture projects the object constructed by the hologram to its real location. The presented solution can realize a compact HHMD system with a large field of view. The basic principle and the structure of the system are described. An augmented reality (AR) prototype with the size of 50 mm×40 mm and the view angle above 60° is demonstrated. [ABSTRACT FROM AUTHOR]

Although the concept of Augmented Reality (AR) has already been proposed more than 20 years ago, most AR-applications are still limited to simple visualization of virtual objects onto spatially limited scenes. Ideas such as the “augmented reality manual”, showing step-by-step instructions to a user wearing a Head-Mounted Display (HMD), have been implemented, but the developed systems did not pass the barrier of demonstration prototypes. One major reason, beside remaining ergonomic and hardware limitations, consists of the large effort required for creating the content of such virtual instructions and for building models allowing accurate tracking. In this paper, we introduce the concept of Cognitive Augmented Reality, which radically revises existing approaches on AR-based assistance systems and proposes a fundamentally new paradigm exploiting prior visual observation and learning of a complete manipulative workflow. More precisely, we present in this paper a complete approach for creating augmented reality content for procedural tasks from video examples, and give then details about the presentation of such content at runtime. We show that this new approach, in spite of its very challenging aspects, is scalable, and valuable from a practicable point of view. [ABSTRACT FROM AUTHOR]

In augmented reality (AR), virtual information is optically combined with the physical environment. In the most frequently used combination technique, optical settings in AR depart from the settings in natural viewing. Depending on the combination of viewing distances of the virtual task and its physical background, this deviation may lower visual performance and cause visual disturbance symptoms. The so-called vergence-accommodation conflict (VAC) has been identified as a cause for the visual disturbance symptoms in AR. In this study, for various distance combinations, the performance and symptoms when performing a search task displayed in a see-through head-mounted display (AR HMD, HoloLens 1st generation, Microsoft, USA) was investigated. The search task was displayed at a virtual distance of either 200 cm or 30 cm, and the real background was viewed either at a distance of 200 cm or 30 cm. Three combinations of viewing distances for the background and the virtual task were studied: 200 cm/200 cm, 200 cm/30 cm, and 30 cm/30 cm. Results revealed that both performance and visual disturbance symptoms depend on the combination of the viewing distances of the physical background and the virtual task. When the physical background was viewed at a distance of 200 cm, younger participants showed a significantly better search performance and reported stronger symptoms compared with older participants, no matter whether the virtual task was performed at 30 cm or at 200 cm. However, with the physical background at a distance of 30 cm, the performance of the younger group dropped to the level of the performance of the older group, and younger participants tended to report a stronger increase in visual disturbance symptoms compared with the older participants. From the AR HMD technology used in this study, it can be concluded that a near viewing distance of the virtual task does not cause a negative impact on performance and visual disturbance symptoms, provided any physical background seen through the AR HMD is not at a near viewing distance. The findings indicate that the VAC, which persists in augmented and virtual reality, depends, in addition to the physical component evaluating the optical distance, on a cognitive component evaluating the perceived distance. AR settings should therefore also be evaluated in terms of possible effects on perceived distance. [ABSTRACT FROM AUTHOR]

The idea of humans and robots coexisting in manufacturing environments has increased the importance of personnel safety. Current practice to provide human safety in a conventional industrial robotic application is to put the robot in a cell enclosed by fences that occupies a large area in the environment. If someone enters this robotic cell, the robot ceases its movement immediately that leads to significant delays in the production cycle. In this paper, we present a new collaborative workspace concept for conventional industrial robots in order to eliminate fences and to avoid delays as much as possible. The proposed concept relies on a novel path adaptation algorithm and an augmented reality-based warning system. The algorithm, which makes the robot adaptable to human interventions, is unique in a sense that it can be applied to variety of industrial robots without requiring changes in the robot controller. Besides, the warning system provides additional safety features to the collaborative workspace through a projector and a head-mounted display. We demonstrate the feasibility of the proposed concept with an industrial robot in a realistic palletizing application. Several experiments were carried out to test the performance along with a user study. Performance test results indicate that the robot is able to perform three actions according to the minimum distance between the human and the robot in case of a human violation: slowdown, avoidance, and full stop. Qualitative performance tests results show that augmented reality improves the perception of safety and collaboration but negatively affects physical comfort. Overall, the proposed concept provides a practical and implementable solution for conventional industrial robots missing collaborative features. [ABSTRACT FROM AUTHOR]

This paper proposes a novel human-machine interface (HMI) for addressing the operational inefficiency problem owing to the asymmetrical workspace mapping in master-slave hydraulic manipulators. An augmented reality-based head-mounted display system combining binocular vision with real-time virtual modeling was designed for demonstrating the commanded motion, and a hidden-appearance visualization rule was established for reducing visual interference, caused by the occlusion effect. An auditory feedback interface was designed for providing an additional cue for the master-slave position error. Eighteen participants were required for executing two typical tasks for validation. The test results indicated that the task completion time was reduced using the proposed HMI, and the number of collisions with surrounding items was reduced. Although the subjective feeling evaluation showed no statistical improvement owing to the visual fatigue and limited definition of the augmented reality scene, the proposed HMI is a practical method for improving the operational efficiency of hydraulic manipulators. • A human-machine interface is proposed for a master-slave hydraulic manipulator. • It consists of a vision enhancement system and an auditory feedback interface. • A hidden-appearance visualization rule is designed to show the master's motion. • Auditory feedback is to give an additional cue of the master-slave position error. • The effectiveness is proven to improve the operational efficiency and safety level. [ABSTRACT FROM AUTHOR]

• We present a systematic review of optical see-through head mounted display (OST-HMD) usage in augmented reality surgery applications from 2013 to 2020. • 91 articles that fulfilled all inclusion criteria were categorized by OST- HMD device, surgical speciality, surgical application context, visualisation content, experimental design and evaluation, accuracy and human factors of human-computer interaction. • Human Factors emerge as significant to OST-HMD utility. • The significant upward trend in published articles is clear, but such devices are not yet established in the operating room and clinical studies showing benefit are lacking. • A focused effort addressing technical registration and perceptual factors in the lab coupled with design that incorporates human factors considerations to solve clear clinical problems should ensure that the significant current research efforts will succeed. [Display omitted] This article presents a systematic review of optical see-through head mounted display (OST-HMD) usage in augmented reality (AR) surgery applications from 2013 to 2020. Articles were categorised by: OST-HMD device, surgical speciality, surgical application context, visualisation content, experimental design and evaluation, accuracy and human factors of human-computer interaction. 91 articles fulfilled all inclusion criteria. Some clear trends emerge. The Microsoft HoloLens increasingly dominates the field, with orthopaedic surgery being the most popular application (28.6%). By far the most common surgical context is surgical guidance (n = 58) and segmented preoperative models dominate visualisation (n = 40). Experiments mainly involve phantoms (n = 43) or system setup (n = 21), with patient case studies ranking third (n = 19), reflecting the comparative infancy of the field. Experiments cover issues from registration to perception with very different accuracy results. Human factors emerge as significant to OST-HMD utility. Some factors are addressed by the systems proposed, such as attention shift away from the surgical site and mental mapping of 2D images to 3D patient anatomy. Other persistent human factors remain or are caused by OST-HMD solutions, including ease of use, comfort and spatial perception issues. The significant upward trend in published articles is clear, but such devices are not yet established in the operating room and clinical studies showing benefit are lacking. A focused effort addressing technical registration and perceptual factors in the lab coupled with design that incorporates human factors considerations to solve clear clinical problems should ensure that the significant current research efforts will succeed. [ABSTRACT FROM AUTHOR]

Digital Twins (DTs) have been developed for several pilot-scale membrane capacitive deionization (mCDI) units that are located in remote communities in China and Australia for desalination of brackish water and treated domestic wastewater. These pilot-scale mCDI units have a production capacity ranging from 5 to 50 m3/day and a water recovery rate of up to 85%. The mCDI DTs use Head-mounted Displays (HMDs) to facilitate the visualisation of transient real-time data and historical data from various sensors in the physical plants. The DTs contain device tag and sensor data display functions which greatly enhance the model functionality and user experience. By combining the DTs with Mixed Reality (MR) technology that blends elements of both Virtual Reality (VR) and Augmented Reality (AR), it was possible to use the DTs for remote control and remote operator training in an immersive environment. Our results suggest that more facile remote control and improved training outcomes could be achieved by use of DTs by the water industry compared to those achieved by conventional control and training methods. [Display omitted] • Digital Twins (DTs) of membrane capacitive deionization (mCDI) units developed. • The DTs blend elements of both Virtual Reality (VR) and Augmented Reality (AR). • Different communication approaches required depending on local infrastructure. • More facile remote control and improved training outcomes achieved by use of DTs. [ABSTRACT FROM AUTHOR]

Head Mounted Display (HMD) based Augmented Reality (AR) is being increasingly used in manufacturing and maintenance. However, limited research has been done to understand user interaction with AR interfaces, which may lead to poor usability, risk of occupational hazards, and low acceptance of AR systems. This paper uses a theoretically-driven approach to interaction design to investigate the impact of different AR modalities in terms of information mode (i.e. video vs. 3D animation) and interaction modality (i.e. hand-gesture vs. voice command) on user performance, workload, eye gaze behaviours, and usability during a maintenance assembly task. The results show that different information modes have distinct impacts compared to paper-based maintenance, in particular, 3D animation led to a 14% improvement over the video instructions in task completion time. Moreover, insights from eye gaze behaviours such as number of fixations and transition between Areas of Interest (AOIs) revealed the differences in attention switching and task comprehension difficulty with the choice of AR modalities. While, subjective user perceptions highlight some ergonomic issues such as misguidance and overreliance, which must be considered and addressed from the joint cognitive systems' (JCSs) perspective and in line with the predictions derived from the Multiple Resources Model. [ABSTRACT FROM AUTHOR]

Augmented Reality (AR) head-mounted displays (HMDs) provide users with an immersive virtual experience in the real world. The portability of this technology affords various information display options for construction workers that are not possible otherwise. However, the impact of these different information presentation options on human performance should be carefully evaluated before such technology is deployed in the jobsite. In this paper, we describe a research effort examining how different information displays presented via AR HMD influence task performance when assembling three sized wooden wall frame assembly tasks. We asked 18 construction engineering students with framing experience to finish three wood frame assembly tasks (large, medium, and small) using one of the three information displays (AR 3D conformal, AR 2D tag-along, and paper blueprints). The task performance was measured by time of completion and framing errors , which were analyzed and compared among each factor. [ABSTRACT FROM AUTHOR]

When looking for an object in a complex visual scene, Augmented Reality (AR) can assist search with visual cues persistently pointing in the target's direction. The effectiveness of these visual cues can be reduced if they are placed at a different visual depth plane to the target they are indicating. To overcome this visual-depth problem, we test the effectiveness of adding simultaneous spatialized auditory cues that are fixed at the target's location. In an experiment we manipulated which cue(s) were available (visual-only vs. visual + auditory), and which disparity plane relative to the target the visual cue was displayed on. Results show that participants were slower at finding targets when the visual cue was placed on a different disparity plane to the target. However, this slowdown in search performance could be substantially reduced with auditory cueing. These results demonstrate the importance of AR cross-modal cueing under conditions of visual uncertainty and show that designers should consider augmenting visual cues with auditory ones. [ABSTRACT FROM AUTHOR]

• We present a novel augmented reality system for head and neck carcinoma imaging. • We emphasize usability by adopting automatic registration and calibration. • Immersive visualization through a head mounted display improves perception of data. • Medical experts accepted our application in a real clinical scenario. Background and Objective: Augmented reality (AR) can help to overcome current limitations in computer assisted head and neck surgery by granting "X-ray vision" to physicians. Still, the acceptance of AR in clinical applications is limited by technical and clinical challenges. We aim to demonstrate the benefit of a marker-free, instant calibration AR system for head and neck cancer imaging, which we hypothesize to be acceptable and practical for clinical use. Methods: We implemented a novel AR system for visualization of medical image data registered with the head or face of the patient prior to intervention. Our system allows the localization of head and neck carcinoma in relation to the outer anatomy. Our system does not require markers or stationary infrastructure, provides instant calibration and allows 2D and 3D multi-modal visualization for head and neck surgery planning via an AR head-mounted display. We evaluated our system in a pre-clinical user study with eleven medical experts. Results: Medical experts rated our application with a system usability scale score of 74.8 ± 15.9, which signifies above average, good usability and clinical acceptance. An average of 12.7 ± 6.6 minutes of training time was needed by physicians, before they were able to navigate the application without assistance. Conclusions: Our AR system is characterized by a slim and easy setup, short training time and high usability and acceptance. Therefore, it presents a promising, novel tool for visualizing head and neck cancer imaging and pre-surgical localization of target structures. [ABSTRACT FROM AUTHOR]

• Augmented reality (AR) can improve manufacturing and control processes in industry. • Impact of AR on the user's visual system has been little investigated (contrary to 3D displays). • In the study subjects, performed assembly tasks with paper or AR instructions (Hololens). • Impact on binocular vision and fatigue were compared. • Results suggest that AR can be safely used by production operators. Many studies have shown the benefits of augmented reality (AR) to improve manufacturing and control processes in industry. However, the presentation of AR content through optical see-through head-mounted displays may induce unnatural viewing conditions, which consequences on the user's visual system have not been investigated yet. This study aimed at assessing whether using AR instructions to guide a manual task, i.e., conditions where the user is forced to repeatedly look at and accommodate in different planes to extract information from both real and virtual environments, could potentially impact the visual system of operators. A before/after design study was carried out, asking 26 participants to perform Lego assemblies for 30 min with either paper or AR instructions. The effects of using AR compared to paper instructions were evaluated both on binocular vision, with classical optometric measurements, and on visual fatigue, with the Virtual Reality Symptoms Questionnaire. No clinically significant differences in optometric measurements (far/near visual acuity, stereoacuity, phoria, convergence, fusional amplitude, accommodation amplitude, accommodative convergence) have been found between AR and paper instructions, and only negligible fatigue symptoms have been seen specifically for AR. Results from both objective and subjective measurements suggest that there is no impact of AR on the oculomotor system and that, in this specific case of use, AR can be safely used for production operators. [ABSTRACT FROM AUTHOR]

Superimposing virtual visual information within the real world is referred to as augmented reality (AR). This technique becomes increasingly popular and due to technical advances can be applied to various purposes ranging from consumer entertainment to industrial use cases. Workplaces equipped with AR assistance-systems promise fast and accurate performance. Within the field of intralogistics, palletization constitutes a field of application which may benefit from guidance by AR. In the present investigation, participants performed a palletization task while they received instructions via three different devices (paper list, tablet, AR glasses). Performance data and subjective ratings were obtained. Participants subjectively preferred to work with the AR device. Number of errors was reduced while at the same time handling times were prolonged when the AR device was used. In all experimental conditions, performance improved in the course of the experiment. Least improvement was observed in the AR condition, which emphasizes that such applications allow decent performance without prior experience. Overall, the present results indicate that the utility of AR applications is limited when task demands are low. Such applications might be more useful in unclear and complex situations. Furthermore, AR possesses high hedonic qualities which may ease the integration of novel work processes. [ABSTRACT FROM AUTHOR]

• Framework for interactive immersion into imaging data. • 3D/4D holographic visualization and interaction with a head mounted display. • Modular, expandable, scalable, and customizable framework. • Real-time segmentation of 4D MRI CINE with machine learning. Modern imaging scanners produce an ever-growing body of 3D/4D multimodal data requiring image analytics and visualization of fused images, segmentations, and information. For the latter, augmented reality (AR) with head-mounted displays (HMDs) has shown potential. This work describes a framework (FI3D) for interactive immersion with data, integration of image processing and analytics, and rendering and fusion with an AR interface. The FI3D was designed and endowed with modules to communicate with peripherals, including imaging scanners and HMDs, and to provide computational power for data acquisition and processing. The core of FI3D is deployed to a dedicated computational unit that performs the computationally demanding processes in real-time, and the HMD is used as a display output peripheral and an input peripheral through gestures and voice commands. FI3D offers user-made processing and analysis dedicated modules. Users can customize and optimize these for a particular workflow while incorporating current or future libraries. The FI3D framework was used to develop a workflow for processing, rendering, and visualization of CINE MRI cardiac sets. In this version, the data were loaded from a remote database, and the endocardium and epicardium of the left ventricle (LV) were segmented using a machine learning model and transmitted to a HoloLens HMD to be visualized in 4D. Performance results show that the system is capable of maintaining an image stream of one image per second with a resolution of 512 × 512. Also, it can modify visual properties of the holograms at 1 update per 16 milliseconds (62.5 Hz) while providing enough resources for the segmentation and surface reconstruction tasks without hindering the HMD. We provide a system design and framework to be used as a foundation for medical applications that benefit from AR visualization, removing several technical challenges from the developmental pipeline. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) systems are becoming common tools in industrial workplaces. However, factory workers are still concerned about whether head-mounted display (HMD)-based AR systems distract their awareness of the environment and therefore pose safety risks. The purpose of this study was to assess users' experience of real-world awareness when using an AR system. 19 study participants played a wooden block logic game in a laboratory with three different setups: real, AR and virtual reality (VR). Based on this study, it can be concluded that HMD-based AR systems do not decrease users' awareness of their surroundings if the virtual content is minimal and the task is done while seated. However, it was seen that more research in this area with more interactive virtual content is required. This study is an important step in understanding how AR may affect future work in industrial and safety-critical environments. [ABSTRACT FROM AUTHOR]