Search

Astronaut training is complex due to its specific infrastructure and equipment requirements, which are often costly. Immersive technologies can simulate entire environments, allowing for interactive user experiences. Previous research has confirmed the effectiveness of immersive technologies in training and learning. However, simulations alone are insufficient; participants must also engage with vehicles, machinery, and spacecraft systems to understand their impact on the environment and adjacent objects. This study develops four scenarios within three modules, inspired by NASA and ESA astronaut training programs. The first module presents a theoretical scenario using mixed reality (MR), incorporating topics from ESA's training modules. The second module offers two practical scenarios: an ISS emergency simulation that includes Hohmann transfer and circular motion concepts, and a spacewalk for repair tasks. In these scenarios, digital twins of the ISS propulsion and navigation systems, as well as a spacesuit, were created. The third module simulates a spacecraft launch, utilizing 3D models from SpaceX's Falcon Heavy and digital twins of its propulsion and navigation systems. Participants interacted with the digital twins and scenarios, generating data that was stored and analyzed against the ISS dataset from the Jet Propulsion Laboratory (JPL) and telemetry from the SpaceX Falcon Heavy launch. In this work's initial phase, the main objective was to assess whether digital twins could be integrated with immersive technologies for effective training. A mean squared error metric was employed to compare the digital twins with the physical object data, confirming the alignment of the developed digital twins with the actual systems, thereby validating their utility for astronaut training. • Mixed reality and digital twins can be used to improve astronaut's training. • Mixed reality scenarios are a safe way to learn and train about a specific topic. • Mixed reality foster training experiences: Promoting creative problem-solving skills. • Digital Twins enhances immersive technologies: Leveraging digital twins capacities for immersive training. [ABSTRACT FROM AUTHOR]

For many applications, remote guidance and telerobotics provide great advantages. For example, tele-ultrasound can bring much-needed expert healthcare to isolated communities. However, existing tele-guidance methods have serious limitations including either low precision for video conference-based systems, or high complexity and cost for telerobotics. A new concept called human teleoperation leverages mixed reality, haptics, and high-speed communication to provide tele-guidance that gives an expert nearly-direct remote control without requiring a robot. This paper provides an overview of the human teleoperation concept and its application to tele-ultrasound. The concept and its impact are discussed. A new approach to remote streaming and control of point-of-care ultrasound systems independent of their manufacturer is described, as is a high-speed communication system for the HoloLens 2 that is compatible with ResearchMode API sensor stream access. Details of these systems are shown in supplementary video demonstrations. Novel interaction methods enabled by HoloLens 2-based pose tracking are also introduced and tests of the communication and user interaction are presented. The results show continued improvement of the system compared to previous work in instrumentation, HCI, and communication. The system thus has good potential for tele-ultrasound, as well as possible other applications of human teleoperation including remote maintenance, inspection, and training. The remote ultrasound streaming and control application is made available open source. [Display omitted] • System improvements to human teleoperation demonstrate its feasibility. • Other devices such as the Nreal Light can be used for implementation. • New device-agnostic remote ultrasound streaming and control demonstrated. • HoloLens pose tracking enables human teleoperation with limited compute resources. • HoloLens-based communication system provides effective sensor data streaming. [ABSTRACT FROM AUTHOR]

Augmented Reality (AR) is a key technology in the transition to Industry 4.0 and smart manufacturing, gaining reputation in a wide range of industrial fields One promising application is in another field of Industry 4.0, the data monitoring, where AR can be used to visualize and interact with complex and big data in real time, potentially improving the efficiency and accuracy of decision-making In this work, we propose a Pervasive AR tool for data monitoring, developed in collaboration with industry partners. An web application of the same data monitoring function is also created for comparison purposes A Human-Centered Design (HCD) methodology was used to identify the needs and requirements of industrial analysts, which led to the development of these systems Preliminary user studies, with 17 participants having distinct levels of expertise in industry, data analysis and computer science, were conducted to collect opinions and suggestions for improvements, as well as, quantitative data regarding the technologies considered A succeeding user study was then prepared, in which 12 participants used two conditions (C1 — Hands-free Pervasive AR tool for HMDs and C2 — Web tool for tablet devices) to fulfill a set of data monitoring tasks in an industry assembly line The result of these studies confirm the potential of Pervasive AR for data monitoring, as it engages users, promote environmental awareness, contextualizes data and allows fast self-localization. [Display omitted] • Two user studies with more than 10 participants each, industrial settings tasks. • Understand which method (Web or AR) stands out according to the task characteristic. • The findings reveal that certain methods were more effective for types of task. • Display methods significantly influenced participants across analysis. • These studies confirm the potential of Pervasive AR for data monitoring. [ABSTRACT FROM AUTHOR]

Virtual reality (VR) and augmented reality (AR) are rapidly developing technologies that may have important healthcare-related applications, including the possible evaluation of the health and performance of astronauts. Advances in the fields of VR/AR and space exploration have independently progressed; however, over the last decade synergies and overlapping applications have emerged between spaceflight and extended reality. This paper is intended to be a resource to update the medical community on the current and future integration of VR and AR into space exploration, particularly as this integration relates to astronaut health. We highlight how these technologies are used in terrestrial healthcare with specific attention to visual function. Furthermore, we elaborate on emerging VR applications being developed for future spaceflight. The paper discusses the current state of the art, such as the Apple Vision Pro headset, and outlines key challenges like mass and volume constraints, power limitations, and the heightened cognitive and physical demands of space travel. It also addresses the potential of VR/AR technologies to mitigate these issues through efficiency, stress reduction, and exercise motivation. Finally, we emphasize the benefits that these technologies provide to overcome possible barriers to long duration spaceflight and manned exploration missions, including spaceflight associated neuro-ocular syndrome (SANS). • Virtual reality (VR) and augmented reality (AR) are rapidly developing technologies that may have important healthcare-related applications, including the possible evaluation of the health and performance of astronauts. • This paper is intended to be a resource to update the medical community on the current and future integration of VR and AR into space exploration, particularly as this integration relates to astronaut health. • We highlight how these technologies are used in terrestrial healthcare with specific attention to visual function. Furthermore, we elaborate on emerging VR applications being developed for future spaceflight. [ABSTRACT FROM AUTHOR]

The perception of size in virtual objects in Augmented Reality (AR) and Virtual Reality (VR) is a not trivial issue, as the effectiveness of manipulating and interacting with virtual content depends on the accuracy of size perception. However, there are missing straightforward comparisons between VR and AR in terms of size perception for the deep understanding of size perceptual differences. Understanding these perceptual differences can inform designers on how to adapt content when transitioning between these two spatial computing platforms. In this paper, we conducted two psychophysical experiments to measure the perceptual thresholds of size discrimination for virtual objects. Our results indicated that users are more sensitive to size changes in VR than in video see-through AR, suggesting that size differences are easier to be perceived in VR than in AR. Additionally, for increase or decrease of sizes, the accuracy of judgments showed an asymmetric trend in video see-through AR. [Display omitted] • A comparative experiment to understand the difference in size perception in AR vs VR. • The thresholds for size discrimination in AR and VR are not the same. • The accuracy of judgments is asymmetric for increases and decreases of sizes in AR. [ABSTRACT FROM AUTHOR]

The retrieval of 3D objects has gained significant importance in recent years due to its broad range of applications in computer vision, computer graphics, virtual reality, and augmented reality. However, the retrieval of 3D objects presents significant challenges due to the intricate nature of 3D models, which can vary in shape, size, and texture, and have numerous polygons and vertices. To this end, we introduce a novel SHREC challenge track that focuses on retrieving relevant 3D animal models from a dataset using sketch queries and expedites accessing 3D models through available sketches. Furthermore, a new dataset named ANIMAR was constructed in this study, comprising a collection of 711 unique 3D animal models and 140 corresponding sketch queries. Our contest requires participants to retrieve 3D models based on complex and detailed sketches. We receive satisfactory results from eight teams and 204 runs. Although further improvement is necessary, the proposed task has the potential to incentivize additional research in the domain of 3D object retrieval, potentially yielding benefits for a wide range of applications. We also provide insights into potential areas of future research, such as improving techniques for feature extraction and matching and creating more diverse datasets to evaluate retrieval performance. • ANIMAR dataset was constructed with 711 3D animal models and 140 sketch queries. • SHREC challenge track focusing on Text-based 3D ANIMAl model fine-grained Retrieval. • We receive satisfactory results from eight teams and 204 runs. [ABSTRACT FROM AUTHOR]

Astronauts are at risk for pneumothorax, a condition where injury or disease introduces air between the chest wall and the lungs (i.e., the pleural cavity). In a worst-case scenario, it can rapidly lead to a fatality if left unmanaged and will require prompt treatment in situ if developed during spaceflight. Chest tube insertion is the definitive treatment for pneumothorax, but it requires a high level of skill and frequent practice for safe use. Physician astronauts may struggle to maintain this skill on medium- and long-duration exploration-class missions, and it is inappropriate for pure just-in-time learning or skill refreshment paradigms. This paper proposes semi-automating tool insertion to reduce the risk of complications in austere environments and describes preliminary experiments providing initial validation of an intelligent prototype system. Specifically, we showcase and analyse motion and force recordings from a sensorized percutaneous access needle inserted repeatedly into an ex vivo tissue phantom, along with relevant physiological data simultaneously recorded from the operator. When coupled with minimal just-in-time training and/or augmented reality guidance, the proposed system may enable non-expert operators to safely perform emergency chest tube insertion without the use of ground resources. • System design of a desktop testbed that facilitates data-driven event labelling. • Use of a lightweight robot enables semi-automation during simulated surgical tasks. • Needle-tissue interaction forces are useful for surgical semi-automation. • Robotics and wearables allow for human-in-the-loop system performance monitoring. [ABSTRACT FROM AUTHOR]

• A systematic literature review of immersive technologies (ImTs) for occupational safety and health (OSH) in construction is conducted. • OSH areas, hazards and conditions that are addressed by ImTs are provided. • Challenges and future research associated with the use of ImTs for OSH are provided. Introduction : The construction industry employs about 7% of global manpower and contributes about 6% to the global economy. However, statistics have depicted that the construction industry contributes significantly to workplace fatalities and injuries despite multiple interventions (including technological applications) implemented by governments and construction companies. Recently, immersive technologies as part of a suite of industry 4.0 technologies, have also strongly emerged as a viable pathway to help address poor construction occupational safety and health (OSH) performance. Method: With the aim of gaining a broad view of different construction OSH issues addressed using immersive technologies, a review on the application of immersive technologies for construction OSH management is conducted using the preferred reporting items for systematic reviews and meta-analysis (PRISMA) approach and bibliometric analysis of literature. This resulted in the evaluation of 117 relevant papers collected from three online databases (Scopus, Web of Science, and Engineering Village). Results : The review revealed that literature have focused on the application of various immersive technologies for hazard identification and visualization, safety training, design for safety, risk perception, and assessment in various construction works. The review identified several limitations regarding the use of immersive technologies, which include the low level of adoption of the developed immersive technologies for OSH management by the construction industry, very limited research on the application of immersive technologies for health hazards, and limited focus on the comparison of the effectiveness of various immersive technologies for construction OSH management. Conclusions and Practical Applications: For future research, it is recommended to identify possible reasons for the low transition level from research to industry practice and proffer solutions to the identified issues. Another recommendation is the study of the effectiveness of the use of immersive technologies for addressing health hazards in comparison to the conventional methods. [ABSTRACT FROM AUTHOR]

Intelligent Virtual Agents (IVAs) can provide users with a friendly experience and have a wide range of applications in the era of artificial intelligence. However, most of existing IVAs are designed for personal computers. Design and user studies of IVAs on smartphones are uncommon. Therefore, developing IVAs for smartphones is an interesting topic. Considering Augmented Reality (AR) technology can provide more potential application value for IVAs, we mainly investigate users' experiences of AR IVAs on smartphones in this paper. To make an IVA more suitable for a smartphone, a lightweight IVA's cognitive architecture is proposed. To find out the factors that affect users' interaction experiences, the effects of humanoid embodiment and emotional expressions of IVAs on users' perceptions and experiences are explored. A museum is used as a specific task scenario to measure users' experiences. Three forms of AR agents are evaluated in this scenario: a voice assistant without an entity, a humanoid IVA without emotional expressions, and a humanoid IVA with emotional expressions. The results show that compared with the voice assistant, a humanoid embodiment can significantly improve the user's experience, and compared with humanoid IVA without emotional expressions, a humanoid IVA with emotional expressions is more welcome. Moreover, we use the cloud model to describe the uncertainty of IVAs' actions (blinking and body orientation). The results show that the uncertainty of actions can increase the believability of IVAs. [ABSTRACT FROM AUTHOR]

Virtual reality (VR), augmented reality (AR) and mixed reality (MR) can evoke a sense of presence, where a person can feel as though they are physically present in a virtual environment. The intention of this research is to understand whether VR/AR/MR can create human connection and experience through interactions with virtual environments to assist in the prevention and treatment of psychological impacts to astronaut health during long-term and long-distance space missions (experiencing Earth from space); and for space tourism, education and outreach purposes as a cost-effective way for people to experience space; to help people understand how important the space industry is to our lives; and to inspire future careers (experiencing space from Earth). Qualitative literature review was undertaken to understand current and potential future VR/AR/MR technologies and how they are being used in psychology; in space; for astronaut mental health in space; for two-way communication; for general tourism, education and outreach purposes; and for space tourism, space education and space outreach purposes. Recommendations are proposed for further study into VR/AR/MR as a tool for astronaut mental health and wellbeing; and for further development of VR/AR/MR experiences for space tourism, education and outreach purposes. • Virtual reality can be used to improve mental health. • Virtual reality can be used to improve mental health in isolation and confinement. • Virtual reality may be used to alleviate mental health issues in astronauts. • Virtual and augmented reality are safe ways to inspire people about space. • Virtual and augmented reality are a cost-effective alternative to space tourism. [ABSTRACT FROM AUTHOR]

In augmented reality (AR) applications, detecting and tracking real-world objects remains a challenge. Another challenge is making the loaded virtual objects truly reflect the lighting and shadow conditions of the actual environment in which they are located. In this paper, we proposed an improved method for real-time light estimation of environmental lighting in AR applications, tracking the movement of physical objects, and making the corresponding virtual objects reflect the lighting and shadows of the actual environment. First, we proposed an effective method for estimating light in an AR scene by training a deep neural network once on different scenes from the input of RGB-D images to determine the direction of the brightest light. Second, we experimented with advanced methods of detecting and tracking different types of real objects (flat surfaces, simple geometries, and complex geometries) for visualizing virtual materials on the top of these surfaces, where virtual materials will acquire the conditions of the real place in terms of lighting, reflection, and shadows depending on our proposed method for light estimation. Our method of light estimation was tested via experiments on the three types of real objects mentioned above. Results indicate that our method achieves higher resolution and lower error rate compared with other work. Our methods have achieved very good results in terms of detection accuracy, tracking accuracy, realistic visualization of materials, matching real shade, and lighting conditions. [Display omitted] [ABSTRACT FROM AUTHOR]

• Novel digital tool for assisting astronaut lunar and planetary geology activities proposed. • Combination of an AR device with an IoT architecture. • Semi-structured interviews with astronauts and experts used to evaluate the AR-IoT tool. • Hands-free operations enhanced by the tool design and architecture. Humans are embarking on a new era of space exploration with the plan of sending crewed spacecraft to the Moon, Mars, and beyond. Extravehicular activities (EVAs) will be an essential part of the scientific activities to be carried out in these missions, and they will involve extensive geological fieldwork. These EVAs entail many challenges as real-time support from ground control cannot be provided to astronauts. Hence, new human–machine interfaces are urgently needed to enhance mission autonomy for astronauts and reduce ground communication dependability for real-time operations. This study introduces an Augmented Reality (AR) Internet of Things tool for astronauts to carry out geological activities. It proposes a theoretically-informed user-centred design method supported by expert feedback and an evaluation method. The tool was assessed via questionnaires and semi-structured interviews with European Space Agency (ESA) astronauts and geological field activities experts. Content analysis of the interviews revealed that user satisfaction was the first most mentioned (32% of 139 quotes) usability aspect. Key design factors identified were: displaying solely important information in the field of view while adjusting it to the user's visual acuity, easy usage, extensibility, and simplicity. User interaction was the second most mentioned (24% of 139 quotes) usability aspect, with voice seen as the most intuitive input. Finally, this research highlights important factors determining the usability and operational feasibility of an AR tool for analogue training missions and provides a foundation for future design iterations and an eventual integration of AR into the spacesuit's visor. [ABSTRACT FROM AUTHOR]

The advancement of women's careers in construction following the Industrial Revolution (IR) 4.0 is a significant concern. This study aims to investigate the primary factors and barriers encountered by women professionals in the construction industry, and how IR 4.0 technologies can mitigate these challenges. Qualitative and quantitative methodologies were employed to identify the most impactful barriers faced by women in construction and assess the effectiveness of IR 4.0 technologies in supporting their careers. Key challenges such as fear of heights, work-life balance, musculoskeletal disorders, and exposure to harmful gases on-site were identified as significant barriers for women in construction. Technological applications such as advanced building technology, autonomous construction, building information modelling (BIM), augmented reality (AR), virtual reality (VR), and unmanned aerial vehicles (UAVs) were identified as having the potential to overcome these barriers. Fear of heights with a relative importance index (RII = 0.785) and work-life balance (RII = 0.772) were determined to be the most influential factors contributing to these barriers for women. Meanwhile, advanced building materials (RII = 0.831) and autonomous construction (RII = 0.806) emerged as the primary technologies within IR 4.0 that could positively impact women's careers in construction. The findings indicate that IR 4.0 technologies effectively address the barriers faced by women in the construction industry. This study sheds light on these barriers and offers guidelines for their resolution, leveraging the advancements of IR 4.0. [ABSTRACT FROM AUTHOR]

Fire safety training is crucial to increase building occupants' chances of surviving a fire emergency. Traditional training methods, such as lectures using video presentations, have limitations that can hinder learning performance. This article describes the development and testing of an alternative training solution using Augmented Reality (AR) technology. Through a controlled between-subject experiment, the AR-based fire safety training method was compared against a conventional video-based one based on participants' knowledge acquisition and retention, intrinsic motivation, and self-efficacy. Results suggest that the AR system was well-designed and as effective as the traditional method in terms of knowledge acquisition and retention and overall learning experience. However, it was found to be superior in terms of intrinsic motivation gain and self-efficacy retention. These findings demonstrate the potential of AR-based training methods to enhance building occupants' safety and provide directions for future developments and research in the field. • An AR fire safety training system is developed and compared to video training. • Training effectiveness is assessed through learning performance measures. • AR training outperformed video training in intrinsic motivation and self-efficacy. • The developed AR training system may enhance building occupants' safety. [ABSTRACT FROM AUTHOR]

• The study investigated AR's effect on reducing energy use in IEQ control devices. • Subjects' stress levels were verified through physiological response. • Reduced energy usage caused IEQ changes, without affecting subjects' stress levels. • The analysis showed links, humidifier energy with BMI and visual comfort with age. • AR system is crucial in development of future smart home energy management system. This study investigated the effects of visualizing energy consumption data of indoor environmental quality (IEQ) control devices (air cooler, humidifier, and light) through an augmented reality (AR) system. The research assessed changes in energy usage, IEQ, and subjective satisfaction. Physiological responses, such as heart rate variability (HRV), electroencephalogram (EEG), electrodermal activity (EDA), and skin temperature (SKT), were measured to quantitatively gauge subjective satisfaction. The study also explored variations in energy consumption, IEQ, satisfaction, and physiological responses based on user characteristics like age, gender, and body mass index (BMI). This study found that visualizing the energy consumption of IEQ control devices through AR significantly reduced their energy usage. The air cooler's energy consumption decreased by 19.36%, the humidifier's by 42.86%, and the light's by 27.57%. This led to changes in indoor temperature and illuminance, without affecting subjects' stress levels. The analysis based on subjects' characteristics revealed correlations, such as skin temperature impacting gender, humidifier energy affecting BMI, and visual sensation being linked to age. Consequently, the study suggests a hyper-personalized AR system, considering occupants' traits, to enhance thermal and visual comfort. This system is anticipated to play a crucial role in the future development of smart home energy management systems. [ABSTRACT FROM AUTHOR]

The deployment of Industry 4.0 emerging technologies such as Augmented reality (AR), Virtual reality (VR), and collaborative Robots enhances flexibility and precision in the manufacturing systems. A flexible, and collaborative manufacturing system with robots and human integrates the prominent attributes of customized automation and fortify the safety of the human workforce. The proposed work involves the approach of AR-assisted assembly layout configurations for the Human-Robot Collaboration (HRC) production system, which is a typical example of the smart manufacturing system. The reconfiguration/configuration of the assembly layout is triggered with respect to the demand of new variants into the market and the upgradation of new technologies in the existing system. A multi-objective algorithm is proposed to maximize the production outlet with effective utilization of floor space, minimum idle time and optimum resource allocation. The selected work is branched into three phases: To begin with, the approach of task allocation is constructed to identify the suitable resources (human, robot) for the appropriate assembly tasks in the HRC system. The outcomes such as resource allocation, number of resources, and footprint of resources are considered as input to the optimization model. Further, modified particle swarm optimization (MPSO) is deployed to generate a feasible assembly layout for the HRC manufacturing system. Additionally, the breath-first approach has been utilized to generate the optimal aisle as a conveyor in the layout. Notably, the comparisons studies have been carried out in this work, which facilitates to explore the robustness of proposed modified PSO algorithms for the selected applications. Subsequently, the virtual and AR tool is deployed for the layout planer to explore and validate the obtained layout from the PSO model in both the virtual and physical environment of the industrial workspace. Finally, an industrial case study of rotary gear pump assembly system is adopted for the implementation of the proposed framework. [ABSTRACT FROM AUTHOR]

Currently, we face an exponentially increasing interest in immersion, especially sensory–driven immersion, mainly due to the rapid development of ideas and business models centered around a digital virtual universe as well as the increasing availability of affordable immersive technologies for education, communication, and entertainment. However, a clear definition of 'immersion', in terms of established neurocognitive concepts and measurable properties, remains elusive, slowing research on the human side of immersive interfaces. To address this problem, we propose a conceptual, taxonomic model of attention in immersion. We argue (a) modeling immersion theoretically as well as studying immersion experimentally requires a detailed characterization of the role of attention in immersion, even though (b) attention, while necessary, cannot be a sufficient condition for defining immersion. Our broader goal is to characterize immersion in terms that will be compatible with established psychophysiolgical measures that could then in principle be used for the assessment and eventually the optimization of an immersive experience. We start from the perspective that immersion requires the projection of attention to an induced reality, and build on accepted taxonomies of different modes of attention for the development of our two–competitor model. The two–competitor model allows for a quantitative implementation and has an easy graphical interpretation. It helps to highlight the important link between different modes of attention and affect in studying immersion. • Induced realities and the actual reality are competing for attention. • Attention is a necessary condition but not a sufficient condition for immersion. • Different modes of attention can be studied by the two-competitor model. • The two–competitor model allows for a quantitative implementation and has an easy graphical interpretation. • The two-competitor model might help to optimize an immersive experience. [ABSTRACT FROM AUTHOR]

Head-mounted Augmented Reality (AR) displays overlay digital information on physical objects. Through eye tracking, they provide insights into user attention, intentions, and activities, and allow novel interaction methods based on this information. However, in physical environments, the implications of using gaze-enabled AR for human activity recognition have not been explored in detail. In an experimental study with the Microsoft HoloLens 2, we collected gaze data from 20 users while they performed three activities: Reading a text, Inspecting a device, and Searching for an object. We trained machine learning models (SVM, Random Forest, Extremely Randomized Trees) with extracted features and achieved up to 89.6% activity-recognition accuracy. Based on the recognized activity, our system—GEAR—then provides users with relevant AR feedback. Due to the sensitivity of the personal (gaze) data GEAR collects, the system further incorporates a novel solution based on the Solid specification for giving users fine-grained control over the sharing of their data. The provided code and anonymized datasets may be used to reproduce and extend our findings, and as teaching material. [ABSTRACT FROM AUTHOR]

Invisible fiducial markers are introduced for localization of Areas Of Interest (AOIs) in mobile eye tracking applications. Fiducial markers are made invisible through the use of film passing Infra-Red (IR) light while blocking the visible spectrum. An IR light source is used to illuminate the markers which are then detected by an IR-sensitive camera, but which are imperceptible by the human eye. We provide the first empirical study that demonstrates such invisible markers are not distracting to a given task, as demonstrated in a flight simulator where distraction of visible and invisible markers are compared between experienced and novice pilots. Fixation frequency and subjective distraction scores showed that visible markers disrupted natural gaze behavior, particularly in novice pilots. Our findings show that invisible markers should be used when there is a need for them to remain inconspicuous. [Display omitted] • Visible markers were shown to disrupt gaze behaviors, especially in novices. • Marker placement may be a contributing factor to gaze behavior disruption. • Novice pilots demonstrate more frequent gaze tunneling behaviors. [ABSTRACT FROM AUTHOR]

Obtaining feedback from time-constrained end-users is a major challenge in evaluating novel systems for specialised applications. The performance and feedback of engineers and surgeons was evaluated through an experiment where participants were asked to identify tumour locations within an anatomically realistic silicon liver model across three different conditions of an Augmented Reality (AR) prototype system (Baseline, Split AR and Full AR). Our findings show that engineers and surgeons share some similarities in their performance, feedback and behaviour, particularly when reliance on the AR system is high for both groups. However, engineers typically focus more on accuracy of the image alignment and are more accurate in their responses when supported by AR. Senior surgeons typically perform faster and use AR as supplementary information, while the performance of junior surgeons is more closely aligned to the performance of engineers. We conclude that engineers could be involved in preliminary evaluations of a surgical system or in evaluations of systems which are aimed at training junior surgeons, but that it is essential to involve surgeons in later evaluations, where ecological validity is a more important consideration. • Level of reliance on a system and expectations around system accuracy can differ between engineers and surgeons. Where surgeons (particularly senior surgeons) were faster and use augmented reality (AR) as supplementary information and engineers were more accurate when using AR system. • We found that testing with engineers could be an ideal alternative when testing digital systems during preliminary stages. • This work recommends that engineers should not replace testing technologies with experienced surgeons, rather this method should only be used as an early indicator in the preliminary stages of a research project. • Engineers could also be involved in evaluations of systems which are aimed at training junior surgeons, but it is essential to involve surgeons in later evaluations, where ecological validity is a more important. [ABSTRACT FROM AUTHOR]

A lack of navigator's Situation Awareness (SA) is one of the leading causes of maritime accidents. Visually observing the area surrounding a vessel continues to be a critical aspect and best practice of safe navigation to establish and maintain SA. Augmented Reality (AR) allows the placement of information in a user's field of view, which can encourage navigators to spend more time looking up at their external environment whilst still having access to operational data. However, empirical evidence on the impact of AR on maritime operations is limited. This paper investigates the effects of AR on navigator SA & Head-Down Time (HDT) using a within-group quasi-experimental design. Seventeen licensed navigators and nautical students analysed twelve navigation scenarios: six non-AR (control) and six AR (experimental) scenarios using a maritime training simulator. SA was measured via SAGAT scores for each scenario and the SA-SWORD to compare preferences. Each scenario was video recorded and analysed for participant's total amount of HDT and head-down occurrences in each scenario. Results found that the addition of AR significantly reduced participant HDT (by a factor of 2.67) and head-down occurrences (by 62%) in comparison to navigation scenarios without AR. Furthermore, AR did not significantly improve mean SA. This study contributes to the limited empirical data on the effects of AR on operator performance, demonstrating the potential value of AR for improving SA and facilitating increased head-up time during maritime navigation, which in turn could improve safety at sea. • We investigate the impact of AR on a maritime navigator's Situation Awareness. • Participants had significantly more head-up time navigating with AR. • Participants were found to have less head-down occurrences navigating with AR. • AR did not significantly increase Situation Awareness in navigation. • Participants preferred using AR and felt it added value to navigation. [ABSTRACT FROM AUTHOR]

Target acquisition tasks quantify human motor and perceptual abilities while performing discrete tasks to support interface design and sensorimotor assessments. This study investigated the effects of display, Touchscreen and Augmented Reality (AR), on a standardized 2D multidirectional target acquisition task. Thirty-two participants performed the target acquisition task with both modality types and at two indexes of difficulty. The touchscreen modality yielded improved performance over AR as measured by accuracy, precision, error rates, throughput, and movement time. Throughput using the nominal index of difficulty was 10.12 bits/s for touchscreen and 3.11 bits/s for AR. AR designers can use the results to improve performance when designing AR interfaces by selecting larger buttons when accuracy and efficiency are required and by embedding perception cues to button target surfaces such as depth and proximity cues. • A multidirectional tapping task with augmented reality and touchscreen is examined. • The touchscreen modality yielded improved user performance compared to AR. • Misalignment errors along all three target axes were the most common AR error type. • AR design recommendations are provided for usability across varied applications. [ABSTRACT FROM AUTHOR]

Regular inspection and management of building defects are vital for the structural integrity of buildings, but traditional manual methods often lead to inefficiencies and misjudgments in assessing defect severity. This paper presents a framework integrating computer vision, augmented reality (AR), and building information modeling (BIM) to enhance defect inspection and management. The framework includes an AR-based defect inspection (ARDI) application with interactive user interfaces and a BIM-based defect data management (DDM) platform. The ARDI application utilizes YOLOv5 + DeepSORT algorithms for effective defect detection and tracking from real-time video streams, precisely mapping defect dimensions from 2D images to 3D coordinates and synchronizing this data with BIM for uploading to the DDM platform. Two experiments confirm that the AR technology achieves centimeter-level precision, and the framework overall enhances inspection efficiency by 78.63% compared to manual methods. This advanced framework not only improves inspection efficiency but also enables managers to comprehensively manage the entire inspection process. • A framework is proposed to inspect and manage building defect, which integrates computer vision, AR, and BIM. • An AR-based defect inspection (ARDI) application with interactive user interfaces is developed for defect inspection • A BIM-based defect data management (DDM) platform is established, where managers can visually manage the inspection data. • One experiment confirmed that HoloLens 2 can generate centimeter-level accuracy. • The proposed framework can improve the inspection efficiency of 78.63% compared with manual-based group. [ABSTRACT FROM AUTHOR]

Forensic anthropologists are often confronted with human remains that have been damaged due to trauma, fire, or postmortem taphonomic alteration, frequently resulting in the fracture and fragmentation of skeletal elements. The augmented reality (AR) technology introduced in this paper builds on familiar 3D visualization methods and utilizes them to make three dimensional holographic meshes of skeletal fragments that can be manipulated, tagged, and examined by the user. Here, CT scans, neural radiance fields (NeRF) artificial intelligence software, and Unreal Engine production software are utilized to construct a three-dimensional holographic image that can be manipulated with HoloLens™ technology to analyze the fracture margin and reconstruct craniofacial elements without causing damage to fragile remains via excessive handling. This allows forensic anthropologists a means of assessing aspects of the biological profile and traumatic injuries without risking further damage to the skeleton. It can also be utilized by students and professional anthropologists to practice refitting before reconstructing craniofacial fragments if refitting is necessary. Additionally, the holographic images can be used to explain complicated concepts in a courtroom without the emotional response related to using bony elements as courtroom exhibits. • Craniofacial reconstruction aids with the biological profile and signs of trauma. • CT-based AR craniofacial reconstruction can reduce bone damage from overhandling. • Neural resonance fields (NeRF) can rapidly convert CT scans into AR visual content. • AR visualization minimizes damage from manual manipulation. • AR allows the viewer to assess the skeleton in three rather than two dimensions. [ABSTRACT FROM AUTHOR]

The ubiquity of smartphone and tablet devices, combined with the increasing availability of serious games, has enabled students to learn various abstract concepts in an appealing and convenient manner. While several researchers have explored the use of Augmented Reality (AR) in serious games, many of these games have not been critically explained or evaluated. To that end, we employed game-based learning methodologies and Game Learning Analytics (GLA) to systematize the design and evaluation of an AR-based serious game to teach programming. We evaluated our game for usability and effectiveness by conducting a user study on twenty-seven undergraduate students. The evaluation primarily consisted of a learning test conducted twice – before and after playing the game – along with a usability questionnaire that players completed after playing the game. Our results showed that players made significant progress after playing the game. The game helped players improve their basic programming skills, especially for the group having lower prior programming skills. The results highlighted various ways in which GLA can be used to benefit different stakeholders in the game. Based on players' qualitative responses, we also identified several areas of improvement, most prominently the trade-off between ease of use and game complexity. We provide suggestions and discuss implications for future work. • Design and evaluation procedures for an Augmented Reality-based serious game to teach programming are presented. • An empirical study with twenty-seven undergraduate students is conducted. • Students with lesser prior programming skills show significant improvement after playing the game. • A Game Learning Analytics framework is incorporated in the game for various stakeholder benefits. • Suggestions and implications for future work are presented. [ABSTRACT FROM AUTHOR]

With recent technological advancements in sports broadcasting, remote spectators are presented with an enriched experience. These enriched experiences include additional content such as statistics and graphics that support game understanding. However, these technological advancements are often not accessible to on-site sports spectators. In this paper, we explore the opportunities of using situated visualization to enrich on-site sports spectating. Situated visualization techniques allow us to display information in a spatial context to its physical reference and have the potential to close the gap between on-site spectating and content access. With regards to this, we developed a framework for situated visualization focusing on sports spectating. We identified components needed for such a use case and developed two novel situated visualization approaches based on the proposed framework: (1) situated broadcast-styled visualization and (2) situated infographics. To evaluate the visualization approaches and explore user preferences, we conducted a lab study and a subsequent on-site study in a stadium. [Display omitted] • Conceptual framework describing components of on-site situated sports spectating. • Main components of sports situated visualization are Canvas, Content and Context. • Framework led to situated broadcast-style visualization and situated infographics. • Positive responses towards situated infographics and an improved game understanding. • No increase in mental demand recorded for situated infographics despite the benefits. [ABSTRACT FROM AUTHOR]

Remote collaboration using Augmented Reality (AR) has potential to support physically distributed team-members that need to achieve a common goal by increasing knowledge retention, improving understanding and awareness of the problem and its context. In this vein, the path to achieve usable, realistic and impactful solutions must entail an explicit understanding regarding how remote collaboration occurs through AR and how it may help contribute to a more effective work effort. Thus, characterization and evaluation of the collaborative process is paramount, but a particularly challenging endeavor, due to the multitude of aspects that define the collaboration effort. In this context, the work presented here contributes with a critical analysis, discussing current evaluation efforts, identifying limitations and opportunities. Then, we outline a conceptual framework to support researchers in conducting evaluations in a more structured manner. To instrument this vision, an evaluation toolkit is proposed to support contextual data collection and analysis in such scenarios and obtain an additional perspective on selected dimensions of collaboration. We illustrate the usefulness and versatility of the toolkit through a case study on remote maintenance, comparing two distinct methods: sharing of video and AR-based annotations. Last, we discuss the results obtained, showing the proposed vision allows to have an additional level of insights to better understand what happened, eliciting a more complete characterization of the work effort. [Display omitted] • How evaluation of the collaborative process is conducted in AR-remote scenarios. • Conceptual framework for conducting evaluations in a more structured manner. • Creation of an evaluation toolkit for improved data collection and analysis. • User study demonstrates the toolkit capabilities in a remote maintenance scenario. • Toolkit elicits a more complete characterization of the collaborative process. [ABSTRACT FROM AUTHOR]

The use of technology in cultural heritage sites for end-users, like virtual audio-visual reconstructions, has become a common solution to improve the communication and perception between these spaces and their visitors. Since humans perceive the world with different senses and in real-time, to evoke more than one or two senses at a time can bring benefits for the user perception. Accordingly, some very different implementations unleashing multisensory experiences have been made. Aiming to understand how different stimuli are being evoked in multisensory experiences to enhance cultural heritage experiences, and how these implementations are being evaluated, this paper presents a systematic review of technological multisensory applications in cultural heritage. Thus, the collected and analyzed data, focused on technologies used, purpose of the experience, stimuli explored, evaluation process, main findings obtained, and limitations found, will provide valuable information for further implementations. The overall results unleash the wide diversity available for multisensory implementations, from technological solutions to available content for users. Covering the pros and cons of such diversity, this study sustains the use of multisensory applications in cultural heritage as a powerful tool to enrich users' visits. [Display omitted] • Systematic review on Multisensory VR and AR implementations in Cultural Heritage. • Analysis of evoked stimuli and of what technologies were used to evoke each stimulus. • Detailed analysis of the purpose of each implementation and of conducted evaluations. • Mention to studies' main findings and main limitations found. • Discussion related to analyzed multisensory studies. [ABSTRACT FROM AUTHOR]

One of the main targets of criticism of head-mounted displays (HMDs) is the field of view (FOV) size, whether in virtual or augmented reality. This limitation is prominent with optical see-through head-mounted displays (OST-HMD), as those with narrow overlay FOV (OFOV) sizes only provide a small window to view virtual objects. We investigated if restricting this OFOV negatively affects a user's ability to memorize spatial locations in a simulation of a work environment, and consequently, long-term memory transfer to an equivalent scenario in the real world two days later. To find empirical evidence, we conducted a within-subjects experiment with 18 participants performing in three phases with an OST-HMD, simulated on an immersive HMD. For each phase, they viewed the training scenario with a different OFOV size of the augmentable area (30°, 70°, 110° diagonal). Results from recall tests showed that smaller OFOV size did not significantly affect user's performance on both short-term and transfer tests, but HMD data revealed that users rotated their heads less with a 110° OFOV. We also found that proximity of objects to memorize had an interaction effect with smaller OFOV sizes. Our findings could have implications on the design and HMD choices of augmented training. [Display omitted] • We investigate the effects of three overlay FOV (OFOV) sizes on spatial memorization. • We highlight the impact of OFOV and object proximity on memorization. • OFOV size did not affect spatial memorization, but wide OFOV reduced head rotation. • Our results imply that a larger OFOV may not necessarily benefit training outcomes. [ABSTRACT FROM AUTHOR]

This paper presents an Augmented Reality (AR)-based network cabling tutoring system that trains users how to interconnect cables within a Virtual Local Area Network (VLAN) on a physical switching rack. AR arrows combined with text-based instruction and a checklist provided assistance during practical learning. When learners made a mistake, an Intelligent Tutoring System (ITS) identified the source of the mistake and provided real-time feedback using text, AR and text-to-speech mechanisms. The design was motivated by the human-cognitive architecture and its five evolutionary principles (proposed by Sweller and Sweller (2006)). Users performed four consecutive network cabling training tasks with assistance from our ITS. We found that users made fewer errors when the AR cues, text-based instruction and checklist solutions were replaced with summarised information and then removed completely in the final task compared to those who used an identical system with fixed instruction. Cognitive Load Theory (CLT) explains our results by suggesting that the instructional mechanisms become redundant as knowledge increases. Implications of the study are discussed as well as how AR can help facilitate knowledge transfer in the network cabling domain. [Display omitted] • Several cognitive load considerations for AR-based network training are proposed. • A novel AR training system combining fading with an ITS is proposed. • Our AR training system reduces mistakes during network security training. • In network security training, AR should be faded out as expertise increases. [ABSTRACT FROM AUTHOR]

Remote Collaboration mediated by Mixed and Augmented Reality (MR/AR) shows great potential in scenarios where physically distributed collaborators need to establish a common ground to achieve a shared goal. So far, most research efforts have been devoted to creating the enabling technology, overcoming engineering hurdles and proposing methods to support its design and development. To contribute to more in-depth knowledge on how remote collaboration occurs through these technologies, it is paramount to understand where the field stands and how characterization and evaluation have been conducted. In this vein, this work reports the results of a literature review which shows that evaluation is frequently performed in ad-hoc manners, i.e., disregarding adapting the evaluation methods to collaborative AR. Most studies rely on single-user methods, which are not suitable for collaborative solutions, falling short of retrieving the necessary amount of contextualized data for more comprehensive evaluations. This suggests minimal support of existing frameworks and a lack of theories and guidelines to guide the characterization of the collaborative process using AR. Then, a critical analysis is presented in which we discuss the maturity of the field and a roadmap of important research actions is proposed, that may help address how to improve the characterization and evaluation of the collaboration process moving forward and, in consequence, improve MR/AR based remote collaboration. [Display omitted] • How evaluation of the collaborative process is conducted in remote scenarios using AR. • Integration of characterization and evaluation methods is of paramount importance. • Critical analysis to discuss the maturity of remote collaboration using AR. • AR remote collaboration is in the Replication-Empiricism phases of the BRETAM model. • Roadmap of research actions to elicit characterization of the collaboration process. [ABSTRACT FROM AUTHOR]

The representation of consistent mixed reality (XR) environments requires adequate real and virtual illumination composition in real-time. Estimating the lighting of a real scenario is still a challenge. Due to the ill-posed nature of the problem, classical inverse-rendering techniques tackle the problem for simple lighting setups. However, those assumptions do not satisfy the current state-of-art in computer graphics and XR applications. While many recent works solve the problem using machine learning techniques to estimate the environment light and scene's materials, most of them are limited to geometry or previous knowledge. This paper presents a CNN-based model to estimate complex lighting for mixed reality environments with no previous information about the scene. We model the environment illumination using a set of spherical harmonics (SH) environment lighting, capable of efficiently represent area lighting. We propose a new CNN architecture that inputs an RGB image and recognizes, in real-time, the environment lighting. Unlike previous CNN-based lighting estimation methods, we propose using a highly optimized deep neural network architecture, with a reduced number of parameters, that can learn high complex lighting scenarios from real-world high-dynamic-range (HDR) environment images. We show in the experiments that the CNN architecture can predict the environment lighting with an average mean squared error (MSE) of 7.85 × 10−4 when comparing SH lighting coefficients. We validate our model in a variety of mixed reality scenarios. Furthermore, we present qualitative results comparing relights of real-world scenes. [Display omitted] • Automatic end-to-end method to estimate the environment lighting in XR applications. • A CNN architecture that learns a latent-space of the environment lighting. • A methodology to generate egocentric mixed-reality-views from HDR panoramas. • Real-time lighting estimation that does not make assumptions about the XR scene. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) and Artificial Intelligence (AI) are technologies pioneers in innovation and alteration in several domains. AR allows the creation of an entirely new and interactive experience for users. However, there are several drawbacks in developing AR applications, such as the marker identification process and the creation of content itself. These are very time-consuming procedures and require ad-hoc development. The advantages of using AI to solve AR limitations have recently been explored in literature. Motivated by these findings, in this paper it is proposed DeepReality, a software toolkit plug-in for Unity 3D. It is conceived for allowing developers to integrate any Deep Learning (DL) models into Unity, through AR Foundation and Barracuda inference engine. DeepReality is aimed at simplifying and streamlining the usage of DL models in conjunction with AR. As such, users skilled in Unity and DL can easily create mobile applications (iOS and Android) to: extract visual features of real-world objects (framed with the device camera) via DL; Show on-screen content on top of those real-world objects, via AR. DeepReality performs object semantic processing within the scene, and extended semantic effects for incongruent objects, overcoming the environmental tracking, which is feature-based. In order to test DeepReality usability, experiments have been performed on the execution time and memory usage data, demonstrating the feasibility and possibility of integrating and using DNNs models in mobile applications for AR. The complexity analysis confirms that DeepReality can be completely executed on mobile devices. DeepReality is also open-source and it is freely available in the Unity asset store. By fostering accessible AI-AR integration, DeepReality addresses key shortcomings in existing approaches, encapsulating contributions such as versatile DL integration, open-source accessibility, operational validation, and comprehensive metrics analysis. DeepReality empowers developers to transcend boundaries, enriching AR applications with AI's transformative potential. Our proposed framework fosters benchmarking, comparison, and a future harmonised by AR-AI synergy. [Display omitted] • DeepReality is an open-source toolkit, freely available in the Unity asset store. • DeepReality integrates Deep Learning models for streamlined mobile apps. • DeepReality leverages AR Foundation and Barracuda inference engine. • It demonstrates efficient execution and memory usage in mobile AR. • Complexity analysis confirms that it can be fully executed on mobile devices. [ABSTRACT FROM AUTHOR]

With the intensifying impacts of the climate crisis becoming more severe day by day, the industries have been compelled to accelerate the implementation of sustainable practices, especially in the fashion industry, as a motivation behind this study. Although, the utilization of digitalization and artificial intelligence as a means of sustainable applications is increasingly growing, the current literature is mainly focused only on conducting fashion shows using augmented reality googles, there is an obvious gap in the literature addressing the mobile applications capabilities in terms of artificial intelligence and augmented reality. Hence, the purpose of this study is to examine artificial intelligence/augmented reality - powered mobile applications from a consumer point of view. In pursuit of this aim, several alternative applications were reviewed, the factors affecting the consumer's decision were identified, and the Organization, Rangement et Synthese de Donnes Relationnelles (ORESTE) technique was utilized as a multi-criteria decision-making technique for the analysis as a contribution of artificial intelligence in the engineering applications. Additionally, the ORESTE technique has been extended with Fermatean Fuzzy sets to make a theoretical contribution to the literature as the main numerical achievement. The findings of the study illustrate that (i) the consumers prefer camera-assisted measurement applications, and (ii) the applications offering synchronized suggestions with existing fashion brand's products are preferred. Moreover, the study provides a significant illustration of the use of artificial intelligence powered mobile applications as an actual practical implication, and it creates a suitable ground for the sector authorities to take strategic decisions as a managerial implication. [ABSTRACT FROM AUTHOR]

Resilient system performance in high-stakes settings, which includes the ability to monitor, respond, anticipate, and learn, can be enhanced for trainees through simulation of realistic scenarios enhanced by augmented reality. Active learning strategies can enhance simulation-based training, particularly the mental model articulation principle where students are prompted to anticipate what will happen next and the reflection principle where students self-assess their performance compared to a gold standard expert model. In this paper, we compared simulation-based training for trauma care with and without active learning strategies during pauses in the simulated action for progressively deteriorating patients. The training was conducted online and real-time without a facilitator, with 42 medical students viewing training materials and then immediately taking an online quiz for three types of trauma cases: hemorrhage, airway obstruction, and tension pneumothorax. Participants were randomly assigned to either the experimental or control condition in a between-subjects design. We compared performance in the control and experimental conditions based on: A) the proportion of cues correctly recognized, B) the proportion of accurate diagnoses, C) the proportion of appropriate treatment interventions, and D) verbal briefing quality on a 1–5 scale. We found that the training intervention increased recognition of subtle cues critical for accurate diagnosis and appropriate treatment interventions; the training did not improve the accuracy of diagnoses or the quality of the verbal briefing. We conclude that incorporating active learning strategies in simulation-based training improved foundational capabilities in detecting subtle cues and intervening to rescue deteriorating patients that can increase the readiness for trainees to contribute to resilient system performance in the high-stakes setting of emergency care in hospitals. [ABSTRACT FROM AUTHOR]

Typical augmented reality (AR) technologies superimpose virtual information onto the real environment for task guidance and assistance by tracking fiducial markers. However, these have narrow recognition ranges due to the markers, and it is difficult to track markers when occlusion occurs. Therefore, it is challenging to seamlessly register real objects in AR scenes captured by the camera without requiring fiducial markers. Deep learning–based approaches to 3D object registration and task assistance have recently indicated promising directions for superimposing 3D virtual objects onto corresponding real objects or their surroundings seamlessly. However, since supervised learning-based object registration requires many pose annotations on images for the locations and orientations of real objects, this process is a time-consuming and labor-intensive task. Thus, this paper proposes a new self-training based augmented reality method for seamless real object registration and task assistance without time-consuming pose annotations. As the industrial components or objects have their 3D geometric models, the proposed approach utilizes them as their synthetic data. The automatically generated synthetic dataset is used for training the deep learning model, which outputs a colored coordinate map of objects to be detected. 2D–3D key point matching between the 2D colored coordinate map and 3D colored coordinate model is then performed for 3D object registration in an AR environment. In particular, self-training is performed for fine-tuning the deep learning model by generating pseudo-labels with the highest probability for mask intersection over union (IoU) in the unlabeled real object dataset. We compared the proposed approach with previous studies to confirm its effectiveness and advantages. Also, we performed ablation studies by training a custom dataset based on various industrial 3D virtual models. Finally, we present AR-based task guidance and assistance, such as industrial robot manipulation. [ABSTRACT FROM AUTHOR]

The advancement of technology towards the realization of the evolving mobile computing paradigm brings a rapid paradigm shift in its usage, especially in the Internet, computation, and communications, that has a profound impact on businesses, services, and users. With the rise in resource-intensive or edge-based mobile applications such as autonomous driving, Amazon Go, virtual and augmented reality, and healthcare-related applications, countless challenges in computation and communication parameters like latency, bandwidth, and energy consumption are evolving. As a result, the multi-access edge computing (MEC) paradigm receives enormous attention where some portions of the user applications are offloaded to powerful machines for their efficient execution to optimize different evaluation metrics or to achieve performance goals. While a few survey works are available in this direction, none of them focuses explicitly on the emerging machine learning (ML) based computation offloading techniques and various associated sub-problems together. This paper aims to provide a detailed but precise overview of the research on using ML techniques for MEC environments. In this survey, we focus on how authors and researchers utilize the ML models in computation offloading problems on MEC architecture. We extend our study by considering several edge architectures, offloading parameters, ML approaches, and problem formulation strategies concerning computation offloading. Additionally, this paper discusses the potential challenges in the direction of computation offloading on MEC architecture. [ABSTRACT FROM AUTHOR]

Cutting-edge flood visualisation technologies are becoming increasingly important in managing urban flood risks, particularly from the perspective of stakeholders who play a crucial role in controlling and reducing the risks associated with flood events. This review study provides a comprehensive overview of stakeholder analysis in this context, highlighting gaps in current research and paving the way for future investigations. For this purpose, scientific literature and critical analysis are conducted based on identified relevant research works to map the mutual role of stakeholders in this context. This study categorises cutting-edge technologies into four groups - virtual reality, augmented reality, mixed reality, and digital twin - and explores their adoption in engaging various stakeholders across the five key stages of risk management: prevention, mitigation, preparation, response, and recovery. Results show that existing research has primarily concentrated on the support to water utilities and the communication with the general public. However, there is a noticeable gap in research regarding the comprehensive engagement of important stakeholders such as policy-makers, researchers, and insurance providers. Furthermore, the study highlights disparities in the involvement of stakeholders in damage assessment studies, particularly with a lack of representation from policy-makers and researchers. Finally, the study introduces the concept of overlooked key stakeholders and the interconnected impacts they have, which has received relatively little attention in previous research. [ABSTRACT FROM AUTHOR]

The perception of orientation in augmented reality, robot grasping, and 3D scene understanding is commonly addressed through the utilization of hand-crafted geometric features. However, machines can also learn the inherent orientation of 3D point clouds through experience, similar to humans. In this paper, we propose a self-supervised spherical vector network that is rotation-equivariant. Specifically, we use density-aware adaptive sampling to construct spherical signal samples to handle distorted point distributions in spherical space. Spherical convolutional vector layers and spherical routing layers are proposed to extract the rotation-equivariant vectors that represent the existence probability of the entity and orientations. Our method learns rotational representations from 3D point clouds through a self-supervised training process. We also provide theoretical proof that our proposed spherical vector networks are rotation-equivariant. Experiments on a variety of public datasets directly and indirectly demonstrate the effectiveness of the proposed method for canonical orientation estimation, even on unknown classes. [ABSTRACT FROM AUTHOR]

Thermotropic liquid crystals have recently become popular for use in optical components for augmented and virtual reality. Chiral liquid crystal (CLC), is of particular interest due to the formation of a photonic bandgap that allows for the creation of polarization-sensitive reflective optical components. Previous investigations have studied 2D diffraction gratings based on long-pitch (5 μm) CLC in combination with a 2D periodic alignment configuration. This yielded 2D diffraction patterns in transmission with the ability to switch between high and low voltage states with hysteresis. In the current study, short-pitch CLC (400 nm and 1 μm) is deposited between two substrates with perpendicularly rotating periodic photoalignment patterns, the shorter pitch resulting in a diffraction grating that produces a 2D diffraction pattern in reflection for wavelengths in the visible range. While a relatively high diffraction efficiency is achieved in the first diffraction order, additional diffraction orders with lower efficiency are present due to the complex CLC configuration in the bulk. • 2D liquid crystal diffraction gratings have been fabricated using a photo-patterned alignment and chiral liquid crystal. • Periodicity in the helical structure of the chiral liquid crystal allows reflection in the visible wavelength range. • Two different states have been discovered in the bulk of the short pitch chiral liquid crystal based 2D diffraction gratings. • Neither diffraction pattern nor the diffraction efficiency depends on the structure in the bulk. • Around the mid-plane, transition layer exists, where in some regions along the diagonal director is close to the z-axis. [ABSTRACT FROM AUTHOR]

Creating realistic digital 3D avatars has been getting more attention thanks to the introduction of new multimedia formats such as augmented and virtual reality. An important factor making avatars realistic is clothes. In this paper, we investigate a new method to reconstruct realistic garments from a set of images and body information. Early methods working on realistic images struggle to faithfully reconstruct the garment details. As deep learning is increasingly applied to geometric data which can conveniently represent garments, we devise a novel deep learning-based solution to the garment reconstruction problem. We offer a new perspective on the reconstruction problem and treat it as a reversion of the smoothing diffusion process. To achieve this goal, we propose to deform the smoothed human mesh into a clothed human via a Double Reverse Diffusion (DReD) process. For the first reverse diffusion, we introduce a novel operator called Graph Long Short-Term Memory (GraphLSTM) which recursively diffuses features to produce a deformed mesh by modeling the relationships between vertices. Then, the output mesh can be repeatedly upsampled and deformed by the above pipeline to obtain finer garment details, which can be seen as another reverse diffusion process. To obtain features for the reverse diffusion, we extract pixel-aligned features transferred from images and explore to incorporate the visibility of garments from the image viewpoints. Through detailed experiments on two public datasets, we demonstrate that DReD synthesizes more realistic wrinkled garments with lower errors and offers faster inference than previous methods. [ABSTRACT FROM AUTHOR]

• An ultra-wide-angle waveguide-based NED to challenge the hegemony of magnifier-based NEDs in the VR sector. Its FOV is 100° (80° (H) × 80° (V)). • Change in the mindset about the placement of in-couplers. Instead of placing the in-couplers inside the FOV region, we relocate them to the sub-FOV regions. • One more channel added to the FOV division. The multi-channel architectures are coming to the fourth iteration, i.e. , quad-channel. • Comparison between the sawtooth grating and slanted grating. Diffraction efficiency and fill factor modulation wise, slanted grating wins. Spectral bandwidth, angular bandwidth and polarization sensitivity wise, sawtooth grating wins. We present a quad-channel waveguide-based near-eye display as an ultra-wide-angle architecture for the metaverse. The core concept is to divide one field of view into four by placing the couplers within the regions, where only the subsets of field of view are located. Compared to its counterparts, including the single-, double- and triple- channels, our quad-channel waveguide can push the envelope of field of view further. With the aid of k -space diagram, the upper limit of field of view is illustrated and deduced. The design rules of the waveguide, sawtooth grating as the in-coupler, and slanted grating as the out-coupler are expounded. Through the rigorous coupled-wave analysis, the diffraction efficiencies of gratings can be calculated and optimized. As an overall evaluation, its key performance indicators are summarized as follows. Field of view is 100° (diagonal), eye relief is 10 mm, exit pupil is 11.5 × 11.5 mm2, average transmittance is 2.56 %, and average uniformity is 75 %. [ABSTRACT FROM AUTHOR]

Low situational awareness contributes to safety incidents in construction. Existing Deep Learning (DL)-based applications lack the capability to provide context-specific and interactive feedback that is essential for workers to fully understand their surrounding environments. This paper proposes the Visual Construction Safety Query (VCSQ) system. The system encompasses real-time Image Captioning (IC), safety-centric Visual Question Answering (VQA), and keyword-based Image-Text Retrieval (ITR), integrated with head-mounted Augmented Reality (AR) devices. System validation includes benchmarks and real-world images. The ITR module posted high recall rates of 0.801 and 0.835 for Recall@5 and @10. The VQA module achieved an 89.7% accuracy rate, and the IC module had a SPICE score of 0.449. Feasibility tests and surveys confirmed the system's practical advantages in different construction scenarios. This study establishes an integration roadmap adaptable to future advancements in interactive DL and immersive AR. • A vision-language system is developed for real-time captions and safety queries. • A dataset is created containing 7 safety considerations for unsafe situation query. • An intuitive AR application is incorporated for an immersive user experience. • The system achieves 89.7% accuracy and 83.5% recall rates against benchmark datasets. [ABSTRACT FROM AUTHOR]

Indoor wayfinding poses significant challenges, as navigating complex environments can be difficult and time-consuming. Augmented reality (AR) has emerged as a promising solution to address these challenges. However, understanding how to effectively leverage AR techniques for large-scale indoor wayfinding scenarios remains unclear. In particular, the implications of AR-enabled egocentric perspectives have not been thoroughly investigated. This study aims to explore the potential benefits of AR-based egocentric perspectives and their impact on indoor wayfinding performance. A human-subject experiment (N = 31) was conducted in a multi-story building, utilizing a self-developed AR system capable of providing both traditional exocentric and innovative egocentric perspectives. The results of the experiment confirmed the performance and cognitive benefits of the AR-enabled egocentric perspective. Participants using the egocentric perspective demonstrated improved wayfinding efficiency, reduced cognitive load, and enhanced spatial awareness. These findings suggest that the AR-based egocentric perspective offers a more intuitive and effective approach to indoor wayfinding, enabling users to better process spatial information and make informed decisions during wayfinding tasks. The findings are expected to provide empirical evidence on the efficacy of AR systems for wayfinding and inspire new user interface (UI) designs that would better leverage the advantages of AR techniques. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) technology is a promising tool for field service applications such as assembly. AR can show computer-aided design (CAD) models of assembly parts at installation locations. This provides a means for delivering assembly instructions to an operator. Furthermore, evaluating the assembly process can be beneficial for preventing assembly errors. Field service engineers who perform assembly operations may choose to use a head-mounted display (HMD) for its good portability and hands-free operation. In this study, we propose a method for efficiently evaluating the misalignment between real and virtual objects, which is a mismatch between the two objects' poses, in an HMD-based AR assembly assistance system. By using a depth camera in the HMD, we evaluate the misalignment by comparing the depth maps of the real and virtual worlds. With our preliminary prototype, the system can evaluate misalignment with an accuracy of ±1 cm at a rate of 30 fps in real time. Moreover, we also propose a coordinate calibration method and establish a basic HMD-based AR assembly assistance system to demonstrate the methods. The efficient evaluation of misalignment could help to monitor operations to prevent assembly errors and decrease operation time. [ABSTRACT FROM AUTHOR]

This research is interested in the effect of augmented reality (AR) technology on ecolabels' effectiveness. Based on a conceptual model, including control variables, the authors tested the effect of augmented and enriched information on evaluating messages around ecolabels and purchase intention for eco-labeled products. A between-subject experiment on fair-trade coffee involving 264 Canadian consumers was conducted. The experimental plan includes five cells: 2 (augmented reality: absence vs. presence) × 2 (information richness: low vs. high) + 1 control group (absence of ecolabel). The results confirm that AR and enriched information significantly increase the perceived quality and credibility of information around ecolabels, as well as purchase intention for eco-labeled products. Moreover, a significant interaction effect between the medium and the level of information detail was noted for purchase intention. This effect is influenced by the perceived novelty and coolness of the packaging, the degree of the perceived complexity of AR, the level of consumer involvement with the product, and income. [ABSTRACT FROM AUTHOR]

• The proposed solution can reconstruct indoor scenes using only a single frame and an RGB camera. • An indoor scene composed by five objects can be reconstructed in just 21 seconds. • The proposed solution obtains a maximum error of 18% for the scaling factor, less than 9cm for the position and less than 18 ∘ for the rotation. • The innovative methodology can be employed for VR and AR applications, thus bridging the gap between the real and virtual words. [Display omitted] Indoor environment reconstruction is a challenging task in Computer Vision and Computer Graphics, especially when Extended Reality (XR) technologies are considered. Current solutions that employ dedicated depth sensors require scanning of the environment and tend to suffer from low resolution and noise, whereas solutions that rely on a single photo of a scene cannot predict the actual position and scale of objects due to scale ambiguity. The proposed system addresses these limitations by allowing the user to capture single views of objects using an Android smartphone equipped with a single RGB camera and supported by Google ARCore. The system includes 1) an Android app tracking the smartphone's position relative to the world, capturing a single RGB image for each object and estimating depth information of the scene, 2) a program running on a server that classifies the framed objects, retrieves the corresponding 3D models from a database and estimates their position, vertical rotation, and scale factor without deforming the shape. The system has been assessed measuring the translational, rotational and scaling errors of the considered objects with respect to the physical ones acting as a ground truth. The main outcomes show that the proposed solution obtains a maximum error of 18% for the scaling factor, less than nine centimeters for the position and less than 18 ∘ for the rotation. These results suggest that the proposed system can be employed for XR applications, thus bridging the gap between the real and virtual worlds. [ABSTRACT FROM AUTHOR]

One key predictor of team performance and productivity is the teams' ability to coordinate its subtasks as precisely as possible over time. Thereby, the quality of the temporal coordination in teams is highly dependent on several cognitive team skills, like for instance the ability to build a precise and stable mental model of the situation (shared situation awareness) and of the team task process (task state awareness/ TSA). To support these cognitive skills in teams, various methods and trainings already exist. However, considering teams that work de-located, the prerequisites and therefore the requirements for team supportive methods change. This work presents the results of an empirical usability and user experience (UX) study of an interface for an Augmented Reality-based assistance system for spatially dispersed teams, called Ambient Awareness Tool (AAT). The AAT interface consists of graphical information about the teamwork process and aims at enhancing the TSA of the team, thereby supporting its temporal coordination. Within the framework of a participatory design process we conducted a two-part expert-user-study, comprising a laboratory experiment for the evaluation of the interface usability and a follow-up online survey to additionally investigate the UX. By means of the usability scores we first inferred three interface configuration clusters. A subsequent UX evaluation then allowed us to designate the configuration cluster with the highest usability and UX scores. As a result, we defined one interface configuration that will be investigated concerning its actual impact on the temporal coordination of spatially dispersed teams in a further study. [ABSTRACT FROM AUTHOR]