Search

Researchers have indicated that integrating augmented reality into robot-assisted language learning modules could visually represent the students' learning needs and extend their embodied cognition of comprehension. To examine the potential of robots through extended embodied cognition, this quasi-experimental study was conducted in the English for Medical Purposes classrooms at the Isfahan University of Medical Sciences. In the 2022 academic year, 526 male and female participants, whose first language is not English, were selected. Students were sorted into three English proficiency levels: pre-intermediate (n = 176), intermediate (n = 175), and upper-intermediate (n = 175), then semi-randomly divided into three-member teams, subsequently, randomly assigned to control (n = 370) or experimental (n = 156) groups. Students in the control group learned English for Medical Purposes listening and reading through online classrooms, while those in the experimental group learned the skills in robot (augmented reality)-assisted classrooms. In the control and experimental groups, flipped classrooms were conducted under the supervision of the teachers. The primary data sources included formative assessments of the participants' English for Medical Purposes listening and reading, as well as interviews. The findings showed that the participants in the robot's augmented reality group achieved significantly higher results, when compared with robot-only and control groups, in English for Medical Purposes listening and reading in academia and the healthcare fields. The positive perception participants had of robot's augmented reality was clear, based on interview results. The outcomes are discussed in detail. • Robot-aided modules facilitated English for Medical Purposes listening and reading. • The glossed robot fostered connection with the world. • Robot's AR enhanced extended embodied cognition for comprehension. • Robot's AR fostered variegated channels for comprehension. • Robot's AR facilitated listening and reading in the fields. [ABSTRACT FROM AUTHOR]

The rapid development of augmented reality (AR), 3D reconstruction, simultaneous localization and mapping (SLAM), and autonomous driving requires off-the-shelf camera calibration solutions that are adaptable to cameras of different configurations in different complex scenarios. To this end, we propose a generic, robust, and accurate camera calibration framework, called Meta-Calib, by using single or multiple novel designed ArUco-encoded meta-board(s), which is dedicated to estimate accurate camera intrinsic parameters and extrinsic transformations of different multi-camera configurations. The ArUco calibration board has been redesigned to facilitate learning-based robust detection and obtain higher precision control point coordinates, which is termed the meta-board. This completely replaces the widely-used chessboard based on the corner extraction scheme to greatly alleviate the impact of image distortion on control points, especially when it is located at the boundary area of the fish-eye camera. A robust two-stage deep learning detection strategy is applied to reliably localize the ArUco-encoded inner coding region of the meta-board followed by identifying two categories of circular shapes representing "0" and "1" encoded in the ArUco pattern for decoding and orientation determination. The center points of circular shapes on the meta-board in the distorted image taken under the perspective view can be approximated through elliptical fitting with contour edges. The deviation between the fitting center points and ground-truth can be greatly suppressed when the refined sub-pixel contour edges extracted on the original image are projected to the orthographic projection view based on the camera intrinsic parameters, distortion coefficients and the prior information of the meta-board. Based on this observation, we propose a systematic iterative refinement approach to achieve the high-precision intrinsic calibration of a camera. This process involves improving the estimation of camera intrinsic parameters and fitting the center control points of circular shapes on the meta-boards in an iterative manner. The progressive nature of our approach permits reliably calibrate large distortion camera models under the presence of noisy measurements, which ensures good convergence. In addition, we also propose a graph-based multi-camera extrinsic calibration method via the corrected control points to reliably estimate both the relative poses of the meta-boards and cameras in the multi-camera system. The proposed method is not constrained by the number of cameras and meta-boards used, which makes our strategy accessible even with inflexible computer vision experts. Furthermore, we have derived the mathematical form for computing the covariance of the extrinsic transformation, which makes it possible to evaluate the uncertainty of the calibration results. Extensive experiments on a large number of both real and synthetic datasets, including perspective, fish-eye, and multiple overlapping cameras, are performed to prove the effectiveness and robustness of the developed Meta-Calib calibration framework. [ABSTRACT FROM AUTHOR]

Astronaut training is complex due to its specific infrastructure and equipment requirements, which are often costly. Immersive technologies can simulate entire environments, allowing for interactive user experiences. Previous research has confirmed the effectiveness of immersive technologies in training and learning. However, simulations alone are insufficient; participants must also engage with vehicles, machinery, and spacecraft systems to understand their impact on the environment and adjacent objects. This study develops four scenarios within three modules, inspired by NASA and ESA astronaut training programs. The first module presents a theoretical scenario using mixed reality (MR), incorporating topics from ESA's training modules. The second module offers two practical scenarios: an ISS emergency simulation that includes Hohmann transfer and circular motion concepts, and a spacewalk for repair tasks. In these scenarios, digital twins of the ISS propulsion and navigation systems, as well as a spacesuit, were created. The third module simulates a spacecraft launch, utilizing 3D models from SpaceX's Falcon Heavy and digital twins of its propulsion and navigation systems. Participants interacted with the digital twins and scenarios, generating data that was stored and analyzed against the ISS dataset from the Jet Propulsion Laboratory (JPL) and telemetry from the SpaceX Falcon Heavy launch. In this work's initial phase, the main objective was to assess whether digital twins could be integrated with immersive technologies for effective training. A mean squared error metric was employed to compare the digital twins with the physical object data, confirming the alignment of the developed digital twins with the actual systems, thereby validating their utility for astronaut training. • Mixed reality and digital twins can be used to improve astronaut's training. • Mixed reality scenarios are a safe way to learn and train about a specific topic. • Mixed reality foster training experiences: Promoting creative problem-solving skills. • Digital Twins enhances immersive technologies: Leveraging digital twins capacities for immersive training. [ABSTRACT FROM AUTHOR]

Precise planning and execution is key for neuroendoscopic interventions, which can be based on different available aiding technologies. The aim of this retrospective study is to report a case-based use of guided neuroendoscopy and to develop a stratification algorithm for the available technologies. We reviewed consecutive neuroendoscopic cases performed at our center from 2016 to 2018. We distinguished between patients receiving a new burr hole (group A) and those with a preexisting burr hole (group B). Case-specific technical requirements for procedure planning and execution, complication rate, surgical outcome, and possible subsequent surgery were evaluated. From this experience, a stratification system was developed to tailor the available guiding technologies. A total of 309 neuroendoscopic interventions in 243 patients were included in the present study. The cases included hydrocephalic (81.6%) and nonhydrocephalic (18.4%) conditions. The interventions were supported by coordinate-based (group A, n = 49; group B, n = 67), guide-based (group A, n = 42; group B, n = 0), ultrasound-guided (group A, n = 50; group B, n = 7), or navigated augmented reality-guided (group A, n = 85; group B, n = 9) techniques. The overall complication rate was 4.5%. Stratified by the surgical indication, fontanel status, entry point localization, presence of a preexisting burr hole, ventricular size, and number of targets, an approach toward image-guided neuroendoscopy is suggested. Planning and technical guidance is essential in neuroendoscopic procedures. The stratified decision-making algorithm for different available technologies aims to achieve lower cost and time consumption, which was found to be safe and efficient. Further investigations are warranted to deliver solid data on procedure efficiency. [ABSTRACT FROM AUTHOR]

Patients with incurable corneal diseases experience visual impairment (VI) despite having a healthy retina and optic pathway. Low-vision aids (LVAs) can optimize the use of remaining vision through magnification and contrast enhancement, but do not harness the full visual capacity because they rely on the optic media. Therefore, we investigated a novel laser eyewear (LEW) technology that bypasses the anterior segment of the eye. Images captured by an integrated camera are projected directly onto the retina using a low-energy laser. The patient is able to view a full-color video, realized as augmented reality. We aimed to evaluate the efficacy of the LEW to enhance the vision of individuals with corneal diseases. Prospective, randomized, crossover clinical trial. We examined the retinal projection glasses in 21 patients (25–69 years) with VI (0.7 logarithm of the minimum angle of resolution [logMAR] or worse) resulting from corneal diseases. Patients with comorbidities that impact vision, such as retinal disorders, were excluded. Standardized measurements of visual acuity (VA) for near vision (NV) and distance vision (DV) were conducted using ETDRS charts with the respective best correction (BC) and then with LEW. In addition reading speed, vision-related quality of life (QoL) and capacity to carry out daily tasks were assessed at an initial visit and at 2 subsequent visits after a home phase with and without the device. Six weeks after last use of the LEW, an ophthalmologic examination including spectral-domain-OCT or full-field-electroretinography was conducted and compared with baseline findings to evaluate the safety of the device. Four patients participated and completed a subsequent 12-month follow-up phase. Improvement of VA using the LEW. Secondary objectives included safety, reading speed, QoL, and usability in daily activities. The mean VA in patients with VI was improved by 0.43 logMAR in DV using the LEW compared with BC (P < 0.0001). Using the ×2 magnification mode of the LEW resulted in an average improvement of 0.66 logMAR compared with BC (P < 0.0001). In NV, an increase of 0.47 logMAR was achieved compared with BC (P < 0.0001). Although only 4 of 21 participants were able to read with BC, 17 of 21 participants were able to read with the LEW. Quality of life significantly improved in the 17 participants who completed all visits. We demonstrated that the retinal projection glasses resulted in enhanced VA for all participants by directly projecting images onto the intact retina. In future, the LEW could represent a new option as an LVA for patients with corneal diseases. No pathological alterations were observed in the safety assessments. The author(s) have no proprietary or commercial interest in any materials discussed in this article. [ABSTRACT FROM AUTHOR]

Objectively examine the effect of 3D-Augmented Reality anatomic review on craniotomy planning among neurosurgical residents as it pertains to craniotomy size, skull positioning, and knowledge of significant anatomic relationships. Postgraduate year 1-7 neurosurgery residents were instructed to review standard 2D radiographs, pin a skull, and tailor a craniotomy for 6 different lesions and case vignettes. Participants then reviewed the lesion in a 3D-augmented reality (AR) environment, followed by repeating the craniotomy station for a variety of lesion types and locations (superficial, subcortical, deep, skull base). Quiz with case-specific anatomic and surgical questions followed by an exit survey for qualitative impressions. Eleven of thirteen eligible residents participated. Skull position significantly changed in 5 out of 6 cases after 3D-AR view (P < 0.05, 20° angular adjustment). No significant change in incision length or craniotomy size. Subgroup analysis of junior versus senior residents revealed that craniotomy size was significantly altered in 2 out of 6 cases. Qualitative testimonials (Likert scale 5 = strongly agree) reported a change in craniotomy approach after 3D-review (3.5), improved appreciation of anatomy (4.2), increased confidence in surgical approach (4.33 junior residents, 3.5 senior residents), smaller incision (3.5 junior residents, 1.75 senior residents), better appreciation of white matter tracts (4.6). The augmented reality platform offers a medium to examine surgical planning skills. Residents uniformly appreciated 3D-AR as a valuable tool for improving appreciation of critical anatomic structures and their relationship to lesional pathology. 3D-AR review significantly altered skull positioning for various lesions and craniotomy approaches, particularly among junior residents. [ABSTRACT FROM AUTHOR]

Neurosurgical technologies have become increasingly more adaptive, featuring real-time and patient-specific guidance in preoperative, intraoperative, and postoperative settings. This review offers insight into how these integrative innovations compare with conventional approaches in spine surgery, focusing on machine learning (ML), artificial intelligence, augmented reality and virtual reality, and spinal navigation systems. Data on technology applications, diagnostic and procedural accuracy, intraoperative times, radiation exposures, postoperative outcomes, and costs were extracted and compared with conventional methods to assess their advantages and limitations. Preoperatively, augmented reality and virtual reality have applications in surgical training and planning that are more immersive, case specific, and risk-free and have been shown to enhance accuracy and reduce complications. ML algorithms have demonstrated high accuracy in predicting surgical candidacy (up to 92.1%) and tailoring personalized treatments based on patient-specific variables. Intraoperatively, advantages include more accurate pedicle screw insertion (96%–99% with ML), enhanced visualization, reduced radiation exposure (49 μSv with O-arm navigation vs. 556 μSv with fluoroscopy), increased efficiency, and potential for fewer intraoperative complications compared with conventional approaches. Postoperatively, certain ML and artificial intelligence models have outperformed conventional methods in predicting all postoperative complications of >6000 patients as well as predicting variables contributing to in-hospital and 90-day mortality. However, applying these technologies comes with limitations, such as longer operative times (up to 35.6% longer) with navigation, dependency on datasets, costs, accessibility, steep learning curve, and inherent software malfunctions. As these technologies advance, continuing to assess their efficacy and limitations will be crucial to their successful integration within spine surgery. [ABSTRACT FROM AUTHOR]

• It is important to involve alpha generation in the conservation of urban heritage as they will be the next generation responsible for its preservation. • The use of augmented reality based on serious game and GPS-guided applications has proven to be effective for heritage conservations. • This research describes a serious game based on the use of GPS-guided augmented reality in a renaissance village. It is important to involve young people in the conservation of urban heritage as they will be the next generation responsible for its preservation. The use of Augmented Reality (AR) based on Serious Game (SG) and GPS-guided applications has proven to be effective for heritage revitalisation. However, there are currently only a few applications that combine GPS AR and SG for urban heritage enhancement, and these are generally aimed at adult tourists rather than the young people of the alpha generation. This research describes a serious game based on the use of GPS-guided augmented reality in the Renaissance village of Caldana, Italy, to raise awareness of the importance of urban heritage and to stimulate its visit by the alpha generation. The application consists of a treasure hunt that children have to solve by meeting and listening in augmented reality to historical characters who built the Renaissance village at specific points of interest, following a route with increasing levels of difficulty. The application was tested using a questionnaire with a sample of 79 children. The results provide useful indications for the development of augmented reality applications to increase the awareness and attitudes of the alpha generation towards knowledge and conservation of urban heritage. [Display omitted] [ABSTRACT FROM AUTHOR]

In the last decade, there has been a great effort in developing new technologies to enhance surgical visualization and guidance. This comprehensive and narrative review aimed to provide a wide and extensive overview of the current state of the art on this topic and their near-future perspectives linked to the development of artificial intelligence (AI), by focusing on the most recent and relevant literature. A comprehensive and narrative review of the literature was performed by searching specific terms on PubMed/MEDLINE, Scopus, and Embase databases, assessing the current state of the art on this topic. Fluorescence-guided surgery, contrast-enhanced ultrasound (CEUS), ultra-high frequency ultrasound (UHFUS), photoacoustic imaging (PAI), and augmented reality (AR) are boosting the field of image-guided techniques as the rapid development of AI in surgery is promising a more automated decision-making and surgical movements in the operating room. Fluorescence-guided surgery, CEUS, UHFUS, PAI, and AR are becoming crucial to give surgeons a new level of information during the intervention, with the right timing and sequence, and represent the future of surgery. As many more controlled studies are needed to validate the employment of these technologies, the next generation of surgeons must become more familiar with the basics of AI to better incorporate new tools into the daily surgical practice of the future. [ABSTRACT FROM AUTHOR]

For many applications, remote guidance and telerobotics provide great advantages. For example, tele-ultrasound can bring much-needed expert healthcare to isolated communities. However, existing tele-guidance methods have serious limitations including either low precision for video conference-based systems, or high complexity and cost for telerobotics. A new concept called human teleoperation leverages mixed reality, haptics, and high-speed communication to provide tele-guidance that gives an expert nearly-direct remote control without requiring a robot. This paper provides an overview of the human teleoperation concept and its application to tele-ultrasound. The concept and its impact are discussed. A new approach to remote streaming and control of point-of-care ultrasound systems independent of their manufacturer is described, as is a high-speed communication system for the HoloLens 2 that is compatible with ResearchMode API sensor stream access. Details of these systems are shown in supplementary video demonstrations. Novel interaction methods enabled by HoloLens 2-based pose tracking are also introduced and tests of the communication and user interaction are presented. The results show continued improvement of the system compared to previous work in instrumentation, HCI, and communication. The system thus has good potential for tele-ultrasound, as well as possible other applications of human teleoperation including remote maintenance, inspection, and training. The remote ultrasound streaming and control application is made available open source. [Display omitted] • System improvements to human teleoperation demonstrate its feasibility. • Other devices such as the Nreal Light can be used for implementation. • New device-agnostic remote ultrasound streaming and control demonstrated. • HoloLens pose tracking enables human teleoperation with limited compute resources. • HoloLens-based communication system provides effective sensor data streaming. [ABSTRACT FROM AUTHOR]

Augmented Reality (AR) is a key technology in the transition to Industry 4.0 and smart manufacturing, gaining reputation in a wide range of industrial fields One promising application is in another field of Industry 4.0, the data monitoring, where AR can be used to visualize and interact with complex and big data in real time, potentially improving the efficiency and accuracy of decision-making In this work, we propose a Pervasive AR tool for data monitoring, developed in collaboration with industry partners. An web application of the same data monitoring function is also created for comparison purposes A Human-Centered Design (HCD) methodology was used to identify the needs and requirements of industrial analysts, which led to the development of these systems Preliminary user studies, with 17 participants having distinct levels of expertise in industry, data analysis and computer science, were conducted to collect opinions and suggestions for improvements, as well as, quantitative data regarding the technologies considered A succeeding user study was then prepared, in which 12 participants used two conditions (C1 — Hands-free Pervasive AR tool for HMDs and C2 — Web tool for tablet devices) to fulfill a set of data monitoring tasks in an industry assembly line The result of these studies confirm the potential of Pervasive AR for data monitoring, as it engages users, promote environmental awareness, contextualizes data and allows fast self-localization. [Display omitted] • Two user studies with more than 10 participants each, industrial settings tasks. • Understand which method (Web or AR) stands out according to the task characteristic. • The findings reveal that certain methods were more effective for types of task. • Display methods significantly influenced participants across analysis. • These studies confirm the potential of Pervasive AR for data monitoring. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) is an emerging technology in neurosurgery with the potential to become a strategic tool in the delivery of care and education for trainees. Advances in technology have demonstrated promising use for improving visualization and spatial awareness of critical neuroanatomic structures. In this report, we employ a novel AR registration system for the visualization and targeting of skull landmarks. A markerless AR system was used to register 3-dimensional reconstructions of suture lines onto the head via a head-mounted display. Participants were required to identify craniometric points with and without AR assistance. Targeting error was measured as the Euclidian distance between the user-defined location and the true craniometric point on the subjects' heads. All participants successfully registered 3-dimensional reconstructions onto the subjects' heads. Targeting accuracy was significantly improved with AR (3.59 ± 1.29 mm). Across all target points, AR increased accuracy by an average of 19.96 ± 3.80 mm. Posttest surveys revealed that participants felt the technology increased their confidence in identifying landmarks (4.6/5) and that the technology will be useful for clinical care (4.2/5). While several areas of improvement and innovation can further enhance the use of AR in neurosurgery, this report demonstrates the feasibility of a markerless headset-based AR system for visualizing craniometric points on the skull. As the technology continues to advance, AR is expected to play an increasingly significant role in neurosurgery, transforming how surgeries are performed and improving patient care. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) is an emerging technology that may accelerate skill acquisition and improve accuracy of thoracolumbar pedicle screw placements. We aimed to quantify the relative assistance of AR compared with freehand (FH) pedicle screw accuracy across different surgical experience levels. A spine fellowship–trained and board-certified attending neurosurgeon, postgraduate year 4 neurosurgery resident, and second-year medical student placed 32 FH and 32 AR-assisted thoracolumbar pedicle screws in 3 cadavers. A cableless, voice-activated AR system was paired with a headset. Accuracy was assessed using χ2 analysis and the Gertzbein-Robbins scale. Angular error, distance error, and time per pedicle screw were collected and compared. The attending neurosurgeon had 91.6% (11/12) clinically acceptable (Gertzbein-Robbins scale A or B) insertion in both FH and AR groups; the resident neurosurgeon had 100% (9/9) FH and AR in both cases; the medical student had 72.3% (8/11) FH accuracy and 81.8% (9/11) AR accuracy. The medical student displayed significantly lower ideal (Gertzbein-Robbins scale A) FH accuracy compared with the resident neurosurgeon (P = 0.017) and attending neurosurgeon (P = 0.005), but no difference when using AR. FH screw placement was faster by both the attending neurosurgeon (median 46 seconds vs. 94.5 seconds, P = 0.0047) and the neurosurgery resident neurosurgeon (median 144 seconds vs. 140 seconds, P = 0.05). Total clinically acceptable AR and FH accuracy was 90.6% (29/32) and 87.5% (28/32), respectively (P = 0.69). AR screw placement allowed an inexperienced medical student to double their accuracy in 1 training session. With subsequent iterations, this promising technology could serve as an important tool for surgical training. [ABSTRACT FROM AUTHOR]

Recently, I was at a Microsoft Technology Center to discuss the augmented reality (AR) technology developed in my lab for treatment of post-traumatic stress disorder (PTSD). We also explored the possibility of using artificial intelligence (AI) to automate interactions with digital AR characters. We were also given a tour of the Microsoft 365 Copilot AI technology. This experience, along with my upcoming experiences with the rapidly evolving and publicly available AI technology, triggered a series of questions and reflections about the use of AI in scientific work and writing. In this article, the author reflects upon recent advancements in artificial intelligence (AI) technologies have led to profound discussions about AI's role in scientific research and education. AI technologies are now capable of summarizing and analyzing large volumes of data, creating presentations, and even drafting parts of scientific papers and grants with minimal human input. With their exponential ongoing advancements, they will become even more capable in doing these tasks. As boundaries between humans and AI blurs, serious questions arise about the future roles, responsibilities, and identity of academic researchers, intellectual property, publishing and grantsmanship, and equity in the world of AI potentiated research. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) is increasingly recognized as a valuable tool in spine surgery. Here we provides an overview of the key developments and technological milestones that have laid the foundation for AR applications in this field. We also assess the quality of existing studies on AR systems in spine surgery and explore potential future applications. The purpose of this narrative review is to examine the role of AR in spine surgery. It aims to highlight the evolution of AR technology in this context, evaluate the existing body of research, and outline potential future directions for integrating AR into spine surgery. Narrative review. We conducted a thorough literature search to identify studies and developments related to AR in spine surgery. Relevant articles, reports, and technological advancements were analyzed to establish the historical context and current state of AR in this field. The review identifies significant milestones in the development of AR technology for spine surgery. It discusses the growing body of research and highlights the strengths and weaknesses of existing investigations. Additionally, it presents insights into the potential for AR to enhance spine surgical education and speculates on future applications. Augmented reality has emerged as a promising adjunct in spine surgery, with notable advancements and research efforts. The integration of AR into the spine surgery operating room holds promise, as does its potential to revolutionize surgical education. Future applications of AR in spine surgery may include real-time navigation, enhanced visualization, and improved patient outcomes. Continued development and evaluation of AR technology are essential for its successful implementation in this specialized surgical field. [ABSTRACT FROM AUTHOR]

Through providing on-demand visual and auditory cues while walking,augmented reality (AR) can theoretically cue spatiotemporal gait adaptations in, populations such as those with Parkinson's disease. However, given the novelty of the, technology, the type and extent of gait adaptations in response to such a cueing, system are unknown. Before such systems can be incorporated into rehabilitation, approaches, it is important to understand how people interact with the technology. Research questions: What are the effects of visual and auditory cues delivered, through AR on spatiotemporal walking patterns and variability in a healthy, young, population? Is there a relationship between system usability and gait variability?, Twenty healthy, young participants walked in four different cueing conditions using an AR headset: No Cues (NC) (i.e., natural gait), Auditory (A), Visual (V), and Auditory + Visual (AV). Inertial measurement units recorded spatiotemporal gait data at 200 Hz, a System Usability Survey was administered afterward, and linear regression models were developed to examine whether gait variability is predictive of system usability. All cueing conditions exhibited a significantly slower cadence compared to, NC trials. Cadence variability was significantly higher for A trials compared to V and, NC. V trials exhibited significantly decreased stride lengths compared to NC. Increased, reported system usability was significantly correlated with decreased stance phase, time variability across A trials. Our findings support that holographic spatial-visual and auditory cues, are promising to evoke spatiotemporal gait adaptations. Results also support the, notion that the type of system and cue delivery design may impact gait outcomes.,Before an AR cueing system can be applied to a specific population in future, interventions, a more holistic approach towards finding the relationship between this, technology and its users is needed. • Augmented reality cues elicit gait adaptations in healthy, young people. • Auditory cueing slows walking cadence regardless of cue type. • Auditory cues increase cadence variability. • Visual cues reduce cadence variability and increase unipedal stance time. • System usability is predicted by certain gait variability measures. [ABSTRACT FROM AUTHOR]

Virtual reality (VR) and augmented reality (AR) are rapidly developing technologies that may have important healthcare-related applications, including the possible evaluation of the health and performance of astronauts. Advances in the fields of VR/AR and space exploration have independently progressed; however, over the last decade synergies and overlapping applications have emerged between spaceflight and extended reality. This paper is intended to be a resource to update the medical community on the current and future integration of VR and AR into space exploration, particularly as this integration relates to astronaut health. We highlight how these technologies are used in terrestrial healthcare with specific attention to visual function. Furthermore, we elaborate on emerging VR applications being developed for future spaceflight. The paper discusses the current state of the art, such as the Apple Vision Pro headset, and outlines key challenges like mass and volume constraints, power limitations, and the heightened cognitive and physical demands of space travel. It also addresses the potential of VR/AR technologies to mitigate these issues through efficiency, stress reduction, and exercise motivation. Finally, we emphasize the benefits that these technologies provide to overcome possible barriers to long duration spaceflight and manned exploration missions, including spaceflight associated neuro-ocular syndrome (SANS). • Virtual reality (VR) and augmented reality (AR) are rapidly developing technologies that may have important healthcare-related applications, including the possible evaluation of the health and performance of astronauts. • This paper is intended to be a resource to update the medical community on the current and future integration of VR and AR into space exploration, particularly as this integration relates to astronaut health. • We highlight how these technologies are used in terrestrial healthcare with specific attention to visual function. Furthermore, we elaborate on emerging VR applications being developed for future spaceflight. [ABSTRACT FROM AUTHOR]

The perception of size in virtual objects in Augmented Reality (AR) and Virtual Reality (VR) is a not trivial issue, as the effectiveness of manipulating and interacting with virtual content depends on the accuracy of size perception. However, there are missing straightforward comparisons between VR and AR in terms of size perception for the deep understanding of size perceptual differences. Understanding these perceptual differences can inform designers on how to adapt content when transitioning between these two spatial computing platforms. In this paper, we conducted two psychophysical experiments to measure the perceptual thresholds of size discrimination for virtual objects. Our results indicated that users are more sensitive to size changes in VR than in video see-through AR, suggesting that size differences are easier to be perceived in VR than in AR. Additionally, for increase or decrease of sizes, the accuracy of judgments showed an asymmetric trend in video see-through AR. [Display omitted] • A comparative experiment to understand the difference in size perception in AR vs VR. • The thresholds for size discrimination in AR and VR are not the same. • The accuracy of judgments is asymmetric for increases and decreases of sizes in AR. [ABSTRACT FROM AUTHOR]

The accurate localization of intracranial lesions is critical in neurosurgery. Most surgeons locate the vast majority of neurosurgical sites through skull surface markers, combined with neuroimaging examination and marking lines. This project's primary purpose was to develop an augmented reality (AR) technology or tool that can be used for surgical positioning using the naked eye. Brain models were predesigned with intracranial lesions using computerized tomography scan, and Digital Imaging and Communications in Medicine data were segmented and modeled by 3D slicer software. The processed data were imported into a smartphone 3D viewing software application (Persp 3D) and were used by a Remebot surgical robot. The localization of intracranial lesions was performed, and the AR localization error was calculated compared with standard robot localization. After mastering the AR localization registration method, surgeons achieved an average localization error of 1.39 ± 0.82 mm. The error of AR positioning technology in surgical simulation tests based on brain modeling was millimeter level, which has verified the feasibility of clinical application. More efficient registration remains a need that should be addressed. [ABSTRACT FROM AUTHOR]

The retrieval of 3D objects has gained significant importance in recent years due to its broad range of applications in computer vision, computer graphics, virtual reality, and augmented reality. However, the retrieval of 3D objects presents significant challenges due to the intricate nature of 3D models, which can vary in shape, size, and texture, and have numerous polygons and vertices. To this end, we introduce a novel SHREC challenge track that focuses on retrieving relevant 3D animal models from a dataset using sketch queries and expedites accessing 3D models through available sketches. Furthermore, a new dataset named ANIMAR was constructed in this study, comprising a collection of 711 unique 3D animal models and 140 corresponding sketch queries. Our contest requires participants to retrieve 3D models based on complex and detailed sketches. We receive satisfactory results from eight teams and 204 runs. Although further improvement is necessary, the proposed task has the potential to incentivize additional research in the domain of 3D object retrieval, potentially yielding benefits for a wide range of applications. We also provide insights into potential areas of future research, such as improving techniques for feature extraction and matching and creating more diverse datasets to evaluate retrieval performance. • ANIMAR dataset was constructed with 711 3D animal models and 140 sketch queries. • SHREC challenge track focusing on Text-based 3D ANIMAl model fine-grained Retrieval. • We receive satisfactory results from eight teams and 204 runs. [ABSTRACT FROM AUTHOR]

Astronauts are at risk for pneumothorax, a condition where injury or disease introduces air between the chest wall and the lungs (i.e., the pleural cavity). In a worst-case scenario, it can rapidly lead to a fatality if left unmanaged and will require prompt treatment in situ if developed during spaceflight. Chest tube insertion is the definitive treatment for pneumothorax, but it requires a high level of skill and frequent practice for safe use. Physician astronauts may struggle to maintain this skill on medium- and long-duration exploration-class missions, and it is inappropriate for pure just-in-time learning or skill refreshment paradigms. This paper proposes semi-automating tool insertion to reduce the risk of complications in austere environments and describes preliminary experiments providing initial validation of an intelligent prototype system. Specifically, we showcase and analyse motion and force recordings from a sensorized percutaneous access needle inserted repeatedly into an ex vivo tissue phantom, along with relevant physiological data simultaneously recorded from the operator. When coupled with minimal just-in-time training and/or augmented reality guidance, the proposed system may enable non-expert operators to safely perform emergency chest tube insertion without the use of ground resources. • System design of a desktop testbed that facilitates data-driven event labelling. • Use of a lightweight robot enables semi-automation during simulated surgical tasks. • Needle-tissue interaction forces are useful for surgical semi-automation. • Robotics and wearables allow for human-in-the-loop system performance monitoring. [ABSTRACT FROM AUTHOR]

Augmented reality devices are increasingly accepted in health care, though most applications involve education and pre-operative planning. A novel augmented reality ultrasound application, HoloUS, was developed for the Microsoft HoloLens 2 to project real-time ultrasound images directly into the user's field of view. In this work, we assessed the effect of using HoloUS on vascular access procedural outcomes. A single-center user study was completed with participants with (N = 22) and without (N = 12) experience performing ultrasound-guided vascular access. Users completed a venipuncture and aspiration task a total of four times: three times on study day 1, and once on study day 2 between 2 and 4 weeks later. Users were randomized to use conventional ultrasound during either their first or second task and the HoloUS application at all other times. Task completion time, numbers of needle re-directions, head adjustments and needle visualization rates were recorded. For expert users, task completion time was significantly faster using HoloUS (11.5 s, interquartile range [IQR] = 6.5–23.5 s vs. 18.5 s, IQR = 11.0–36.5 s; p = 0.04). The number of head adjustments was significantly lower using the HoloUS app (1.0, IQR = 0.0–1.0 vs. 3.0, IQR = 1.0–5.0; p < 0.0001). No significant differences were identified in other measured outcomes. This is the first investigation of augmented reality-based ultrasound-guided vascular access using the second-generation HoloLens. It demonstrates equivalent procedural efficiency and accuracy, with favorable usability, ergonomics and user independence when compared with traditional ultrasound techniques. [ABSTRACT FROM AUTHOR]

One of the most common reasons for poor patient outcomes and revision surgery in spinal fusion is hardware failure. Screw loosening or pullout occurs in up to one-quarter of all cases. It is known that even small screw–rod misalignments can cause significant mechanical overloads during rod fixation, which can result in hardware failure. To address this crucial surgical step, a novel augmented reality-assisted software was developed to generate custom rod templates that are precisely adapted to the individual patient. The novel software, which runs on a tablet, is used in spinal fusion surgery and is based on the use of a specific pedicle screw system, in which the polyaxial screw heads are connected to detachable guides. These guides can be recognized by the tablet camera and a light detection and ranging scanner. This image information is processed to determine the spatial positions of the screw heads and to calculate an ideally fitting rod template. The calculated rod template is displayed in a 1-to-1 scale on the tablet screen. This template is used to cut and bend the rods of the pedicle screw system. Finally, the custom bent rod can be inserted into the screw heads without tension. The augmented reality-assisted software is intended to give surgeons access to patient-specific intraoperative real-time data, helping them in bending rods that are more precisely adapted to the individual patient compared with the freehand technique. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) is becoming more common and slowly being integrated into the surgical field. With the continuous progression of navigation and visualization techniques, AR has great potential to improve surgical quality and safety. Nevertheless, the effects of AR on surgical outcomes and surgeons' well-being remains poorly studied. The present prospective controlled study aims to assess the effect of surgery assisted with AR smart glasses on adolescent idiopathic scoliosis (AIS) deformity correction outcomes and surgeon fatigue. AIS patients scheduled for surgical deformity correction were prospectively recruited and assigned to standard or AR-supported surgery, using lightweight AR smart glasses. The demographic and clinical features were recorded. The pre- and postoperative spinal features, operative time, and blood loss were recorded and compared. Finally, the participating surgeons were asked to complete a questionnaire (e.g., visual analog scale for fatigue) to compare the effects of AR on their well-being. Our results have shown enhanced spinal deformity corrections with Cobb angle (−35.7° vs. −46.9°), thoracic kyphosis (8.1° vs. 11.6°), and vertebral rotation (−9.3° vs. −13.8°) changes favoring AR-supported surgery. Moreover, AR resulted in significantly lower violation rates per patient (7.5% vs. 6.6%; P = 0.023). Finally, the visual analog scale for fatigue scores consistently showed a significant reduction in fatigue (5.7 ± 1.7 vs. 3.3 ± 1.2; P < 0.001) and other fatigue classifiers for the surgeons after AR-supported surgery. Our controlled study has highlighted the enhanced spinal correction rates in AR-supported surgery and also improved surgeons' well-being and fatigue. These results endorse the adaptation of AR techniques to support AIS surgical correction. [Display omitted] [ABSTRACT FROM AUTHOR]

Use of augmented reality is increasingly applied in medical education and practice. The main advantage of this technology is the display of relevant information in the visual field of multiple operators. Here we provide a critical analysis of the potential application of augmented reality in regional anaesthesia. [ABSTRACT FROM AUTHOR]

Spine surgery has undergone significant changes in approach and technique. With the adoption of intraoperative navigation, minimally invasive spinal surgery (MISS) has arguably become the gold standard. Augmented reality (AR) has now emerged as a front-runner in anatomical visualization and narrower operative corridors. In effect, AR is poised to revolutionize surgical training and operative outcomes. Our study examines the current literature on AR-assisted MISS, synthesizes findings, and creates a narrative highlighting the history and future of AR in spine surgery. Relevant literature was gathered using the PubMed (Medline) database from 1975 to 2023. Pedicle screw placement models were the primary intervention in AR. These were compared to the outcomes of traditional MISS We found that AR devices on the market show promising clinical outcomes in preoperative training and intraoperative use. Three prominent systems were as follows: XVision, HoloLens, and ImmersiveTouch. In the studies, surgeons, residents, and medical students had opportunities to operate AR systems, showcasing their educational potential across each phase of learning. Specifically, one facet described training with cadaver models to gauge accuracy in pedicle screw placement. AR-MISS exceeded free-hand methods without unique complications or contraindications. While still in its infancy, AR has already proven beneficial for educational training and intraoperative MISS applications. We believe that with continued research and advancement of this technology, AR is poised to become a dominant player within the fundamentals of surgical education and MISS operative technique. [ABSTRACT FROM AUTHOR]

Neurosurgery residents face a learning curve at the beginning of residency. Virtual reality (VR) training may alleviate challenges through an accessible, reusable, anatomical model. Medical students performed external ventricular drain placements in VR to characterize the learning curve from novice to proficient. Distance from catheter to foramen of Monro and location with respect to ventricle were recorded. Changes in attitudes toward VR were assessed. Neurosurgery residents performed external ventricular drain placements to validate proficiency benchmarks. Resident and student impressions of the VR model were compared. Twenty-one students with no neurosurgical experience and 8 neurosurgery residents participated. Student performance improved significantly from trial 1 to 3 (15 mm [12.1–20.70] vs. 9.7 [5.8–15.3], P = 0.02). Student attitudes regarding VR utility improved significantly posttrial. The distance to foramen of Monro was significantly shorter for residents than for students in trial 1 (9.05 [8.25–10.73] vs. 15 [12.1–20.70], P = 0.007) and trial 2 (7.45 [6.43–8.3] vs. 19.5 [10.9–27.6], P = 0.002). By trial 3 there was no significant difference (10.1 [8.63–10.95 vs. 9.7 [5.8–15.3], P = 0.62). Residents and students provided similarly positive feedback for VR in resident curricula, patient consent, preoperative practice and planning. Residents provided more neutral-to-negative feedback regarding skill development, model fidelity, instrument movement, and haptic feedback. Students showed significant improvement in procedural efficacy which may simulate resident experiential learning. Improvements in fidelity are needed before VR can become a preferred training technique in neurosurgery. [ABSTRACT FROM AUTHOR]

Most cultural promotion and dissemination are nowadays performed through the digitization of heritage sites and museums, a necessary requirement to meet the new needs of the public. Augmented Reality (AR), Mixed Reality (MR), and Virtual Reality (VR) have the potential to improve the experience quality and educational effect of these sites by stimulating users' senses in a more natural and vivid way. In this respect, head-mounted display (HMD) devices allow visitors to enhance the experience of cultural sites by digitizing information and integrating additional virtual cues about cultural artifacts, resulting in a more immersive experience that engages the visitor both physically and emotionally. This study contributes to the development and incorporation of AR, MR, and VR applications in the cultural heritage domain by providing an overview of relevant studies utilizing fully immersive systems, such as headsets and CAVE systems, emphasizing the advantages that they bring when compared to handheld devices. We propose a framework study to identify the key features of headset-based Extended Reality (XR) technologies used in the cultural heritage domain that boost immersion, sense of presence, and agency. Furthermore, we highlight core characteristics that favor the adoption of these systems over more traditional solutions (e.g., handheld devices), as well as unsolved issues that must be addressed to improve the guests' experience and the appreciation of the cultural heritage. An extensive search of Google Scholar, Scopus, IEEE Xplore, ACM Digital Library, and Wiley Online Library databases was conducted, including papers published from January 2018 to September 2022. To improve review reporting, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were used. Sixty-five papers met the inclusion criteria and were classified depending on the study's purpose: education, entertainment, edutainment, touristic guidance systems, accessibility, visitor profiling, and management. Immersive cultural heritage systems allow visitors to feel completely immersed and present in the virtual environment, providing a stimulating and educational cultural experience that can improve the quality and learning purposes of cultural visits. Nonetheless, the analyzed studies revealed some limitations that must be faced to give a further impulse to the adoption of these technologies in the cultural heritage domain. [ABSTRACT FROM AUTHOR]

Several barriers prevent the integration and adoption of augmented reality (AR) in robotic renal surgery despite the increased availability of virtual three-dimensional (3D) models. Apart from correct model alignment and deformation, not all instruments are clearly visible in AR. Superimposition of a 3D model on top of the surgical stream, including the instruments, can result in a potentially hazardous surgical situation. We demonstrate real-time instrument detection during AR-guided robot-assisted partial nephrectomy and show the generalization of our algorithm to AR-guided robot-assisted kidney transplantation. We developed an algorithm using deep learning networks to detect all nonorganic items. This algorithm learned to extract this information for 65 927 manually labeled instruments on 15 100 frames. Our setup, which runs on a standalone laptop, was deployed in three different hospitals and used by four different surgeons. Instrument detection is a simple and feasible way to enhance the safety of AR-guided surgery. Future investigations should strive to optimize efficient video processing to minimize the 0.5-s delay currently experienced. General AR applications also need further optimization, including detection and tracking of organ deformation, for full clinical implementation. [ABSTRACT FROM AUTHOR]

• A systematic literature review of immersive technologies (ImTs) for occupational safety and health (OSH) in construction is conducted. • OSH areas, hazards and conditions that are addressed by ImTs are provided. • Challenges and future research associated with the use of ImTs for OSH are provided. Introduction : The construction industry employs about 7% of global manpower and contributes about 6% to the global economy. However, statistics have depicted that the construction industry contributes significantly to workplace fatalities and injuries despite multiple interventions (including technological applications) implemented by governments and construction companies. Recently, immersive technologies as part of a suite of industry 4.0 technologies, have also strongly emerged as a viable pathway to help address poor construction occupational safety and health (OSH) performance. Method: With the aim of gaining a broad view of different construction OSH issues addressed using immersive technologies, a review on the application of immersive technologies for construction OSH management is conducted using the preferred reporting items for systematic reviews and meta-analysis (PRISMA) approach and bibliometric analysis of literature. This resulted in the evaluation of 117 relevant papers collected from three online databases (Scopus, Web of Science, and Engineering Village). Results : The review revealed that literature have focused on the application of various immersive technologies for hazard identification and visualization, safety training, design for safety, risk perception, and assessment in various construction works. The review identified several limitations regarding the use of immersive technologies, which include the low level of adoption of the developed immersive technologies for OSH management by the construction industry, very limited research on the application of immersive technologies for health hazards, and limited focus on the comparison of the effectiveness of various immersive technologies for construction OSH management. Conclusions and Practical Applications: For future research, it is recommended to identify possible reasons for the low transition level from research to industry practice and proffer solutions to the identified issues. Another recommendation is the study of the effectiveness of the use of immersive technologies for addressing health hazards in comparison to the conventional methods. [ABSTRACT FROM AUTHOR]

Yearly, more than 20,000 children experience a cardiac arrest. High-quality pediatric cardiopulmonary resuscitation (CPR) is generally challenging for community hospital teams, where pediatric cardiac arrest is infrequent. Current feedback systems are insufficient. Therefore, we developed an augmented reality (AR) CPR feedback system for use in many settings. We aimed to evaluate whether AR-CPR improves chest compression (CC) performance in non–pediatric-specialized community emergency departments (EDs). We performed an unblinded, randomized, crossover simulation-based study. A convenience sample of community ED nonpediatric nurses and technicians were included. Each participant performed three 2-min cycles of CC during a simulated pediatric cardiac arrest. Participants were randomized to use AR-CPR in one of three CC cycles. Afterward, participants participated in a qualitative interview to inquire about their experience with AR-CPR. Of 36 participants, 18 were randomized to AR-CPR in cycle 2 (group A) and 18 were randomized to AR-CPR in cycle 3 (group B). When using AR-CPR, 87–90% (SD 12–13%) of all CCs were in goal range, analyzed as 1-min intervals, compared with 18–21% (SD 30–33%) without feedback (p < 0.001). Analysis of qualitative themes revealed that AR-CPR may be usable without a device orientation, be effective at cognitive offloading, and reduce anxiety around and enhance confidence in the CC delivered. The novel CPR feedback system, AR-CPR, significantly changed the CC performance in community hospital non–pediatric-specialized general EDs from 18–21% to 87–90% of CC epochs at goal. This study offers preliminary evidence suggesting AR-CPR improves CC quality in community hospital settings. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) and virtual reality (VR) implementation in spinal surgery has expanded rapidly over the past decade. This systematic review summarizes the use of AR/VR technology in surgical education, preoperative planning, and intraoperative guidance. A search query for AR/VR technology in spine surgery was conducted through PubMed, Embase, and Scopus. After exclusions, 48 studies were included. Included studies were then grouped into relevant subsections. Categorization into subsections yielded 12 surgical training studies, 5 preoperative planning, 24 intraoperative usage, and 10 radiation exposure. VR-assisted training significantly reduced penetration rates or increased accuracy rates compared to lecture-based groups in 5 studies. Preoperative VR planning significantly influenced surgical recommendations and reduced radiation exposure, operating time, and estimated blood loss. For 3 patient studies, AR-assisted pedicle screw placement accuracy ranged from 95.77% to 100% using the Gertzbein grading scale. Head-mounted display was the most common interface used intraoperatively followed by AR microscope and projector. AR/VR also had applications in tumor resection, vertebroplasty, bone biopsy, and rod bending. Four studies reported significantly reduced radiation exposure in AR group compared to fluoroscopy group. AR/VR technologies have the potential to usher in a paradigm shift in spine surgery. However, the current evidence indicates there is still a need for 1) defined quality and technical requirements for AR/VR devices, 2) more intraoperative studies that explore usage outside of pedicle screw placement, and 3) technological advancements to overcome registration errors via the development of an automatic registration method. [ABSTRACT FROM AUTHOR]

Accurate and efficient object pose estimation holds the indispensable part of virtual/augmented reality (VR/AR) and many other applications. While previous works focus on directly regressing 6D pose from RGB and depth image and thus suffer from the non-linearity of rotation space, we propose an iterative 3D keypoints voting network, named as KVNet. Specifically, our method decouples the pose into separate translation and rotation branch, both estimated by Hough voting scheme. By treating the uncertainty of keypoints' vote as the Lipschitz continuous function of seed points' fused embedding feature, our method is able to adaptively select the optimal keypoints vote. In this way, we argue that KVNet bridges the gap between the non-linear rotation space and linear Euclidean space, which introduces inductive bias for our network to learn the intrinsic pattern and infer 6D pose from RGB and depth images. Furthermore, our model will refine the initial keypoints localization with iterative fashion. Experiments show that across three challenging benchmark datasets (LineMOD, YCB-Video and Occlusion LineMOD), our method exhibits excellent performance. [ABSTRACT FROM AUTHOR]

• Nine novel AR interfaces for pedestrian-vehicle communication were evaluated using an online questionnaire study. • A total of 992 respondents from Germany, the Netherlands, Norway, Sweden, and the United Kingdom completed the study. • Interfaces that employed design elements from existing traffic, and head-up displays, received the highest ratings overall. • There were no statistically significant effects of country, age, and gender on interface acceptance. • The mean ratings of AR interfaces showed strong convergence for various facets of acceptance. Augmented Reality (AR) technology could be utilised to assist pedestrians in navigating safely through traffic. However, whether potential users would understand and use such AR solutions is currently unknown. Nine novel AR interfaces for pedestrian-vehicle communication, previously developed using an experience-based design method, were evaluated through an online questionnaire study completed by 992 respondents in Germany, the Netherlands, Norway, Sweden, and the United Kingdom. The AR indicated whether it was safe to cross the road in front of an approaching automated vehicle. Each interface was rated for its intuitiveness and convincingness, aesthetics, and usefulness. Moreover, comments were collected for qualitative analysis. The results indicated that interfaces that employed traditional design elements from existing traffic, and head-up displays, received the highest ratings overall. Statistical results also showed that there were no significant effects of country, age, and gender on interface acceptance. Thematic analysis of the textual comments offered detail on each interface design's stronger and weaker points, and revealed unintended effects of certain designs. In particular, some of the interfaces were commented on as being dangerous or scary, or were criticised that they could be misinterpreted in that they signal that something is wrong with the vehicle, or that they could occlude the view of the vehicle. The current findings highlight the limitations of experience-based design, and the importance of applying legacy design principles and involving target users in design and evaluation. Future research should be conducted in scenarios in which pedestrians actually interact with approaching vehicles. [ABSTRACT FROM AUTHOR]

Analysis of point clouds through deep convolutional neural networks is an active area of research due to their massive real-world applications including autonomous driving, indoor navigation, robotics, virtual/augmented reality, unmanned aerial vehicles, and drone technology. However, capturing the fine-grained geometric and semantic properties for the underlying recognition task with raw unstructured point cloud is highly challenging due to the lack of explicit neighborhood relationship and sparsity among the points. In this paper, we have introduced a deep, hierarchical 3D point based architecture for object classification and part segmentation that is able to learn robust geometric features which remain invariant to both the geometry and orientation of the local patches. The proposed architecture consists of multiple layers of sampling, concentric annular convolution, pooling, and residual feature propagation blocks. In the skip connections of our deep residual design, we propose to use a combination of linear projection shortcut and nonlinear ReLU group normalization shortcut with batch normalization, to improve both the optimization landscape and the representational power. Our network achieves on par or even better than state-of-the-art results on synthetic and real-world object classification (i.e., ModelNet40 and ScanObjectNN) and part segmentation (i.e., ShapeNet-part) benchmark datasets. The implementation and results have been made available at ...https://github.com/Rabbia-Hassan/Deep_Annular_Residual_Feature_Learning_for_3dPointCloudsGitHub-link [ABSTRACT FROM AUTHOR]

Intelligent Virtual Agents (IVAs) can provide users with a friendly experience and have a wide range of applications in the era of artificial intelligence. However, most of existing IVAs are designed for personal computers. Design and user studies of IVAs on smartphones are uncommon. Therefore, developing IVAs for smartphones is an interesting topic. Considering Augmented Reality (AR) technology can provide more potential application value for IVAs, we mainly investigate users' experiences of AR IVAs on smartphones in this paper. To make an IVA more suitable for a smartphone, a lightweight IVA's cognitive architecture is proposed. To find out the factors that affect users' interaction experiences, the effects of humanoid embodiment and emotional expressions of IVAs on users' perceptions and experiences are explored. A museum is used as a specific task scenario to measure users' experiences. Three forms of AR agents are evaluated in this scenario: a voice assistant without an entity, a humanoid IVA without emotional expressions, and a humanoid IVA with emotional expressions. The results show that compared with the voice assistant, a humanoid embodiment can significantly improve the user's experience, and compared with humanoid IVA without emotional expressions, a humanoid IVA with emotional expressions is more welcome. Moreover, we use the cloud model to describe the uncertainty of IVAs' actions (blinking and body orientation). The results show that the uncertainty of actions can increase the believability of IVAs. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) technology has played an increasing role in cerebrovascular neurosurgery over the last 2 decades. Hence, we aim to evaluate the technical and educational value of head-mounted AR in cerebrovascular procedures. This is a single-center retrospective study of patients who underwent open surgery for cranial and spinal cerebrovascular lesions between April and August 2022. In all cases, the Medivis Surgical AR platform and HoloLens 2 were used for preoperative and intraoperative (preincision) planning. Surgical plan adjustment due to the use of head-mounted AR and subjective educational value of the tool were recorded. A total of 33 patients and 35 cerebrovascular neurosurgical procedures were analyzed. Procedures included 12 intracranial aneurysm clippings, 6 brain and 1 spinal arteriovenous malformation resections, 2 cranial dural arteriovenous fistula obliterations, 3 carotid endarterectomies, two extracranial-intracranial direct bypasses, two encephaloduroangiosynostosis for Moyamoya disease, 1 biopsy of the superficial temporal artery, 2 microvascular decompressions, 2 cavernoma resections, 1 combined intracranial aneurysm clipping and encephaloduroangiosynostosis for Moyamoya disease, and 1 percutaneous feeder catheterization for arteriovenous malformation embolization. Minor changes in the surgical plan were recorded in 16 of 35 procedures (45.7%). Subjective educational value was scored as "very helpful" for cranial, spinal arteriovenous malformations, and carotid endarterectomies; "helpful" for intracranial aneurysm, dural arteriovenous fistulas, direct bypass, encephaloduroangiosynostosis, and superficial temporal artery-biopsy; and "not helpful" for cavernoma resection and microvascular decompression. Head-mounted AR can be used in cerebrovascular neurosurgery as an adjunctive tool that might influence surgical strategy, enable 3-dimensional understanding of complex anatomy, and provide great educational value in selected cases. [ABSTRACT FROM AUTHOR]

Virtual reality (VR), augmented reality (AR) and mixed reality (MR) can evoke a sense of presence, where a person can feel as though they are physically present in a virtual environment. The intention of this research is to understand whether VR/AR/MR can create human connection and experience through interactions with virtual environments to assist in the prevention and treatment of psychological impacts to astronaut health during long-term and long-distance space missions (experiencing Earth from space); and for space tourism, education and outreach purposes as a cost-effective way for people to experience space; to help people understand how important the space industry is to our lives; and to inspire future careers (experiencing space from Earth). Qualitative literature review was undertaken to understand current and potential future VR/AR/MR technologies and how they are being used in psychology; in space; for astronaut mental health in space; for two-way communication; for general tourism, education and outreach purposes; and for space tourism, space education and space outreach purposes. Recommendations are proposed for further study into VR/AR/MR as a tool for astronaut mental health and wellbeing; and for further development of VR/AR/MR experiences for space tourism, education and outreach purposes. • Virtual reality can be used to improve mental health. • Virtual reality can be used to improve mental health in isolation and confinement. • Virtual reality may be used to alleviate mental health issues in astronauts. • Virtual and augmented reality are safe ways to inspire people about space. • Virtual and augmented reality are a cost-effective alternative to space tourism. [ABSTRACT FROM AUTHOR]

Facing up to the global challenges of designing climate-resilient biotech crops involves a great deal of out-of-the-box thinking. Extended reality is coming of age in digital agricultural biotechnology. Here, we seek to stimulate technological innovation by empowering future innovators, researchers, academics, and startups to think and partner creatively. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) employs an optical projection directly onto the user's retina, allowing complex image overlay on the natural visual field. In general, pedicle screw accuracy rates have improved with increasingly use of technology, with navigation-based instrumentation described as accurate in 89%–100% of cases. Emerging AR technology in spine surgery builds upon current spinal navigation to provide 3-dimensional imaging of the spine and powerfully reduce the impact of inherent ergonomic and efficiency difficulties. This publication describes the first known series of in vivo pedicle screws placed percutaneously using AR technology for MIS applications. After IRB approval, 3 senior surgeons at 2 institutions contributed cases from June, 2020 – March, 2022. 164 total MIS cases in which AR used for placement of percutaneous pedicle screw instrumentation with spinal navigation were identified prospectively. 155 (94.5%) were performed for degenerative pathology, 6 (3.6%) for tumor and 3 (1.8%) for spinal deformity. These cases amounted to a total of 606 pedicle screws; 590 (97.3%) were placed in the lumbar spine, with 16 (2.7%) thoracic screws placed. Patient demographics and surgical metrics including total posterior construct time (defined as time elapsed from preincision instrument registration to final screw placement), clinical complications and instrumentation revision rates were recorded in a secure and de-identified database. The AR system used features a wireless headset with transparent near-eye display which projects intra-operative 3D imaging directly onto the surgeon's retina. After patient positioning, 1 percuntaneous and 1 superficial reference marker are placed. Intra-operative CT data is processed to the headset and integrates into the surgeon's visual field creating a "see-through" 3D effect in addition to 2D standard navigation images. MIS pedicle screw placement is then carried out percutaneously through single line of sight using navigated instruments. Time elapsed from registration and percutaneous approach to final screw placement averaged 3 minutes and 54 seconds per screw. Analysis of the learning curve revealed similar surgical times in the early cases compared to the cases performed with more experience with the system. No instrumentation was revised for clinical or radiographic complication at final available follow-up ranging from 6–24 months. A total of 3 screws (0.49%) were replaced intra-operatively. No clinical effects via radiculopathy or neurologic deficit postoperatively were noted. This is the first report of the use of AR for placement of spinal pedicle screws using minimally invasive techniques. This series of 164 cases confirmed efficiency and safety of screw placement with the inherent advantages of AR technologies over legacy enabling technologies. [ABSTRACT FROM AUTHOR]

• Evaluation of Detection Response Task for measuring workload of different HMIs. • Comparison of Head-Down Display, Head-Up Display, and Augmented-Reality HUD. • Due to hedonic aspects, drivers prefer the AR-HUD display technology. • Participants accept the Detection Response Task for workload assessment. • The Detection Response Task allows a valid and objective workload measurement. Augmented reality (AR) technology could establish direct relationships between displayed information and objects in the real driving environment, e.g. by highlighting relevant objects in the traffic environment. However, it is unclear how these potential benefits of augmentation affect drivers' distraction from the driving task and their level of workload. Since gaze detection cannot separate glances at AR-elements from glances at the driving scene, new evaluation criteria are needed for measuring the distraction potential of AR-elements. The paper evaluates a methodology based on the Detection Response Task (DRT) for measuring the distraction and workload effects of three different display technologies: Head-Down Display (HDD), Head-Up Display (HUD), and Augmented-Reality HUD (AR-HUD). The DRT measures cognitive and visual workload based on a recognition-response task using the two parameters, reaction time and misses. Two experimental studies using driving simulation (N = 24) and production vehicles in real traffic (N = 24) are reported. As use cases, the subjects had to navigate through an urban traffic environment and pass different intersections. During that they were assisted by navigation systems based on the above mentioned display technologies. The resulting workload is compared to a baseline drive without the need to navigate and a reference drive in which the subjects had to work on a secondary task while driving. The results of the DRT are compared with subjective measures, gaze behaviour, and driving performance. In addition, drivers' preference and user experience regarding the three display technologies and their feedback concerning the DRT method were assessed. The results of both experimental studies show that drivers prefer the AR-HUD display technology. Reasons are mainly the hedonic aspects of the AR-HUD such as the novelty of the technology. Nevertheless, drivers report that AR-elements could obstruct important areas of the road and distract more from the driving task than HUDs. The subjective and visual-cognitive workload measured by the DRT do not differ between the HMI conditions. An analysis for each type of intersection yields most misses for the AR-HUD condition and fewest misses in the HDD condition. However, visual workload (i.e. eyes-off-road time) is higher for HDD compared to (AR-)HUD. Feedback by participants reveals that the DRT as a method for workload assessment is well accepted. All in all, the two DRT measures, misses and reaction time, allow a valid and objective workload measurement and could serve as a supplement or substitute for gaze analysis. The discussion gives indications for the measurement of workload effects caused by AR-HUD using DRT. [ABSTRACT FROM AUTHOR]

In this era of imagination and technological innovation, mixed reality systems such as virtual reality (VR) and augmented reality (AR) are contributing to a wide array of neurosurgical care, from the betterment of surgical planning and surgical comfort to novel treatments and improved resident education. These systems can augment procedures that require high-level dexterity such as minimally invasive surgery and tumor excisions, as well as peripheral and neurovascular surgery. We define and compare the technological features, indications, and characterized outcomes of VR and AR systems in the context of neurosurgery through a review of the literature to date. Moreover, this review discusses the limitations of VR and AR and includes an overview of the cost-effectiveness of each of these systems. An extensive review of the literature on AR and VR was performed using PubMed, OVID Medline, and Embase from January 1, 2006 to April 2, 2022. Terms used for the search included AR, spinal surgery, VR, and neurosurgery. The search yielded full-text English language–related articles regarding VR and AR application, limitations, and functional outcomes in neurosurgery. An initial set of 121 studies were screened and reviewed for content. Thirteen studies were included, which involved 162 patients, 550 screw placements, 58 phantom spines, and learning points from simulation training of 276 involved residents. This literature review examines recent research into VR and AR applications in neurosurgical care. The literature establishes that there are technological features, indications, outcomes, limitations, and cost-effectiveness differences between these systems. Based on ongoing and evolving applications of the VR and AR systems, the innovative potential that they make available to the future of neurosurgical patient care makes clear the need for further studies to understand the nuances between their differing technological advances. [ABSTRACT FROM AUTHOR]

Point clouds contain high detail and high accuracy geometry representation of the scanned Earth surface parts. To manage the huge amount of data, the point clouds are traditionally organized on location and map-scale; e.g. in an octree structure, where top-levels of the tree contain few points suitable for small scale overviews and lower levels of the tree contain more points suitable for large scale detailed views. The drawback of this solution is that it is based on discrete levels, causing visual artifacts in the form of data density shocks when creating the commonly used perspective views. This paper presents a method based on an optimized distribution of points over continuous levels, avoiding the visualization shocks. The traditional distribution ratio's of data amounts over discrete levels of raster or vector data is considered the reference. How to convert this to point clouds with continuous levels (still benefiting from the proven advantages of the data distribution in discrete levels for efficient access at a wide range of scales)? In our solution, for each point a cLoD (continuous Level of Detail) value is computed and added as dimension to the point. A SFC (Space Filling Curve)-based nD data clustering technique can be used to organize the points, so that they can be efficiently queried. It should be noted that also other multi-dimensional indexing and clustering techniques could be applied to realize continuous levels based on the cLoD value. Besides the mathematical foundation of the approach also several implementations are described, varying from a 3D web-browser based solution to an augmented reality point cloud app in a mobile phone. The cLoD enables interactive real-time visualization using perspective views without data density shocks, while supporting continuous zoom-in/out and progressive data streaming between server and client. The described cLoD based approach is generic and supports different types of point clouds: from airborne, terrestrial, mobile and indoor laser scanning, but also from dense matching optical imagery or multi-beam echo soundings. [ABSTRACT FROM AUTHOR]

In augmented reality (AR) applications, detecting and tracking real-world objects remains a challenge. Another challenge is making the loaded virtual objects truly reflect the lighting and shadow conditions of the actual environment in which they are located. In this paper, we proposed an improved method for real-time light estimation of environmental lighting in AR applications, tracking the movement of physical objects, and making the corresponding virtual objects reflect the lighting and shadows of the actual environment. First, we proposed an effective method for estimating light in an AR scene by training a deep neural network once on different scenes from the input of RGB-D images to determine the direction of the brightest light. Second, we experimented with advanced methods of detecting and tracking different types of real objects (flat surfaces, simple geometries, and complex geometries) for visualizing virtual materials on the top of these surfaces, where virtual materials will acquire the conditions of the real place in terms of lighting, reflection, and shadows depending on our proposed method for light estimation. Our method of light estimation was tested via experiments on the three types of real objects mentioned above. Results indicate that our method achieves higher resolution and lower error rate compared with other work. Our methods have achieved very good results in terms of detection accuracy, tracking accuracy, realistic visualization of materials, matching real shade, and lighting conditions. [Display omitted] [ABSTRACT FROM AUTHOR]

Image-guided surgery has shown great utility in neurosurgery, especially in allowing for more accurate surgical planning and navigation. The current gold standard for image-guided neurosurgery is neuronavigation, which provides millimetric accuracy on such tasks. However, these approaches often require a complicated setup and have high cost, hindering their potential in low- and middle-income countries. The aim of this study was to develop and evaluate the performance of a mobile-based augmented reality neuronavigation solution under different conditions in a preclinical environment. The application was developed using the Swift programming language and was tested on a replica of a human scalp under variable lighting, with different numbers of registration points and target point position conditions. For each condition, reference points were input into the application, and the target points were computed for 10 iterations. The mean registration error and target error were used to assess the performance of the application. In the best-case scenario, the proposed solution had a mean target error of 2.6 ± 1.6 mm. Our approach provides a viable, low-cost, easy-to-use, portable method for locating points on the scalp surface with an accuracy of 2.6 ± 1.6 mm in the best-case scenario. [ABSTRACT FROM AUTHOR]

Primary tumors involving the spine are relatively rare but represent surgically challenging procedures with high patient morbidity. En bloc resection of these tumors necessitates large exposures, wide tumor margins, and poses risks to functionally relevant anatomical structures. Augmented reality neuronavigation (ARNV) represents a paradigm shift in neuronavigation, allowing on-demand visualization of 3D navigation data in real-time directly in line with the operative field. Here, we describe the first application of ARNV to perform distal sacrococcygectomies for the en bloc removal of sacral and retrorectal lesions involving the coccyx in 2 patients, as well as a thoracic 9–11 laminectomy with costotransversectomy for en bloc removal of a schwannoma in a third patient. In our experience, ARNV allowed our teams to minimize the length of the incision, reduce the extent of bony resection, and enhanced visualization of critical adjacent anatomy. All tumors were resected en bloc, and the patients recovered well postoperatively, with no known complications. Pathologic analysis confirmed the en bloc removal of these lesions with negative margins. We conclude that ARNV is an effective strategy for the precise, en bloc removal of spinal lesions including both sacrococcygeal tumors involving the retrorectal space and thoracic schwannomas. [ABSTRACT FROM AUTHOR]

Virtual reality (VR) and augmented reality (AR) are rapidly growing technologies. Both have been applied within neurosurgery for presurgical planning and intraoperative navigation, but VR and AR technology is particularly promising for the education of neurosurgical trainees. With the increasing demand for high impact yet efficient educational strategies, VR- and AR-based simulators allow neurosurgical residents to practice technical skills in a low-risk setting. Initial studies have confirmed that such simulators increase trainees' confidence, improve their understanding of operative anatomy, and enhance surgical techniques. Knowledge of the history and conceptual underpinnings of these technologies is useful to understand their current and future applications towards neurosurgical training. The technological precursors for VR and AR were introduced as early as the 1800s, and draw from the fields of entertainment, flight simulation, and education. However, computer software and processing speeds are needed to develop widespread VR- and AR-based surgical simulators, which have only been developed within the last 15 years. During that time, several devices had become rapidly adopted by neurosurgeons, and some programs had begun to incorporate them into the residency curriculum. With ever-improving technology, VR and AR are promising additions to a multi-modal training program, enabling neurosurgical residents to maximize their efforts in preparation for the operating room. In this review, we outline the historical development of the VR and AR systems that are used in neurosurgical training and discuss representative examples of the current technology. [ABSTRACT FROM AUTHOR]

Today's orthopedic surgery could not be imagined without intraoperative x-ray-based imaging. This enables surgeons to assess operative interim steps as well as the result before wound closure and finishing the procedure. Although there have been mobile C-arms used for decades, there are recent advances that do not only affect the quality of the imaging itself but also the way, the information is processed and presented. These very exciting developments will change the integration of imaging into the surgical workflows, giving options of augmented reality, reduction of radiation dose, automatized acquisition and analysis of images and low-level guidance in procedures. This paper gives a review of current innovations and possible future trends in fluoroscopic 2D and 3D imaging. [ABSTRACT FROM AUTHOR]

• We summarise works in image digital manipulation with the purpose of facial beautification. • We study the impact of enhancement and Augmented Reality filters on face detection and recognition. • We develop a method to reverse the applied manipulations that entail eye obfuscation. • We study if training the recognition system with manipulated images helps to increase accuracy. Beautification and augmented reality filters are very popular in applications that use selfie images. However, they can distort or modify biometric features, severely affecting the ability to recognise the individuals' identity or even detect the face. Accordingly, we address the effect of such filters on the accuracy of automated face detection and recognition. The social media image filters studied modify the image contrast, illumination, or occlude parts of the face. We observe that the effect of some of these filters is harmful to face detection and identity recognition, especially if they obfuscate the eye or (to a lesser extent) the nose. To counteract such effect, we develop a method to reverse the applied manipulation with a modified version of the U-NET segmentation network. This method is observed to contribute to better face detection and recognition accuracy. From a recognition perspective, we employ distance measures and trained machine learning algorithms applied to features extracted using several CNN backbones. We also evaluate if incorporating filtered images into the training set of machine learning approaches is beneficial. Our results show good recognition when filters do not occlude important landmarks, especially the eyes. The combined effect of the proposed approaches also allows mitigating the impact produced by filters that occlude parts of the face. [ABSTRACT FROM AUTHOR]