Search

Abstract In several orthopedic procedures, the accurate use of surgical power tools is critical to avoid damage to surrounding tissues. As such, various guidance techniques and safety measures were developed. Augmented reality (AR) guidance shows promise but requires validation. We evaluated a new approach using an inside-out infrared tracking solution for the HoloLens to compensate for its limited tracking performance. Eighteen participants with varying levels of experience (student, trainee, expert) each drilled twelve trajectories (six perpendicular, six oblique) in equidimensional wooden logs. Three different techniques were evaluated: freehand drilling; proprioception-guided drilling towards the contralateral index finger; and AR-guided drilling using a tracked drill and a virtual overlay of the log with predefined guidance vectors. The angular errors between planned and performed trajectories were compared using a mixed-design ANOVA. The results demonstrated that guidance technique (p

In this paper, we introduce MiVitals—a Mixed Reality (MR) system designed for healthcare professionals to monitor patients in wards or clinics. We detail the design, development, and evaluation of MiVitals, which integrates real-time vital signs from a biosensor-equipped wearable, VitalitiTM. The system generates holographic visualizations, allowing healthcare professionals to interact with medical charts and information panels holographically. These visualizations display vital signs, trends, other significant physiological signals, and medical early warning scores in a comprehensive manner. We conducted a User Interface/User Experience (UI/UX) study focusing on novel holographic visualizations and interfaces that intuitively present medical information. This approach brings traditional bedside medical information to life in the real environment through non-contact 3D images, supporting rapid decision-making, vital pattern and anomaly detection, and enhancing clinicians' performance in wards. Additionally, we present findings from a usability study involving medical doctors and healthcare practitioners to assess MiVitals' efficacy. The System Usability Scale study yielded a score of 84, indicating that the MiVitals system has high usability.

The new advancement in percutaneous intervention procedures, which provides more accurate needle placement compared to other modalities, is Augmented Reality (AR). It provides the operator with 3-dimensional (3D) or 2-dimensional (2D) structures of the anatomy along with the target during surgery to get a precise needle placement into the target. The review highlights that using AR systems for needle navigation and placement can become productivity in various interventional procedures. The AR navigation system reduces radiation exposure, procedural time, risk to the patient, and ensures higher accuracy of needle placement than conventional Computed Tomography (CT)-guided percutaneous procedure systems. The AR system can provide precisely guided needle insertion and reduce the risk of complications. As technology and research are increasing, new techniques, such as automatic image-hologram registration, can be brought into the picture. This review compares conventional image-guided procedure systems with the AR system in the interventional radiology field. This review also provides detailed information about AR as a promising tool for optimising needle placement in interventional procedures, paving the way for safer and more effective clinical practices.

Abstract In total hip arthroplasty (THA), determining the center of rotation (COR) and diameter of the hip joint (acetabulum and femoral head) is essential to restore patient biomechanics. This study investigates on-the-fly determination of hip COR and size, using off-the-shelf augmented reality (AR) hardware. An AR head-mounted device (HMD) was configured with inside-out infrared tracking enabling the determination of surface coordinates using a handheld stylus. Two investigators examined 10 prosthetic femoral heads and cups, and 10 human femurs. The HMD calculated the diameter and COR through sphere fitting. Results were compared to data obtained from either verified prosthetic geometry or post-hoc CT analysis. Repeated single-observer measurements showed a mean diameter error of 0.63 mm ± 0.48 mm for the prosthetic heads and 0.54 mm ± 0.39 mm for the cups. Inter-observer comparison yielded mean diameter errors of 0.28 mm ± 0.71 mm and 1.82 mm ± 1.42 mm for the heads and cups, respectively. Cadaver testing found a mean COR error of 3.09 mm ± 1.18 mm and a diameter error of 1.10 mm ± 0.90 mm. Intra- and inter-observer reliability averaged below 2 mm. AR-based surface mapping using HMD proved accurate and reliable in determining the diameter of THA components with promise in identifying COR and diameter of osteoarthritic femoral heads.

In the global context, advancements in technology and science have rendered virtual, augmented, and mixed-reality technologies capable of transforming clinical care and medical environments by offering enhanced features and improved healthcare services. This paper aims to present a mixed reality-based system to control a robotic wheelchair for people with limited mobility. The test group comprised 11 healthy subjects (six male, five female, mean age 35.2 ± 11.7 years). A novel platform that integrates a smart wheelchair and an eye-tracking-enabled head-mounted display was proposed to reduce the cognitive requirements needed for wheelchair movement and control. The approach's effectiveness was demonstrated by evaluating our system in realistic scenarios. The demonstration of the proposed AR head-mounted display user interface for controlling a smart wheelchair and the results provided in this paper could highlight the potential of the HoloLens 2-based innovative solutions and bring focus to emerging research topics, such as remote control, cognitive rehabilitation, the implementation of patient autonomy with severe disabilities, and telemedicine. [ABSTRACT FROM AUTHOR]

The new advancement in percutaneous intervention procedures, which provides more accurate needle placement compared to other modalities, is Augmented Reality (AR). It provides the operator with 3-dimensional (3D) or 2-dimensional (2D) structures of the anatomy along with the target during surgery to get a precise needle placement into the target. The review highlights that using AR systems for needle navigation and placement can become productivity in various interventional procedures. The AR navigation system reduces radiation exposure, procedural time, risk to the patient, and ensures higher accuracy of needle placement than conventional Computed Tomography (CT)-guided percutaneous procedure systems. The AR system can provide precisely guided needle insertion and reduce the risk of complications. As technology and research are increasing, new techniques, such as automatic image-hologram registration, can be brought into the picture. This review compares conventional image-guided procedure systems with the AR system in the interventional radiology field. This review also provides detailed information about AR as a promising tool for optimising needle placement in interventional procedures, paving the way for safer and more effective clinical practices. [ABSTRACT FROM AUTHOR]

Background: Acquired adult flatfoot deformity (AAFD) results in a loss of the medial longitudinal arch of the foot and dysfunction of the posteromedial soft tissues. Hintermann osteotomy (H-O) is often used to treat stage II AAFD. The procedure is challenging because of variations in the subtalar facets and limited intraoperative visibility. We aimed to assess the impact of augmented reality (AR) guidance on surgical accuracy and the facet violation rate. Methods: Sixty AR-guided and 60 conventional osteotomies were performed on foot bone models. For AR osteotomies, the ideal osteotomy plane was uploaded to a Microsoft HoloLens 1 headset and carried out in strict accordance with the superimposed holographic plane. The conventional osteotomies were performed relying solely on the anatomy of the calcaneal lateral column. The rate and severity of facet joint violation was measured, as well as accuracy of entry and exit points. The results were compared across AR-guided and conventional osteotomies, and between experienced and inexperienced surgeons. Results: Experienced surgeons showed significantly greater accuracy for the osteotomy entry point using AR, with the mean deviation of 1.6 ± 0.9 mm (95% CI 1.26, 1.93) compared to 2.3 ± 1.3 mm (95% CI 1.87, 2.79) in the conventional method (P =.035). The inexperienced had improved accuracy, although not statistically significant (P =.064), with the mean deviation of 2.0 ± 1.5 mm (95% CI 1.47, 2.55) using AR compared with 2.7 ± 1.6 mm (95% CI 2.18, 3.32) in the conventional method. AR helped the experienced surgeons avoid full violation of the posterior facet (P =.011). Inexperienced surgeons had a higher rate of middle and posterior facet injury with both methods (P =.005 and.021). Conclusion: Application of AR guidance during H-O was associated with improved accuracy for experienced surgeons, demonstrated by a better accuracy of the osteotomy entry point. More crucially, AR guidance prevented full violation of the posterior facet in the experienced group. Further research is needed to address limitations and test this technology on cadaver feet. Ultimately, the use of AR in surgery has the potential to improve patient and surgeon safety while minimizing radiation exposure. Clinical Relevance: Subtalar facet injury during lateral column lengthening osteotomy represents a real problem in clinical orthopaedic practice. Because of limited intraoperative visibility and variable anatomy, it is hard to resolve this issue with conventional means. This study suggests the potential of augmented reality to improve the osteotomy accuracy. [ABSTRACT FROM AUTHOR]

The application of augmented reality in training health science students is increasingly widespread. The aim of this work was to assess the usability and user expectations of an augmented reality application for smart glasses (Microsoft HoloLens) that can be used to train on four invasive procedures (i.e. intramuscular injection, nasogastric tube insertion, endotracheal intubation and suctioning via tracheostomy tube). A descriptive study was conducted with nursing students from three Spanish universities. Participants answered a questionnaire to assess the use of the ARSim2care application. This application offers the possibility of visualizing the internal anatomical structures during the training of the clinical technical skills for the performance of the mentioned invasive techniques. The questionnaire included demographic data, the System Usability Scale and questions about the user expectations in relation to learning with the use of augmented reality. In total, 61 participants responded to the questionnaire after using the ARSim2care application. The mean score of the System Usability Scale was 73.15 (standard deviation: 15.04) and 62.4% (n = 38) of the participants considered their experience with the application as excellent or good. In relation to user expectations, more than 90% of students indicated that the use of the application could improve their motivation and stimulation in learning, their content retention and their anatomical understanding. The developed ARSim2care application for Microsoft HoloLens showed a high level of usability and acceptance as a learning tool for training certain clinical procedures by visualizing the internal structures of the body. [ABSTRACT FROM AUTHOR]

In order to solve the problems of large workload and low versatility of QR code labelling and complex implementation and difficult deployment of existing no-registration recognition methods in the auxiliary maintenance of coal mine electromechanical equipments, a coal mine electromechanical equipments maintenance guidance system based on improved YOLOv5s is proposed. The system consists of a equipment no-registration recognition module, a fault maintenance guidance module, and a remote expert access guidance module. The equipment no-registration recognition module collects images of faulty equipments through the camera on HoloLens glasses, and analyzes and processes them through an improved YOLOv5s image recognition algorithm to recognize the faulty equipment model. The fault maintenance guidance module automatically matches and calls the preset mixed reality disassembly and assembly model based on the model of the faulty equipment, forming a maintenance guidance solution. The remote expert access guidance module achieves interaction between remote experts and on-site maintenance personnel through audio and video sessions, virtual annotation, and other methods. In order to ensure an immersive experience for users when using mixed reality equipment, ShuffleNetV2 is used to replace the Backbone in YOLOv5s to obtain the YOLOv5s-SN2 network, which reduces the number of model parameters and computational overhead. The experimental results show that YOLOv5s-SN2 has a slight decrease in precision compared to YOLOv5s, but the number of floating-point operations per second (FLOPS) has decreased from 16.5×109 to 7.6×109, and the number of parameters has decreased from 15.6×106 to 8.2×106. Among the YOLO series models, YOLOv5s-SN2 has the best performance. Taking the three leaf Roots blower as an example to verify the overall effectiveness of the system, the results show that YOLOv5s-SN2 can quickly recognize the motor model, call the matching virtual model and maintenance process. The remote experts can assist on-site personnel in electromechanical equipment maintenance through methods such as audio and video access and annotation.

Abstract Background Mixed reality offers potential educational advantages in the delivery of clinical teaching. Holographic artefacts can be rendered within a shared learning environment using devices such as the Microsoft HoloLens 2. In addition to facilitating remote access to clinical events, mixed reality may provide a means of sharing mental models, including the vertical and horizontal integration of curricular elements at the bedside. This study aimed to evaluate the feasibility of delivering clinical tutorials using the Microsoft HoloLens 2 and the learning efficacy achieved. Methods Following receipt of institutional ethical approval, tutorials on preoperative anaesthetic history taking and upper airway examination were facilitated by a tutor who wore the HoloLens device. The tutor interacted face to face with a patient and two-way audio-visual interaction was facilitated using the HoloLens 2 and Microsoft Teams with groups of students who were located in a separate tutorial room. Holographic functions were employed by the tutor. The tutor completed the System Usability Scale, the tutor, technical facilitator, patients, and students provided quantitative and qualitative feedback, and three students participated in semi-structured feedback interviews. Students completed pre- and post-tutorial, and end-of-year examinations on the tutorial topics. Results Twelve patients and 78 students participated across 12 separate tutorials. Five students did not complete the examinations and were excluded from efficacy calculations. Student feedback contained 90 positive comments, including the technology’s ability to broadcast the tutor’s point-of-vision, and 62 negative comments, where students noted issues with the audio-visual quality, and concerns that the tutorial was not as beneficial as traditional in-person clinical tutorials. The technology and tutorial structure were viewed favourably by the tutor, facilitator and patients. Significant improvement was observed between students’ pre- and post-tutorial MCQ scores (mean 59.2% Vs 84.7%, p

Background: Mixed reality offers potential educational advantages in the delivery of clinical teaching. Holographic artefacts can be rendered within a shared learning environment using devices such as the Microsoft HoloLens 2. In addition to facilitating remote access to clinical events, mixed reality may provide a means of sharing mental models, including the vertical and horizontal integration of curricular elements at the bedside. This study aimed to evaluate the feasibility of delivering clinical tutorials using the Microsoft HoloLens 2 and the learning efficacy achieved. Methods: Following receipt of institutional ethical approval, tutorials on preoperative anaesthetic history taking and upper airway examination were facilitated by a tutor who wore the HoloLens device. The tutor interacted face to face with a patient and two-way audio-visual interaction was facilitated using the HoloLens 2 and Microsoft Teams with groups of students who were located in a separate tutorial room. Holographic functions were employed by the tutor. The tutor completed the System Usability Scale, the tutor, technical facilitator, patients, and students provided quantitative and qualitative feedback, and three students participated in semi-structured feedback interviews. Students completed pre- and post-tutorial, and end-of-year examinations on the tutorial topics. Results: Twelve patients and 78 students participated across 12 separate tutorials. Five students did not complete the examinations and were excluded from efficacy calculations. Student feedback contained 90 positive comments, including the technology's ability to broadcast the tutor's point-of-vision, and 62 negative comments, where students noted issues with the audio-visual quality, and concerns that the tutorial was not as beneficial as traditional in-person clinical tutorials. The technology and tutorial structure were viewed favourably by the tutor, facilitator and patients. Significant improvement was observed between students' pre- and post-tutorial MCQ scores (mean 59.2% Vs 84.7%, p < 0.001). Conclusions: This study demonstrates the feasibility of using the HoloLens 2 to facilitate remote bedside tutorials which incorporate holographic learning artefacts. Students' examination performance supports substantial learning of the tutorial topics. The tutorial structure was agreeable to students, patients and tutor. Our results support the feasibility of offering effective clinical teaching and learning opportunities using the HoloLens 2. However, the technical limitations and costs of the device are significant, and further research is required to assess the effectiveness of this tutorial format against in-person tutorials before wider roll out of this technology can be recommended as a result of this study [ABSTRACT FROM AUTHOR]

Copyright of Journal of Mine Automation is the property of Industry & Mine Automation Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Digital transformation is much needed in the healthcare sector to maximize the efficiency and productivity of human health. Adding technologies is an important addition to the continuity of growing the industry, and it is a promising opportunity for human care growth; This research conjectured that while healthcare is a vibrant and continuously evolving field, it becomes essential to present a detailed categorization for the main recent technologies used in the healthcare. To do this conceptual research, we reviewed a deal of existing research, about 31 research papers, that share common characteristics. The papers chosen for review are all written in English language, published in the last three years; From 2021 to 2023. And shown by Google Scholar, IEEE, PubMed and Emerald Insight. Our study presented a more unambiguous categorization. Existing literature showed three main categories, namely Preoperative; Intraoperative; And Post-operative. This research suggests adding a new one category related to Inorganic illness to address psychological and relaxation aspects, which need access to devices in each psychological state and relaxation; Preoperative; Intraoperative; And Post-operative. To conclude, technology has a great role in increasing efficiency and accuracy and reducing material and human losses. The medical sector seeks for more extended, and effective use of technology leverage the quality of life without suffering pain. This is certainly besides the economic prosperity of the medical infrastructure. [ABSTRACT FROM AUTHOR]

The ongoing COVID-19 pandemic has had a significant impact globally, and the understanding of the disease's clinical features and impacts remains insufficient. An important metric to evaluate the severity of pneumonia in COVID-19 is the CT Involvement Score (CTIS), which is determined by assessing the proportion of infections in the lung field region using computed tomography (CT) images. Interactive augmented reality visualization and quantification of COVID-19 infection from CT allow us to augment the traditional diagnostic techniques and current COVID-19 treatment strategies. Thus, in this paper, we present a system that combines augmented reality (AR) hardware, specifically the Microsoft HoloLens, with deep learning algorithms in a user-oriented pipeline to provide medical staff with an intuitive 3D augmented reality visualization of COVID-19 infections in the lungs. The proposed system includes a graph-based pyramid global context reasoning module to segment COVID-19-infected lung regions, which can then be visualized using the HoloLens AR headset. Through segmentation, we can quantitatively evaluate and intuitively visualize which part of the lung is infected. In addition, by evaluating the infection status in each lobe quantitatively, it is possible to assess the infection severity. We also implemented Spectator View and Sharing a Scene functions into the proposed system, which enable medical staff to present the AR content to a wider audience, e.g., radiologists. By providing a 3D perception of the complexity of COVID-19, the augmented reality visualization generated by the proposed system offers an immersive experience in an interactive and cooperative 3D approach. We expect that this will facilitate a better understanding of CT-guided COVID-19 diagnosis and treatment, as well as improved patient outcomes. [ABSTRACT FROM AUTHOR]

Total hip arthroplasty (or total hip replacement) is the current surgical solution for the treatment of advanced coxarthrosis, with the objective of providing mobility and pain relief to patients. For this purpose, surgery can be planned using preoperative images acquired from the patient and navigation systems can also be used during the intervention. Robots have also been used to assist in interventions. In this work, we propose a new mixed reality application for total hip arthroplasty. The surgeon only has to wear HoloLens 2. The application does not require acquiring preoperative or intraoperative images of the patient and uses hand interaction. Interaction is natural and intuitive. The application helps the surgeon place a virtual acetabular cup onto the patient's acetabulum as well as define its diameter. Similarly, a guide for drilling and implant placement is defined, establishing the abduction and anteversion angles. The surgeon has a direct view of the operating field at all times. For validation, the values of the abduction and anteversion angles offered by the application in 20 acetabular cup placements have been compared with real values (ground-truth). From the results, the mean (standard deviation) is 0.375 (0.483) degrees for the error in the anteversion angle and 0.1 (0.308) degrees for the abduction angle, with maximum discrepancies of 1 degree. A study was also carried out on a cadaver, in which a surgeon verified that the application is suitable to be transferred to routine clinical practice, helping in the guidance process for the implantation of a total hip prosthesis. [ABSTRACT FROM AUTHOR]

In recent times, digital transformations and Industry 4.0 have revolutionized real-time bridge monitoring and its inspection. The use of smart Structural Health Monitoring (SHM) techniques is becoming powerful with the competencies of Building Information Modeling (BIM) tools, Artificial Intelligence (AI), Internet of Things (IoT), and Virtual/Augmented (VR/AR) technologies. However, the lack of interconnectivity between these tools limits their functionality. This research has addressed this problem by developing an integrated framework to assess serviceability and implement a smart SHM for a newly constructed extradosed bridge. Using Finite Element Analysis (FEA), the study proposes an integrated SHM system that utilizes various IoT sensors, including Wired Strain Gauges (WSG), Liquid Levelling Sensors (LLS), MEMS accelerometers, and a Weather Monitoring Station (WMS) to monitor concrete deformations, vertical displacements, structural vibrations, and weather conditions. BIM tool is used to develop the virtual replica of the proposed SHM system which is then used in the 3D Game Engine (GE) to develop an AR application. This application is then successfully deployed and tested in the AR headset (HoloLens) where its capabilities for onsite bridge health monitoring are discovered. This approach overcomes the limitations of HoloLens devices by providing real-time access to SHM data through a web platform, enabling on-site or remote AR-based bridge health monitoring. Conclusively, this paper emphasizes the numerical modeling of bridges for the design of a health monitoring system, that highlights the importance of robust SHM techniques in assessing bridge conditions. Moreover, it introduces a novel approach for smart bridge inspection and onsite visualization of structural defects in an AR environment.

Scaffoldings play a critical role as temporary structures in supporting construction processes. Accidents at construction sites frequently stem from issues related to scaffoldings, including insufficient support caused by deviations from the construction design, insecure rod connections, or absence of cross-bracing, which result in uneven loading and potential collapse, leading to casualties. This research introduces a novel approach employing deep learning (i.e., YOLO v5) and augmented reality (AR), termed the scaffolding assembly deficiency detection system (SADDS), designed to aid field inspectors in discerning deficiencies within scaffolding assemblies. Inspectors have the flexibility to utilize SADDS through various devices, such as video cameras, mobile phones, or AR goggles, for the automated identification of deficiencies in scaffolding assemblies. The conducted test yielded satisfactory results, with a mean average precision of 0.89 and individual precision values of 0.96, 0.82, 0.90, and 0.89 for qualified frames and frames with the missing cross-tie rod, missing lower-tie rod, and missing footboard deficiencies, respectively. Subsequent field tests conducted at two construction sites demonstrated improved system performance compared to the training test. Furthermore, the advantages and disadvantages of employing mobile phones and AR goggles were discussed, elucidating certain limitations of the SADDS system, such as self-occlusion and efficiency issues. [ABSTRACT FROM AUTHOR]

Background: Computer assisted surgical navigation systems are designed to improve outcomes by providing clinicians with procedural guidance information. The use of new technologies, such as mixed reality, offers the potential for more intuitive, efficient, and accurate procedural guidance. The goal of this study is to assess the positional accuracy and consistency of a clinical mixed reality system that utilizes commercially available wireless head-mounted displays (HMDs), custom software, and localization instruments. Methods: Independent teams using the second-generation Microsoft HoloLens© hardware, Medivis SurgicalAR© software, and localization instruments, tested the accuracy of the combined system at different institutions, times, and locations. The ASTM F2554-18 consensus standard for computer-assisted surgical systems, as recognized by the U.S. FDA, was utilized to measure the performance. 288 tests were performed. Results: The system demonstrated consistent results, with an average accuracy performance that was better than one millimeter (.75 ± SD.37 mm). Conclusion: Independently acquired positional tracking accuracies exceed conventional in-market surgical navigation tracking systems and FDA standards. Importantly, the performance was achieved at two different institutions, using an international testing standard, and with a system that included a commercially available off-the-shelf wireless head mounted display and software. [ABSTRACT FROM AUTHOR]

In this paper, we present an exploratory study on the potential impact of holographic heart models and mixed reality technology on medical training, and in particular in teaching complex Congenital Heart Diseases (CHD) to medical students. Fifty-nine medical students were randomly allocated into three groups. Each participant in each group received a 30-minute lecture on a CHD condition interpretation and transcatheter treatment with different instructional tools. The participants of the first group attended a lecture in which traditional slides were projected onto a flat screen (group “regular slideware”, RS). The second group was shown slides incorporating videos of holographic anatomical models (group “holographic videos”, HV). Finally, those in the third group wore immersive, head-mounted devices (HMD) to interact directly with holographic anatomical models (group “mixed reality”, MR). At the end of the lecture, the members of each group were asked to fill in a multiple-choice questionnaire aimed at evaluating their topic proficiency, as a proxy to evaluate the effectiveness of the training session (in terms of acquired notions); participants from group MR were also asked to fill in a questionnaire regarding the recommendability and usability of the MS Hololens HMDs, as a proxy of satisfaction regarding its use experience (UX). The findings show promising results for usability and user acceptance. [ABSTRACT FROM AUTHOR]

The use of extended reality technologies such as virtual reality and mixed reality or augmented reality has increased significantly in multiple areas in recent years, including in orthopedic surgery. In shoulder replacement, there are applications of these technologies for preoperative planning and intraoperative execution as well as for education and training, and possibly also for postoperative rehabilitation. [ABSTRACT FROM AUTHOR]

The calibration of three-dimensional (3D) coordinates in augmented reality systems is a complex activity. It involves the recognition of environmental characteristics and technology that models the 3D space of the device, determining its position and orientation. Single markers suffer from numerical instability, particularly when they are small within the camera image. In the industrial environment, it is common for augmented reality applications to cover large spaces, making it difficult to maintain attributes such as precision and accuracy. To address this issue, our study proposes a two-step calibration model that leverages multiple markers for accurate localization in a larger indoor environment. We developed the calibration model using Unity3D, Mixed Reality ToolKit, and Vuforia and evaluated it in terms of precision and accuracy in a proof of concept using the MS Hololens device. Our findings reveal that employing two markers significantly reduces angular discrepancies between points in the real and augmented environments. Moreover, our results underscore that registration accuracy improves as the number of calibration points increases. The results show improvements in determining the axes that define the 3D space, with a direct influence on the position of the points observed in the experiment. [ABSTRACT FROM AUTHOR]

Inaccurate screw insertion has been a major concern in pedicle screw placement in spinal surgery because of the potential for postoperative complications. Efforts to improve screw insertion have led to the development of novel surgical navigation systems. However, current surgical navigation systems have several problems in that the attention of surgeons is frequently interrupted during surgery, and complications occur in their depth perception because of the limitations of 2D monitors. This study proposes a surgical navigation system to address these issues using mixed reality (MR)-based smart glasses. We developed a navigation system that provides 3D visualization of the surgical tool with respect to the patient's anatomy. It utilizes preoperative surgical planning data to provide visual feedback and guidance. An orthopedic surgeon performed pedicle screw placement with the navigation system in two different environments, using Microsoft HoloLens (HoloLens) and a 2D monitor. The results were evaluated according to procedural time, translational error, angular error, and clinical accuracy. The mean procedural time was 111.3 ± 52.7 s with the HoloLens and 192.1 ± 104.0 s with the 2D monitor. The mean translational error was 2.14 ± 1.13 mm at the entry and 3.14 ± 0.90 mm at the target with HoloLens. With the 2D monitor, the mean translational error was 2.10 ± 0.97 mm and 3.41 ± 2.16 mm at the entry and the target, respectively. The mean angular error was 6.44 ± 1.94 deg with HoloLens and 7.14 ± 4.20 deg with the 2D monitor. All screws were inserted intrapedicularly in both environments. The navigation system enables free visualization, reflects the human eye's perspective, and retains the advantages of MR-based smart glasses. A navigation system compatible with minimally invasive surgery should be developed in the future. [ABSTRACT FROM AUTHOR]

Craniomaxillofacial surgeries are performed using custom-made physical cutting guides and resin dental splints that present several drawbacks (e.g. time and cost required for their design and production). The literature commonly provides augmented/mixed reality (AR/MR) solutions for assisting maxillofacial osteotomies and repositioning without any interactive guide. This work proposes a new MR application, useful for osteotomy and repositioning, providing interactive, fast, and intuitive feedback to the surgeon, who is then supported in performing the bone fragment resection and replacement frame by frame. The proposed application speeds up the surgery and reduces under/overshooting errors. Moreover, the idea of integrating osteotomy and repositioning assistance in the same MR application is rarely found in the literature. It is an entirely novel approach to craniomaxillofacial surgery. The MR application has been designed with a three-button menu. The "App Start" calibrates the app, the "Osteotomy Mode" visualises the holograms of the cutting lines and drilling points, and the "Repositioning Mode" visualises the step-by-step real-time feedback to precisely support the surgeon placing the osteotomised bone fragment towards the final pre-planned position. The MR app has been developed in Unity and deployed on Microsoft HoloLens V2. A laboratory test bench was realised to validate the accuracy of the proposed MR-based approach. The validation protocol consists of two tasks to test the osteotomy and repositioning modes using a 3D-printed skull phantom. For osteotomy, the accuracy is 0.89 mm (genioplasty), 1.24 mm (maxillary osteotomy), 1.33 mm (orthognathic surgery), and 2.89 mm (mandibular angle osteotomy). For repositioning, the accuracy is 0.6 mm (anteroposterior deviation), 0.7 mm (mediolateral deviation), and 0.6° (angular deviation). [ABSTRACT FROM AUTHOR]

Background: Connectomic modeling studies are expanding understanding of the brain networks that are modulated by deep brain stimulation (DBS) therapies. However, explicit integration of these modeling results into prospective neurosurgical planning is only beginning to evolve. One challenge of employing connectomic models in patient-specific surgical planning is the inherent 3D nature of the results, which can make clinically useful data integration and visualization difficult. Methods: We developed a holographic stereotactic neurosurgery research tool (HoloSNS) that integrates patient-specific brain models into a group-based visualization environment for interactive surgical planning using connectomic hypotheses. HoloSNS currently runs on the HoloLens 2 platform and it enables remote networking between headsets. This allowed us to perform surgical planning group meetings with study co-investigators distributed across the country. Results: We used HoloSNS to plan stereo-EEG and DBS electrode placements for each patient participating in a clinical trial (NCT03437928) that is targeting both the subcallosal cingulate and ventral capsule for the treatment of depression. Each patient model consisted of multiple components of scientific data and anatomical reconstructions of the head and brain (both patient-specific and atlas-based), which far exceed the data integration capabilities of traditional neurosurgical planning workstations. This allowed us to prospectively discuss and evaluate the positioning of the electrodes based on novel connectomic hypotheses. Conclusions: The 3D nature of the surgical procedure, brain imaging data, and connectomic modeling results all highlighted the utility of employing holographic visualization to support the design of unique clinical experiments to explore brain network modulation with DBS.

Increasing patient safety during operations is an inevitable topic in anaesthesiology. The cockpit strategy presented by Vogelsang et al. addresses this issue with elements such as closed loop communication and the use of checklists. In this paper, the use of checklists is taken up further. An AR application is presented that augments checklists for anaesthesia induction into the user’s field of view. The application is to be implemented in teaching and help prospective physicians to internalise the workflows in anaesthesiology more quickly and sustainably. A working prototype has been created and loaded onto the HoloLens 2. The prototype is being assessed and evaluated by an anaesthetist. The feedback will be included into the following development steps. The application will be further adapted to the needs and requirements of the users. In order to evaluate the usability and the user experience, students and medical professionals should test and evaluate the application repeatedly.

Abstract Background The purpose of this study was to explore the applicability of application effect of head-mounted mixed reality (MR) equipment combined with a three-dimensional (3D) printed model in neurosurgical ventricular and haematoma puncture training. Methods Digital Imaging and Communications in Medicine (DICOM) format image data of two patients with common neurosurgical diseases (hydrocephalus and basal ganglia haemorrhage) were imported into 3D Slicer software for 3D reconstruction, saved, and printed using 3D printing to produce a 1:1-sized head model with real person characteristics. The required model (brain ventricle, haematoma, puncture path, etc.) was constructed and imported into the head-mounted MR device, HoloLens, and a risk-free, visual, and repeatable system was designed for the training of junior physicians. A total of 16 junior physicians who studied under this specialty from September 2020 to March 2022 were selected as the research participants, and the applicability of the equipment and model during training was evaluated with assessment score sheets and questionnaires after training. Results According to results of the assessment and questionnaire, the doctors trained by this system are more familiar with the localization of the lateral anterior ventricle horn puncture and the common endoscopic surgery for basal ganglia haemorrhage, as well as more confident in the mastery of these two operations than the traditional training methods. Conclusions The use of head-mounted MR equipment combined with 3D printing models can provide an ideal platform for the operation training of young doctors. Through holographic images created from the combination of virtual and real images, operators can be better immersed in the operation process and deepen their understanding of the operation and related anatomical structures. The 3D printed model can be repeatedly reproduced so that doctors can master the technology, learn from mistakes, better achieve the purpose of teaching and training, and improve the effect of training.

In the global context, advancements in technology and science have rendered virtual, augmented, and mixed-reality technologies capable of transforming clinical care and medical environments by offering enhanced features and improved healthcare services. This paper aims to present a mixed reality-based system to control a robotic wheelchair for people with limited mobility. The test group comprised 11 healthy subjects (six male, five female, mean age 35.2 ± 11.7 years). A novel platform that integrates a smart wheelchair and an eye-tracking-enabled head-mounted display was proposed to reduce the cognitive requirements needed for wheelchair movement and control. The approach’s effectiveness was demonstrated by evaluating our system in realistic scenarios. The demonstration of the proposed AR head-mounted display user interface for controlling a smart wheelchair and the results provided in this paper could highlight the potential of the HoloLens 2-based innovative solutions and bring focus to emerging research topics, such as remote control, cognitive rehabilitation, the implementation of patient autonomy with severe disabilities, and telemedicine.

Augmented reality devices are increasingly accepted in health care, though most applications involve education and pre-operative planning. A novel augmented reality ultrasound application, HoloUS, was developed for the Microsoft HoloLens 2 to project real-time ultrasound images directly into the user's field of view. In this work, we assessed the effect of using HoloUS on vascular access procedural outcomes. A single-center user study was completed with participants with (N = 22) and without (N = 12) experience performing ultrasound-guided vascular access. Users completed a venipuncture and aspiration task a total of four times: three times on study day 1, and once on study day 2 between 2 and 4 weeks later. Users were randomized to use conventional ultrasound during either their first or second task and the HoloUS application at all other times. Task completion time, numbers of needle re-directions, head adjustments and needle visualization rates were recorded. For expert users, task completion time was significantly faster using HoloUS (11.5 s, interquartile range [IQR] = 6.5–23.5 s vs. 18.5 s, IQR = 11.0–36.5 s; p = 0.04). The number of head adjustments was significantly lower using the HoloUS app (1.0, IQR = 0.0–1.0 vs. 3.0, IQR = 1.0–5.0; p < 0.0001). No significant differences were identified in other measured outcomes. This is the first investigation of augmented reality-based ultrasound-guided vascular access using the second-generation HoloLens. It demonstrates equivalent procedural efficiency and accuracy, with favorable usability, ergonomics and user independence when compared with traditional ultrasound techniques. [ABSTRACT FROM AUTHOR]

Connectomic modeling studies are expanding understanding of the brain networks that are modulated by deep brain stimulation (DBS) therapies. However, explicit integration of these modeling results into prospective neurosurgical planning is only beginning to evolve. One challenge of employing connectomic models in patient-specific surgical planning is the inherent 3D nature of the results, which can make clinically useful data integration and visualization difficult. We developed a holographic stereotactic neurosurgery research tool (HoloSNS) that integrates patient-specific brain models into a group-based visualization environment for interactive surgical planning using connectomic hypotheses. HoloSNS currently runs on the HoloLens 2 platform and it enables remote networking between headsets. This allowed us to perform surgical planning group meetings with study co-investigators distributed across the country. We used HoloSNS to plan stereo-EEG and DBS electrode placements for each patient participating in a clinical trial (NCT03437928) that is targeting both the subcallosal cingulate and ventral capsule for the treatment of depression. Each patient model consisted of multiple components of scientific data and anatomical reconstructions of the head and brain (both patient-specific and atlas-based), which far exceed the data integration capabilities of traditional neurosurgical planning workstations. This allowed us to prospectively discuss and evaluate the positioning of the electrodes based on novel connectomic hypotheses. The 3D nature of the surgical procedure, brain imaging data, and connectomic modeling results all highlighted the utility of employing holographic visualization to support the design of unique clinical experiments to explore brain network modulation with DBS. • Developed a collaborative visualization tool for stereotactic neurosurgical planning. • Generated surgical plans for six TRD subjects treated with SCC and VC DBS. • Interactively visualized multi-modal brain connectivity between the DBS and sEEG electrodes. [ABSTRACT FROM AUTHOR]

Purpose: Over the last years, interest and efforts to implement augmented reality (AR) in orthopedic surgery through head-mounted devices (HMD) have increased. However, the majority of experiments were preclinical and within a controlled laboratory environment. The operating room (OR) is a more challenging environment with various confounding factors potentially affecting the performance of an AR-HMD. The aim of this study was to assess the performance of an AR-HMD in a real-life OR setting. Methods: An established AR application using the HoloLens 2 HMD was tested in an OR and in a laboratory by two users. The accuracy of the hologram overlay, the time to complete the trial, the number of rejected registration attempts, the delay in live overlay of the hologram, and the number of completely failed runs were recorded. Further, different OR setting parameters (light condition, setting up partitions, movement of personnel, and anchor placement) were modified and compared. Results: Time for full registration was higher with 48 s (IQR 24 s) in the OR versus 33 s (IQR 10 s) in the laboratory setting (p < 0.001). The other investigated parameters didn't differ significantly if an optimal OR setting was used. Within the OR, the strongest influence on performance of the AR-HMD was different light conditions with direct light illumination on the situs being the least favorable. Conclusion: AR-HMDs are affected by different OR setups. Standardization measures for better AR-HMD performance include avoiding direct light illumination on the situs, setting up partitions, and minimizing the movement of personnel. [ABSTRACT FROM AUTHOR]