Search

Background: Endoscope retrograde cholangiopancreatography is a standard surgical treatment for gallbladder and pancreatic diseases. However, surgeons is at high risk and require sufficient surgical experience and skills., Methods: (1) The simultaneous localisation and mapping technique to reconstruct the surgical environment. (2) The preoperative 3D model is transformed into the intraoperative video environment to implement the multi-modal fusion. (3) A framework for virtual-to-real projection based on hand-eye alignment. For the purpose of projecting the 3D model onto the imaging plane of the camera, it uses position data from electromagnetic sensors., Results: Our AR-assisted navigation system can accurately guide physicians, which means a distance of registration error to be restricted to under 5 mm and a projection error of 5.76 ± 2.13, and the intubation procedure is done at 30 frames per second., Conclusions: Coupled with clinical validation and user studies, both the quantitative and qualitative results indicate that our navigation system has the potential to be highly useful in clinical practice., (© 2024 John Wiley & Sons Ltd.)

Background: Controlling a multi-grasp prosthetic hand still remains a challenge. This study explores the influence of merging gaze movements and augmented reality in bionics on improving prosthetic hand control., Methods: A control system based on gaze movements, augmented reality, and myoelectric signals (i-MYO) was proposed. In the i-MYO, the GazeButton was introduced into the controller to detect the grasp-type intention from the eye-tracking signals, and the proportional velocity scheme based on the i-MYO was used to control hand movement., Results: The able-bodied subjects with no prior training successfully transferred objects in 91.6% of the cases and switched the optimal grasp types in 97.5%. The patient could successfully trigger the EMG to control the hand holding the objects in 98.7% of trials in around 3.2 s and spend around 1.3 s switching the optimal grasp types in 99.2% of trials., Conclusions: Merging gaze movements and augmented reality in bionics can widen the control bandwidth of prosthetic hand. With the help of i-MYO, the subjects can control a prosthetic hand using six grasp types if they can manipulate two muscle signals and gaze movement., (© 2023 John Wiley & Sons Ltd.)

Background: In order to provide accurate and reliable image guidance for augmented reality (AR) spinal surgery navigation, a spatial registration method has been proposed., Methods: In the AR spinal surgery navigation system, grayscale-based 2D/3D registration technology has been used to register preoperative computed tomography images with intraoperative X-ray images to complete the spatial registration, and then the fusion of virtual image and real spine has been realised., Results: In the image registration experiment, the success rate of spine model registration was 90%. In the spinal model verification experiment, the surface registration error of the spinal model ranged from 0.361 to 0.612 mm, and the total average surface registration error was 0.501 mm., Conclusion: The spatial registration method based on 2D/3D registration technology can be used in AR spinal surgery navigation systems and is highly accurate and minimally invasive., (© 2023 John Wiley & Sons Ltd.)

Background: Liver medical procedures are considered one of the most challenging because of the liver's complex geometry, heterogeneity, mechanical properties, and movement due to respiration. Haptic features integrated into needle insertion systems and other medical devices could support physicians but are uncommon. Additional training time and safety concerns make it difficult to implement in robot-assisted surgery. The main challenges of any haptic device in a teleoperated system are the stability and transparency levels required to develop a safe and efficient system that suits the physician's needs., Purpose: The objective of the review article is to investigate whether haptic-based teleoperation potentially improves the efficiency and safety of liver needle insertion procedures compared with insertion without haptic feedback. In addition, it looks into haptic technology that can be integrated into simulators to train novice physicians in liver procedures., Methods: This review presents the physician's needs during liver interventions and the consequent requirements of haptic features to help the physician. This paper provides an overview of the different aspects of a teleoperation system in various applications, especially in the medical field. It finally presents the state-of-the-art haptic technology in robot-assisted procedures for the liver. This includes 3D virtual models of the liver and force measurement techniques used in haptic rendering to estimate the real-time position of the surgical instrument relative to the liver., Results: Haptic feedback technology can be used to navigate the surgical tool through the desired trajectory to reach the target accurately and avoid critical regions. It also helps distinguish between various textures of liver tissue., Conclusion: Haptic feedback can complement the physician's experience to compensate for the lack of real-time imaging during Computed Tomography guided (CT-guided) liver procedures. Consequently, it helps the physician mitigate the destruction of healthy tissues and takes less time to reach the target., (© 2023 The Authors. The International Journal of Medical Robotics and Computer Assisted Surgery published by John Wiley & Sons Ltd.)

Background: Ventricular puncture is a common procedure in neurosurgery and the first that resident must learn. Ongoing education is critical to improving patient outcomes. However, training at the expense of potential risk to patients warrants new and safer training methods for residents., Methods: An augmented reality (AR) simulator for the practice of ventricular punctures was designed. It consists of a navigation system with a virtual 3D projection of the anatomy over a 3D-printed patient model. Forty-eight participants from neurosurgery staff performed two free-hand ventricular punctures before and after a training session., Results: Participants achieved enhanced accuracy in reaching the target at the Monro foramen after practicing with the system. Additional metrics revealed significantly better trajectories after the training., Conclusion: The study confirms the feasibility of AR as a training tool. This motivates future work towards standardising new educative methodologies in neurosurgery., (© 2023 The Authors. The International Journal of Medical Robotics and Computer Assisted Surgery published by John Wiley & Sons Ltd.)

Background: The existing augmented reality (AR) dental implant surgery navigation system usually uses markers to complete image guidance. However, markers often affect dentists' operations and make patients uncomfortable., Methods: To solve the problems caused by markers, this paper proposes an effective marker-less image guidance method. After initialisation is completed by contour matching, the corresponding relationship is obtained by matching the feature points between the current frame and the preloaded initial frame. The camera pose is estimated by solving the Perspective-n-Point problem., Results: The registration error of AR images is 0.731 ± 0.144 mm. The planting errors are 1.174 ± 0.241 mm at the neck, 1.433 ± 0.389 mm at the apex and 5.566 ± 2.102° for the angle. The maximum error and standard deviation meet the clinical requirements., Conclusions: We demonstrate that the proposed method can accurately guide dentists to perform dental implant surgery., (© 2023 John Wiley & Sons Ltd.)

Background: Augmented reality (AR) is a new human-computer interaction technology that combines virtual reality, computer vision, and computer networks. With the rapid advancement of the medical field towards intelligence and data visualisation, AR systems are becoming increasingly popular in the medical field because they can provide doctors with clear enough medical images and accurate image navigation in practical applications. However, it has been discovered that different display types of AR systems have different effects on doctors' perception of the image after virtual-real fusion during the actual medical application. If doctors cannot correctly perceive the image, they may be unable to correctly match the virtual information with the real world, which will have a significant impact on their ability to recognise complex structures., Methods: This paper uses Citespace, a literature analysis tool, to visualise and analyse the research hotspots when AR systems are used in the medical field., Results: A visual analysis of the 1163 articles retrieved from the Web of Science Core Collection database reveals that display technology and visualisation technology are the key research directions of AR systems at the moment., Conclusion: This paper categorises AR systems based on their display principles, reviews current image perception optimisation schemes for various types of systems, and analyses and compares different display types of AR systems based on their practical applications in the field of smart medical care so that doctors can select the appropriate display types based on different application scenarios. Finally, the future development direction of AR display technology is anticipated in order for AR technology to be more effectively applied in the field of smart medical care. The advancement of display technology for AR systems is critical for their use in the medical field, and the advantages and disadvantages of various display types should be considered in different application scenarios to select the best AR system., (© 2023 John Wiley & Sons Ltd.)

Purpose: The work presents outcomes of simulated maxillofacial skeleton osteotomies supported with cutting guides or one of two AR-based intraoperative navigation systems., Material and Methods: The series of osteotomies supported with a cutting guide, simple AR (sAR) or navigated AR (nAR) module were carried out on 15 (five per each method) skull models according to the virtual surgical plan (VSP). Each method was used to support 40 osteotomies involving the upper jaw and 40 involving the orbital region (16 osteotomies on each model). Postoperative computed tomography scans were fused with the VSP to analyse angular deviations from the planned cutting trajectory (°) and the deviations of labelled control points (mm)., Results: Guides provided the highest accuracy, with a mean osteotomy angular deviation of 3.73 ± 2.94° and a mean control point deviation of 1.30 ± 0.73 mm. Mean angular deviations for the sAR- and nAR-assisted osteotomies were 5.93 ± 5.12° and 6.75 ± 5.33°, and mean control point deviations amounted to 1.86 ± 0.88 mm and 1.97 ± 0.70 mm., (© 2023 John Wiley & Sons Ltd.)

Background: This systematic review aims to critically appraise the intraoperative use of augmented and mixed reality technology to improve surgical outcomes., Method: A literature search of PubMed, Scopus, Cochrane Central Register of Controlled Trials, ClinicalTrials.gov and WHO ICTRP was performed in accordance with Cochrane Handbook for Systematic Reviews of Interventions., Results: This review included 94 studies on 2473 patients, comprising 78 studies on augmented reality and 16 on mixed reality. This technology has seen broad intraoperative application. Augmented and mixed reality can reduce operative duration, blood loss, and the duration of inpatient care. Current evidence shows that they achieve this most in percutaneous surgery., Conclusions: Augmented and mixed reality technology improve surgical outcomes by increasing navigational speed and reducing navigational error intraoperatively. However, they have technical limitations which are the subject of ongoing research. Further studies are necessary to define how this technology is best applied intraoperatively., Systematic Review Registration: PROSPERO CRD42020205892., (© 2022 John Wiley & Sons Ltd.)

Background: Oral and maxillofacial surgery demands high precision navigation to achieve optimal surgical outcomes. Augmented reality (AR) has been a promising solution for intuitive and accurate guidance, whereas existing systems have room for improvement., Methods: We propose a high quality and accurate in situ AR navigation system. Enhanced image quality is achieved by deploying eye tracking in lenticular-based autostereoscopic display. Viewpoint tracking and optical tracking are integrated to assure accurate in situ images., Results: The proposed system can provide in situ AR images in real-time, with a viewing angle of 59.5° × 49.1°, 3D image resolution of 0.35 × 0.21 mm, and image accuracy of 0.99 ± 0.70 mm. A maxillary drilling experiment obtains average position error of 1.12 ± 0.30 mm and localization time of 8.7 ± 3.9 s, significantly better than conventional 2D navigation system., Conclusions: The proposed system can display high-quality AR images and therefore reduce positioning error and operating time., (© 2022 John Wiley & Sons Ltd.)

Background: Recognition of markers in the augmented reality system can reduce the additional cost of a guide plate required for the removal of benign tumours in oral and maxillofacial surgery, but the use of markers often has complex problems., Method: In order to avoid the complex problem of using markers, an augmented reality system based on a marker-free registration method was proposed to track the contour of the mandible edge. Use the computer to perform preoperative planning on the jaw model, select and mark the path of maxillofacial lesion resection., Results: This method has an error of the surface matching was 0.6453 ± 0.2826 $0.6453\pm 0.2826$ mm, and an error of the surgical resection was 0.4858 ± 0.3712 $0.4858\pm 0.3712$ mm., Conclusions: We demonstrate that the system can accurately enhance the display of the surgical path and provide guidance for the tradition of maxillofacial surgery., (© 2022 John Wiley & Sons Ltd.)

Introduction: MRI-guided targeted biopsy has become standard of care for diagnosis of prostate cancer, with establishment of several biopsy techniques and platforms. Augmented reality smart glasses have emerged as novel technology to support image-guided interventions. We aimed to investigate its usage while prostate biopsy., Methods: MRI with PIRADS-lesions ≥3 was uploaded to smart glasses (Vuzix Blade R ) and augmented reality smart glasses-assisted targeted biopsy (SMART-TB) of the prostate was performed using cognitive fusion technology at the point of care. Detection rates were compared to systematic biopsy. Feasibility for SMART-TB was assessed (10 domains from bad [1] to excellent [10])., Results: SMART-TB was performed for four patients. Prostate cancer detection was more likely for SMART-TB (46%; 13/28) than for systematic biopsy (27%; 13/48). Feasibility scores were high [8-10] for practicality, multitasking, execution speed, comfort and device weight and low [1-4] for handling, battery and image quality. Median execution time: 28 min; Investment cost smart glass: 1017 USD., Conclusion: First description of SMART-TB demonstrated convenient feasibility. This novel technology might enhance diagnosis of prostate cancer in future., (© 2022 The Authors. The International Journal of Medical Robotics and Computer Assisted Surgery published by John Wiley & Sons Ltd.)

Background: Augmented reality (AR) has been widely researched for use in healthcare. Prior AR for robot-assisted minimally invasive surgery has mainly focussed on superimposing preoperative three-dimensional (3D) images onto patient anatomy. This article presents alternative interactive AR tools for robotic surgery., Methods: We designed, built and evaluated four voice-controlled functions: viewing a live video of the operating room, viewing two-dimensional preoperative images, measuring 3D distances and warning about out-of-view instruments. This low-cost system was developed on a da Vinci Si, and it can be integrated into surgical robots equipped with a stereo camera and a stereo viewer., Results: Eight experienced surgeons performed dry-lab lymphadenectomies and reported that the functions improved the procedure. They particularly appreciated the possibility of accessing the patient's medical records on demand, measuring distances intraoperatively and interacting with the functions using voice commands., Conclusions: The positive evaluations garnered by these alternative AR functions and interaction methods provide support for further exploration., (© 2021 The Authors. The International Journal of Medical Robotics and Computer Assisted Surgery published by John Wiley & Sons Ltd.)

Background: The existing augmented reality (AR) based neuronavigation systems typically require markers and additional tracking devices for model registration, which causes excessive preparatory steps., Methods: For fast and accurate intraoperative navigation, this work proposes a marker-less AR system that tracks the head features with the monocular camera. After the semi-automatic initialization process, the feature points between the captured image and the pre-loaded keyframes are matched for obtaining correspondences. The camera pose is estimated by solving the Perspective-n-Point problem., Results: The localization error of AR visualization on scalp and falx meningioma is 0.417 ± 0.057 and 1.413 ± 0.282 mm, respectively. The maximum localization error is less than 2 mm. The AR system is robust to occlusions and changes in viewpoint and scale., Conclusions: We demonstrate that the developed system can successfully display the augmented falx meningioma with enough accuracy and provide guidance for neurosurgeons to locate the tumour in brain., (© 2021 John Wiley & Sons Ltd.)

Background: Defects of the facial skeleton often require complex reconstruction with vascularized grafts. This trial elucidated the usability, visual perception and accuracy of a markerless augmented reality (AR)-guided navigation for harvesting iliac crest transplants., Methods: Random CT scans were used to virtually plan two common transplant configurations on 10 iliac crest models, each printed four times. The transplants were harvested using projected AR and cutting guides. The duration and accuracies of the angulation, distance and volume between the planned and executed osteotomies were measured., Results: AR was characterized by the efficient use of time and accurate rendition of preoperatively planned geometries. However, vertical osteotomies and complex anatomical settings displayed significant inferiority of AR guidance compared to cutting guides., Conclusions: This study demonstrated the usability of a markerless AR setup for harvesting iliac crest transplants. The visual perception and accuracy of the AR-guided osteotomies constituted remaining weaknesses against cutting guide technology., (© 2021 The Authors. The International Journal of Medical Robotics and Computer Assisted Surgery published by John Wiley & Sons Ltd.)

Background: Insulinomas are rare insulin-producing pancreatic neuroendocrine tumours leading to severe episodes of hypoglycaemia. Surgery is the predominant curative therapy., Methods: We report here the first paediatric case of an insulinoma of the pancreatic body resected completely robotically under ultrasound guidance in a 10-year-old male with multiple endocrine neoplasia type 1. The port set-up was adapted for the narrowed dimensions of the paediatric peritoneal space. We comment on technical key steps for the organ-preserving procedure that was performed in close proximity to critical anatomic structures, with supporting video. Preoperative diagnostics, including endoscopic ultrasound, to determine surgical management are highlighted., Results: Following an uneventful post-operative course, the boy was discharged on day 11 with normalised glucose-metabolism. A pseudocyst developing after 4 weeks was treated with endoscopic stenting., Conclusions: The applicability of a robotic surgical system in limited space conditions such as found in the paediatric abdominal cavity is demonstrated here for pancreatic surgery., (© 2021 The Authors. The International Journal of Medical Robotics and Computer Assisted Surgery published by John Wiley & Sons Ltd.)

Background: User interfaces play a vital role in the planning and execution of an interventional procedure. The objective of this study is to investigate the effect of using different user interfaces for planning transrectal robot-assisted MR-guided prostate biopsy (MRgPBx) in an augmented reality (AR) environment., Method: End-user studies were conducted by simulating an MRgPBx system with end- and side-firing modes. The information from the system to the operator was rendered on HoloLens as an output interface. Joystick, mouse/keyboard, and holographic menus were used as input interfaces to the system., Results: The studies indicated that using a joystick improved the interactive capacity and enabled operator to plan MRgPBx in less time. It efficiently captures the operator's commands to manipulate the augmented environment representing the state of MRgPBx system., Conclusions: The study demonstrates an alternative to conventional input interfaces to interact and manipulate an AR environment within the context of MRgPBx planning., (© 2021 John Wiley & Sons Ltd.)

Background: Tele-mentoring facilitates the transfer of surgical knowledge. The objective of this work is to develop a tele-mentoring framework that enables a specialist surgeon to mentor an operating surgeon by transferring information in a form of surgical instruments' motion required during a minimally invasive surgery., Method: A tele-mentoring framework is developed to transfer video stream of the surgical field, poses of the scope and port placement from the operating room to a remote location. From the remote location, the motion of virtual surgical instruments augmented onto the surgical field is sent to the operating room., Results: The proposed framework is suitable to be integrated with laparoscopic as well as robotic surgeries. It takes on average 1.56 s to send information from the operating room to the remote location and 0.089 s for vice versa over a local area network., Conclusions: The work demonstrates a tele-mentoring framework that enables a specialist surgeon to mentor an operating surgeon during a minimally invasive surgery., (© 2021 The Authors. The International Journal of Medical Robotics and Computer Assisted Surgery published by John Wiley & Sons Ltd.)

Background and Aim: Most of the mixed reality models used in the surgical telepresence are suffering from the discrepancies in the boundary area and spatial-temporal inconsistency due to the illumination variation in the video frames. The aim behind this work is to propose a new solution that helps produce the composite video by merging the augmented video of the surgery site and virtual hand of the remote expertise surgeon. The purpose of the proposed solution is to decrease the processing time and enhance the accuracy of merged video by decreasing the overlay and visualization error and removing occlusion and artefacts., Methodology: The proposed system enhanced mean-value cloning algorithm that helps to maintain the spatial-temporal consistency of the final composite video. The enhanced algorithm includes the three-dimensional mean-value coordinates and improvised mean-value interpolant in the image cloning process, which helps to reduce the sawtooth, smudging and discolouration artefacts around the blending region., Results: The accuracy in terms of overlay error of the proposed solution is improved from 1.01 to 0.80 mm, whereas the accuracy in terms of visualization error is improved from 98.8% to 99.4%. The processing time is reduced to 0.173 s from 0.211 s. The processing time and the accuracy of the proposed solution are enhanced as compared to the state-of-art solution., Conclusion: Our solution helps make the object of interest consistent with the light intensity of the target image by adding the space distance that helps maintain the spatial consistency in the final merged video., (© 2021 John Wiley & Sons Ltd.)

Background and Aim: Image registration and alignment are the main limitations of augmented reality (AR)-based knee replacement surgery. This research aims to decrease the registration error, eliminate outcomes that are trapped in local minima to improve the alignment problems, handle the occlusion and maximize the overlapping parts., Methodology: Markerless image registration method was used for AR-based knee replacement surgery to guide and visualize the surgical operation. While weight least square algorithm was used to enhance stereo camera-based tracking by filling border occlusion in right-to-left direction and non-border occlusion from left-to-right direction., Results: This study has improved video precision to 0.57-0.61 mm alignment error. Furthermore, with the use of bidirectional points, that is, forward and backward directional cloud point, the iteration on image registration was decreased. This has led to improve the processing time as well. The processing time of video frames was improved to 7.4-11.74 frames per second., Conclusions: It seems clear that this proposed system has focused on overcoming the misalignment difficulty caused by the movement of patient and enhancing the AR visualization during knee replacement surgery. The proposed system was reliable and favourable which helps in eliminating alignment error by ascertaining the optimal rigid transformation between two cloud points and removing the outliers and non-Gaussian noise. The proposed AR system helps in accurate visualization and navigation of anatomy of knee such as femur, tibia, cartilage, blood vessels and so forth., (© 2021 John Wiley & Sons Ltd.)

Background: Existing surgical navigation approaches of the rod bending procedure in spinal fusion rely on optical tracking systems that determine the location of placed pedicle screws using a hand-held marker., Methods: We propose a novel, marker-less surgical navigation proof-of-concept to bending rod implants. Our method combines augmented reality with on-device machine learning to generate and display a virtual template of the optimal rod shape without touching the instrumented anatomy. Performance was evaluated on lumbosacral spine phantoms against a pointer-based navigation benchmark approach and ground truth data obtained from computed tomography., Results: Our method achieved a mean error of 1.83 ± 1.10 mm compared to 1.87 ± 1.31 mm measured in the marker-based approach, while only requiring 21.33 ± 8.80 s as opposed to 36.65 ± 7.49 s attained by the pointer-based method., Conclusion: Our results suggests that the combination of augmented reality and machine learning has the potential to replace conventional pointer-based navigation in the future., (© 2020 John Wiley & Sons Ltd.)

Background: Mixed reality (MR) visualization is gaining popularity in image-guided surgery (IGS) systems, especially for hard and soft tissue surgeries. However, a few MR systems are implemented in real time. Some factors are limiting MR technology and creating a difficulty in setting up and evaluating the MR system in real environments. Some of these factors include: the end users are not considered, the limitations in the operating room, and the medical images are not fully unified into the operating interventions., Methodology: The purpose of this article is to use Data, Visualization processing, and View (DVV) taxonomy to evaluate the current MR systems. DVV includes all the components required to be considered and validated for the MR used in hard and soft tissue surgeries. This taxonomy helps the developers and end users like researchers and surgeons to enhance MR system for the surgical field., Results: We evaluated, validated, and verified the taxonomy based on system comparison, completeness, and acceptance criteria. Around 24 state-of-the-art solutions that are picked relate to MR visualization, which is then used to demonstrate and validate this taxonomy. The results showed that most of the findings are evaluated and others are validated., Conclusion: The DVV taxonomy acts as a great resource for MR visualization in IGS. State-of-the-art solutions are classified, evaluated, validated, and verified to elaborate the process of MR visualization during surgery. The DVV taxonomy provides the benefits to the end users and future improvements in MR., (© 2020 John Wiley & Sons Ltd.)

Background: Neurosurgery has exceptionally high requirements for minimally invasive and safety. This survey attempts to analyze the practical application of AR in neurosurgical navigation. Also, this survey describes future trends in augmented reality neurosurgical navigation systems., Methods: In this survey, we searched related keywords "augmented reality", "virtual reality", "neurosurgery", "surgical simulation", "brain tumor surgery", "neurovascular surgery", "temporal bone surgery", and "spinal surgery" through Google Scholar, World Neurosurgery, PubMed and Science Direct. We collected 85 articles published over the past five years in areas related to this survey., Results: Detailed study has been conducted on the application of AR in neurosurgery and found that AR is constantly improving the overall efficiency of doctor training and treatment, which can help neurosurgeons learn and practice surgical procedures with zero risks., Conclusions: Neurosurgical navigation is essential in neurosurgery. Despite certain technical limitations, it is still a necessary tool for the pursuit of maximum security and minimal intrusiveness. This article is protected by copyright. All rights reserved., (This article is protected by copyright. All rights reserved.)

Background and Aim: Most of the Mixed Reality models used in the surgical telepresence are suffering from the discrepancies in the boundary area and spatial-temporal inconsistency due to the illumination variation in the video frames. The aim behind this work is to propose a new solution that helps produce the composite video by merging the augmented video of the surgery site and virtual hand of the remote expertise surgeon. The purpose of the proposed solution is to decrease the processing time and enhance the accuracy of merged video by decreasing the overlay and visualization error and removing occlusion and artefacts., Methodology: The proposed system enhanced the mean value cloning algorithm that helps to maintain the spatial-temporal consistency of the final composite video. The enhanced algorithm includes the 3D mean value coordinates and improvised mean value interpolant in the image cloning process, which helps to reduce the sawtooth, smudging and discoloration artefacts around the blending region RESULTS: As compared to the state of art solution, the accuracy in terms of overlay error of the proposed solution is improved from 1.01mm to 0.80mm whereas the accuracy in terms of visualization error is improved from 98.8% to 99.4%. The processing time is reduced to 0.173 seconds from 0.211 seconds CONCLUSION: Our solution helps make the object of interest consistent with the light intensity of the target image by adding the space distance that helps maintain the spatial consistency in the final merged video. This article is protected by copyright. All rights reserved., (This article is protected by copyright. All rights reserved.)

Background and Aim: Image registration and alignment are the main limitations of augmented reality-based knee replacement surgery. This research aims to decrease the registration error, eliminate outcomes that are trapped in local minima to improve the alignment problems, handle the occlusion and maximize the overlapping parts., Methodology: markerless image registration method was used for Augmented reality-based knee replacement surgery to guide and visualize the surgical operation. While weight least square algorithm was used to enhance stereo camera-based tracking by filling border occlusion in right to left direction and non-border occlusion from left to right direction., Results: This study has improved video precision to 0.57 mm ∼ 0.61 mm alignment error. Furthermore, with the use of bidirectional points, i.e. Forwards and backwards directional cloud point, the iteration on image registration was decreased. This has led to improved the processing time as well. The processing time of video frames was improved to 7.4 ∼11.74 fps., Conclusions: It seems clear that this proposed system has focused on overcoming the misalignment difficulty caused by movement of patient and enhancing the AR visualization during knee replacement surgery. The proposed system was reliable and favourable which helps in eliminating alignment error by ascertaining the optimal rigid transformation between two cloud points and removing the outliers and non-Gaussian noise. The proposed augmented reality system helps in accurate visualization and navigation of anatomy of knee such as femur, tibia, cartilage, blood vessels, etc. This article is protected by copyright. All rights reserved., (This article is protected by copyright. All rights reserved.)

Background: Augmented reality (AR) is gaining attention in medicine because of the convenience and innovation that it brings to operating rooms. Furthermore, oral and maxillofacial surgery (OMS), which is one of sensitive and narrow spatial surgery, requires high accuracy in image registration and low processing time of the system. However, the current systems are suffering from image registration problems while matching two different posture images. We thus aimed to increase that overlay accuracy and decrease the processing time., Methodology: The proposed system consists of an Iterative Closest Point (ICP) algorithm, which is the combination of a rotation invariant and Manhattan error metric, to provide the best initial parameters and to decrease the computational cost by sorting high and low processing pixel images, respectively., Result: The study on maxillary and mandibular jaw bone demonstrates that the proposed work overlay accuracy ranges from 0.22 to 0.30 mm, and processing time ranges from 10 to 14 frames per second as opposed to the 0.23- to 0.35-mm overlay accuracy and the current 8 to 12 frames per second processing time., Conclusion: This research aimed to improve the visualization and fast AR system for the OMS. Thus, the proposed system achieved an improvement in overlay accuracy and processing time by implementing the Rotation Invariant and Manhattan error metric ICP algorithm., (© 2020 John Wiley & Sons, Ltd.)

Background and Aim: Jaw surgery based on augmented reality (AR) still has limitations in terms of navigating narrow areas. Surgeons need to avoid nerves, vessels, and teeth in their entirety, not just root canals. Inaccurate positioning of the surgical instrument may lead to positional or navigational errors and can result in cut blood vessels, nerve channels, or root canals. This research aims to decrease the positional error during surgery and improve navigational accuracy by reducing the positional error., Methodology: The proposed 2D/3D system tracks the surgical instrument, consisting of the shaft and the cutting element, each part being assigned a different feature description. In the case of the 3D position estimation, the input vector is composed of image descriptors of the instrument and the output value consists of 3D coordinates of the cutter., Results: Sample results from a jawbone-maxillary and mandibular jaw-demonstrate that the positional error is reduced. The system, thus, led to an improvement in alignment of the video accuracy by 0.25 to 0.35 mm from 0.40 to 0.55 mm and a decrease in processing time of 11 to 14 frames per second (fps) against 8 to 12 fps of existing solutions., Conclusion: The proposed system is focused on overlaying only on the area to be operated on. Thus, this AR-based study contributes to accuracy in navigation of the deeper anatomical corridors through increased accuracy in positioning of surgical instruments., (© 2020 John Wiley & Sons, Ltd.)

Background: Augmented reality (AR) is still a primarily theoretical concept in areas such as bowel, liver, gallbladder, and jaw surgeries because of the limitation of visualization accuracy of hidden organs and internal structures. This paper aims to improve the cutting accuracy, visualizing accuracy, and processing time of the augmented video., Methodology: The proposed system consists of an enhanced block-matching algorithm (BMA) with ghosting map technique., Results: Results proved that proposed system reduced the visualization error, which ranges from 1.48 to 1.83 mm against the existing system visualization error 1.67 to 2.0. Similarly, the processing time also improved 59 to 72 ms/frame over the 50 to 58 ms/frame., Conclusion: This study showed the improvement and solved the problem soft tissue reconstruction and visualization on the AR video that used in bowel and gallbladder surgeries., (© 2019 John Wiley & Sons, Ltd.)

Background: Innovation in computer-assisted surgery (CAS) aims to increase operative accuracy and improve safety by decreasing procedure-related complications. The application of reality technologies, to CAS has begun to revolutionize orthopedic training and practice., Methods: For this review, relevant published reports were found via searches of Medline (PubMed) data base using the following medical subject headings (MeSH) terms: "virtual reality" or "augmented reality" or "mixed reality" with "orthopedics" or "orthopedic surgery" and all relevant reports we utilized., Results: Trainees now have authentic and highly interactive operative simulations without the need for supervision. The practicing orthopedic surgeon is better able to pre-operatively plan and intra-operatively navigate without the use of fluoroscopy, gain access to three-dimensional reconstructions of patient imaging, and remotely interact with colleagues located outside the operating room., Conclusion: This review provides a current and comprehensive examination of the reality technologies and their applications in Orthopedic surgery., (© 2019 John Wiley & Sons, Ltd.)

The purpose of this study is to replace the manual process (selecting the landmarks on mesh and anchor points on the video) by Intensity-based Automatic Registration method to reach registration accuracy and low processing time. The proposed system consists of an Enhanced Intensity-based Automatic Registration (EIbAR) using Modified Zero Normalized Cross Correlation (MZNCC) algorithm. The proposed system was implemented on videos of breast cancer tumors. Results showed that the proposed algorithm-as compared to a reference-improved registration accuracy by an average of 2 mm. In addition, the proposed algorithm-as compared to a reference-reduced the number of pixel matching, thereby reducing processing time on the video by an average of 22 ms/frame. The proposed system can, thus, provide an acceptable accuracy and processing time during scene augmentation of videos, which provides a seamless use of augmented-reality for surgeons in visualizing cancer tumors., (© 2019 John Wiley & Sons, Ltd.)

Background: We aimed to assess the feasibility of a video-augmented fluoroscopy (VAF) technique using a camera-augmented mobile C-arm (CamC) for distal interlocking of intramedullary nails., Methods: Three surgeons performed distal interlocking on seven pairs of cadaveric bovine carpal bones using the VAF system and conventional fluoroscopy. We compared radiation exposure, procedure time, and drilling quality between the VAF system and conventional fluoroscopic guidance., Results: Distal interlocking using VAF significantly reduced the number of fluoroscopic images compared with conventional fluoroscopy (P < 0.05). No significant difference in overall procedure time (P = 0.96) or drilling quality (P = 0.12) was detected. VAF demonstrated improvement in radiation exposure when used by a less experienced surgeon (P < 0.05)., Conclusion: VAF is a feasible technique for distal interlocking. Overlaid visualization of the osseous anatomy in relation to the surgical field of view appears to improve surgeons' perception of relevant structures and their spatial orientation for the use of surgical instruments., (© 2019 John Wiley & Sons, Ltd.)

Background: Augmented reality (AR) allows the surgeon to represent holographic patient-specific anatomical information and surgical instruments in the physical world. To correctly superimpose virtual and physical objects, a hand-eye (HE) calibration method for mapping the virtual and physical spaces was proposed., Methods: Mathematical relationships between the virtual camera and the physical space were derived. Finally, the accuracy and robustness of the proposed HE calibration method were qualitatively and quantitatively evaluated., Results: The proposed calibration method allows us to determine an optimal invariant spatiotemporal mapping between the virtual camera and the physical space., Conclusion: Qualitatively and quantitatively reliable and accurate estimates for the physical-virtual mapping transformation were verified. Consequently, imaging data and surgical instruments holograms can be precisely represented in the physical space., (© 2018 John Wiley & Sons, Ltd.)

Background: Augmented reality (AR) surgery has not been successfully implemented in knee replacement surgery due to the negative effect of cutting errors. This research aims to decrease the cutting error to reduce the chronic pain after knee replacement., Methodology: The proposed system consists of a volume subtraction technique that considers the history of the area that has been cut and measures it against the target shape., Results: Results minimized the cutting error by about 1 mm. Therefore, it provides a significant video accuracy improvement in alignment to 0.40 to 0.55 mm from 0.55 to 0.64 and a decrease in processing time from 12 to 13 fs/s to 9 to10 fs/s., Conclusion: The proposed system is focused on overlaying only the remaining areas of surgery that need to be completed. Finally, this study solves the issues of navigation with AR when cutting bones in a scheduled direction and depth., (© 2018 John Wiley & Sons, Ltd.)

Background: Navigation support in interventional magnetic resonance imaging (MRI) is separated from the operating field, which makes it difficult to interpret positions and orientations and to coordinate the necessary hand movements., Methods: We developed a projector-based augmented reality system to enable visual navigation of tracked instruments on pre-planned paths and visualization of risk structures directly on the patient inside the MRI bore. To assess the accuracy of the system, a user study was carried out with clinicians in a needle navigation test scenario., Results: The targets were reached with an error of 1.7 ± 0.5 mm and the entry points with an error of 1.7 ± 0.8 mm., Conclusion: The accuracy results are similar to those reached by live image-guided interventions and related work and confirm that this projective augmented reality prototype for the interventional MRI can serve as a platform for current and future research in augmented reality visualization and dynamic registration., (© 2018 John Wiley & Sons, Ltd.)

Background: We previously developed an image-guided navigation system (IG-NS) using augmented reality technology for hepatobiliary and pancreatic (HBP) surgery. This system superimposed a 3D model onto a stereoscope-captured surgical field (i.e., the scope method). Unfortunately, this method requires an expensive stereoscope, surgeons have to shift their eyesight away from the surgical field, and the method has poor controllability. Therefore, an IG-NS using a tablet PC (i.e., the tablet method) was developed. The aim of the current study is to evaluate the efficiency of this novel method., Methods: We studied 9 patients, for whom a 3D model was created from computed tomography images. After registration was performed, the 3D model was superimposed onto the surgical field, which was captured by the tablet PC's camera., Results: The IG-NS could be applied with very little time lag. The visibility and controllability of the tablet method were superior to those of the scope method. It was especially useful in surgery for multiple metastatic liver carcinoma due to easy localization of the position of the carcinomas and vessels., Conclusions: We successfully developed the tablet method and tested it in a clinical setting. This system may contribute to surgical efficacy and improve the educational effects., (© 2018 John Wiley & Sons, Ltd.)

Background: Human-computer interaction (HCI) is an important feature of augmented reality (AR) technology. The naturalness is the inevitable trend of HCI. Gesture is the most natural and frequently used body auxiliary interaction mode in daily interactions except for language. However, there are often meaningless, subconscious gesture intervals between the two adjacent dynamic gestures. So, continuous dynamic gesture spotting is the premise and basis of dynamic gesture recognition, but there is no mature and unified algorithm to solve this problem., Aims: In order to realize the natural HCI based on gesture recognition entirely, a general AR application development platform is presented in this paper., Methods: According to the position and pose tracking data of the user's hand, the dynamic gesture spotting algorithm based on evidence theory is proposed. Firstly, Through analysis of the speed change of hand motion during the dynamic gestures, three knowledge rules are summed up. Then, accurate dynamic gesture spotting is realized with the application of evidence reasoning. Moreover, this algorithm first detects the starting point of gesture in the rising trend of hand motion speed, eliminates the delay between spotting and recognition, and thus ensures real-time performance. Finally, the algorithm is verified in several AR applications developed on the platform., Results: There are two main experimental results. First, there are six users participating in the dynamic gesture spotting experiment, and the gesture spotting accuracy can meet the demand. Second, The accuracy of recognition after spotting is higher than that of the simultaneous recognition and spotting., Conclusion: So, It can be concluded that the proposed continuous dynamic gesture spotting algorithm based on Dempster-Shafer theory can extract almost all the effective dynamic gestures in the HCI of our AR platform, and on this basis, it can effectively improve the accuracy of the subsequent dynamic gesture recognition., (© 2018 John Wiley & Sons, Ltd.)

Introduction: Since the introduction of wearable head-up displays, there has been much interest in the surgical community adapting this technology into routine surgical practice., Methods: We used the keywords augmented reality OR wearable device OR head-up display AND surgery using PubMed, EBSCO, IEEE and SCOPUS databases. After exclusions, 74 published articles that evaluated the utility of wearable head-up displays in surgical settings were included in our review., Results: Across all studies, the most common use of head-up displays was in cases of live streaming from surgical microscopes, navigation, monitoring of vital signs, and display of preoperative images. The most commonly used head-up display was Google Glass. Head-up displays enhanced surgeons' operating experience; common disadvantages include limited battery life, display size and discomfort., Conclusions: Due to ergonomic issues with dual-screen devices, augmented reality devices with the capacity to overlay images onto the surgical field will be key features of next-generation surgical head-up displays., (Copyright © 2018 John Wiley & Sons, Ltd.)

Background: Augmented reality-based surgeries have not been successfully implemented in oral and maxillofacial areas due to limitations in geometric accuracy and image registration. This paper aims to improve the accuracy and depth perception of the augmented video., Methodology: The proposed system consists of a rotational matrix and translation vector algorithm to reduce the geometric error and improve the depth perception by including 2 stereo cameras and a translucent mirror in the operating room., Results: The results on the mandible/maxilla area show that the new algorithm improves the video accuracy by 0.30-0.40 mm (in terms of overlay error) and the processing rate to 10-13 frames/s compared to 7-10 frames/s in existing systems. The depth perception increased by 90-100 mm., Conclusion: The proposed system concentrates on reducing the geometric error. Thus, this study provides an acceptable range of accuracy with a shorter operating time, which provides surgeons with a smooth surgical flow., (Copyright © 2018 John Wiley & Sons, Ltd.)

Background: While endoscopic skull base surgery (ESBS) has emerged as an alternative surgical option, the limited field of view of the endoscope may lead to the surgeon's fatigue and discomfort., Methods: The developed navigation system includes extended augmented reality (AR), which can provide an extended viewport to a conventional endoscopic view by overlaying 3D anatomical models generated from preoperative medical images onto endoscope images. To enhance the accuracy of the developed system, we adopted state-of-the-art endoscopic calibration and tracking techniques based on an optical tracking system., Results: The mean spatial errors of AR was ~1 mm, which falls in the acceptable range of accuracy for ESBS. For the simulated surgical tasks with the developed system, the number and duration of error events were decreased., Conclusions: The results show that the human subject can perform the task more precisely and safely with the developed AR-based navigation system than with the conventional endoscopic system., (Copyright © 2017 John Wiley & Sons, Ltd.)

Background: Wearable technology is growing in popularity as a result of its ability to interface with normal human movement and function., Methods: Using proprietary hardware and software, neuronavigation images were captured and transferred wirelessly via a password-encrypted network to the head-up display. The operating surgeon wore a loupe-mounted wearable head-up display during image-guided parieto-occipital ventriculoperitoneal shunt placement in two patients., Results: The shunt placement was completed successfully without complications. The tip of the catheter ended well within the ventricles away from the ventricular wall. The wearable device allowed for continuous monitoring of neuronavigation images in the right upper corner of the surgeon's visual field without the need for the surgeon to turn his head to view the monitors., Conclusions: The adaptable nature of this proposed system permits the display of video data to the operating surgeon without diverting attention away from the operative task. This technology has the potential to enhance image-guided procedures., (Copyright © 2017 John Wiley & Sons, Ltd.)

Background: Oral and maxillofacial surgery has not been benefitting from image guidance techniques owing to the limitations in image registration., Methods: A real-time markerless image registration method is proposed by integrating a shape matching method into a 2D tracking framework. The image registration is performed by matching the patient's teeth model with intraoperative video to obtain its pose. The resulting pose is used to overlay relevant models from the same CT space on the camera video for augmented reality., Results: The proposed system was evaluated on mandible/maxilla phantoms, a volunteer and clinical data. Experimental results show that the target overlay error is about 1 mm, and the frame rate of registration update yields 3-5 frames per second with a 4 K camera., Conclusions: The significance of this work lies in its simplicity in clinical setting and the seamless integration into the current medical procedure with satisfactory response time and overlay accuracy. Copyright © 2016 John Wiley & Sons, Ltd., (Copyright © 2016 John Wiley & Sons, Ltd.)

Background: Surgical simulation based on augmented reality (AR), mixing the benefits of physical and virtual simulation, represents a step forward in surgical training. However, available systems are unable to update the virtual anatomy following deformations impressed on actual anatomy., Methods: A proof-of-concept solution is described providing AR visualization of hidden deformable tubular structures using nitinol tubes sensorized with electromagnetic sensors. This system was tested in vitro on a setup comprised of sensorized cystic, left and right hepatic, and proper hepatic arteries. In the trial session, the surgeon deformed the tubular structures with surgical forceps in 10 positions., Results: The mean, standard deviation, and maximum misalignment between virtual and real arteries were 0.35, 0.22, and 0.99 mm, respectively., Conclusion: The alignment accuracy obtained demonstrates the feasibility of the approach, which can be adopted in advanced AR simulations, in particular as an aid to the identification and isolation of tubular structures. Copyright © 2015 John Wiley & Sons, Ltd., (Copyright © 2015 John Wiley & Sons, Ltd.)

Background: Surgical image guidance systems to date have tended to rely on reconstructions of preoperative datasets. This paper assesses the accuracy of these reconstructions to establish whether they are appropriate for use in image guidance platforms., Methods: Nine raters (two experts in image interpretation and preparation, three in image interpretation, and four in neither interpretation nor preparation) were asked to perform a segmentation of ten renal tumours (four cystic and six solid tumours). These segmentations were compared with a gold standard consensus segmentation generated using a previously validated algorithm., Results: Average sensitivity and positive predictive value (PPV) were 0.902 and 0.891, respectively. When assessing for variability between raters, significant differences were seen in the PPV, sensitivity and incursions and excursions from consensus tumour boundary., Conclusions: This paper has demonstrated that the interpretation required for the segmentation of preoperative imaging of renal tumours introduces significant inconsistency and inaccuracy. Copyright © 2015 John Wiley & Sons, Ltd., (Copyright © 2015 John Wiley & Sons, Ltd.)

Background: Depth perception is a major issue in augmented reality (AR)-based surgical navigation. We propose an AR and virtual reality (VR) switchable visualization system with distance information, and evaluate its performance in a surgical navigation set-up., Methods: To improve depth perception, seamless switching from AR to VR was implemented. In addition, the minimum distance between the tip of the surgical tool and the nearest organ was provided in real time. To evaluate the proposed techniques, five physicians and 20 non-medical volunteers participated in experiments., Results: Targeting error, time taken, and numbers of collisions were measured in simulation experiments. There was a statistically significant difference between a simple AR technique and the proposed technique., Conclusions: We confirmed that depth perception in AR could be improved by the proposed seamless switching between AR and VR, and providing an indication of the minimum distance also facilitated the surgical tasks., (Copyright © 2015 John Wiley & Sons, Ltd.)

Background: With the development of novel augmented reality operating platforms the way surgeons utilise imaging as a real-time adjunct to surgical technique is changing., Methods: A questionnaire was distributed via the European Robotic Urological Society mailing list. The questionnaire had three themes: surgeon demographics, current use of imaging and potential uses of an augmented reality operating environment in robotic urological surgery., Results: 117 of the 239 respondents (48.9%) were independently practising robotic surgeons. 74% of surgeons reported having imaging available in theatre for prostatectomy 97% for robotic partial nephrectomy and 95% cystectomy. 87% felt there was a role for augmented reality as a navigation tool in robotic surgery., Conclusions: This survey has revealed the contemporary robotic surgeon to be comfortable in the use of imaging for intraoperative planning it also suggests that there is a desire for augmented reality platforms within the urological community., (Copyright © 2014 John Wiley & Sons, Ltd.)

Background: One of the main sources of error in commercial surgical navigation systems is the tracking of surgical tools. Mainstream systems typically use optical or electromagnetic tracking technologies, which exhibit accuracies of the order of 1 mm. The objective of this study was to introduce a lightweight high-precision passive coordinate measurement arm into an augmented reality-based surgical navigation system to track a rigid endoscope., Methods: A series of dry laboratory experiments were run to compare the tracking performance of an optical tracking device, a passive coordinate measurement arm and a hybrid set-up., Results: The optical device displayed overlay errors in the range 1.5-3 mm. For the precision measurement arm, 96% of overlay errors were < 1 mm. The hybrid set-up exhibited overlay errors in the range 0.8-1.5 mm., Conclusions: The reported experiments showed that a high-precision articulated measurement arm could be used as a motion-tracking device for surgical instruments in augmented-reality surgical navigation., (Copyright © 2013 John Wiley & Sons, Ltd.)

Background: Despite the popular use of virtual and physical reality simulators in laparoscopic training, the educational potential of augmented reality (AR) has not received much attention. A major challenge is the robust tracking and three-dimensional (3D) pose estimation of the endoscopic instrument, which are essential for achieving interaction with the virtual world and for realistic rendering when the virtual scene is occluded by the instrument. In this paper we propose a method that addresses these issues, based solely on visual information obtained from the endoscopic camera., Methods: Two different tracking algorithms are combined for estimating the 3D pose of the surgical instrument with respect to the camera. The first tracker creates an adaptive model of a colour strip attached to the distal part of the tool (close to the tip). The second algorithm tracks the endoscopic shaft, using a combined Hough-Kalman approach. The 3D pose is estimated with perspective geometry, using appropriate measurements extracted by the two trackers., Results: The method has been validated on several complex image sequences for its tracking efficiency, pose estimation accuracy and applicability in AR-based training. Using a standard endoscopic camera, the absolute average error of the tip position was 2.5 mm for working distances commonly found in laparoscopic training. The average error of the instrument's angle with respect to the camera plane was approximately 2°. The results are also supplemented by video segments of laparoscopic training tasks performed in a physical and an AR environment., Conclusions: The experiments yielded promising results regarding the potential of applying AR technologies for laparoscopic skills training, based on a computer vision framework. The issue of occlusion handling was adequately addressed. The estimated trajectory of the instruments may also be used for surgical gesture interpretation and assessment., (Copyright © 2013 John Wiley & Sons, Ltd.)

Background: Development of new imaging technologies and advances in computing power have enabled the physicians to perform medical interventions on the basis of high-quality 3D and/or 4D visualization of the patient's organs. Preoperative imaging has been used for planning the surgery, whereas intraoperative imaging has been widely employed to provide visual feedback to a clinician when he or she is performing the procedure. In the past decade, such systems demonstrated great potential in image-guided minimally invasive procedures on different organs, such as brain, heart, liver and kidneys. This article focuses on image-guided interventions and surgery in renal and hepatic surgeries., Methods: A comprehensive search of existing electronic databases was completed for the period of 2000-2011. Each contribution was assessed by the authors for relevance and inclusion. The contributions were categorized on the basis of the type of operation/intervention, imaging modality and specific techniques such as image fusion and augmented reality, and organ motion tracking., Results: As a result, detailed classification and comparative study of various contributions in image-guided renal and hepatic interventions are provided. In addition, the potential future directions have been sketched., Conclusion: With a detailed review of the literature, potential future trends in development of image-guided abdominal interventions are identified, namely, growing use of image fusion and augmented reality, computer-assisted and/or robot-assisted interventions, development of more accurate registration and navigation techniques, and growing applications of intraoperative magnetic resonance imaging., (Copyright © 2012 John Wiley & Sons, Ltd.)

Background: Computerized management of medical information and 3D imaging has become the norm in everyday medical practice. Surgeons exploit these emerging technologies and bring information previously confined to the radiology rooms into the operating theatre. The paper reports the authors' experience with integrated stereoscopic 3D-rendered images in the da Vinci surgeon console., Methods: Volume-rendered images were obtained from a standard computed tomography dataset using the OsiriX DICOM workstation. A custom OsiriX plugin was created that permitted the 3D-rendered images to be displayed in the da Vinci surgeon console and to appear stereoscopic. These rendered images were displayed in the robotic console using the TilePro multi-input display. The upper part of the screen shows the real endoscopic surgical field and the bottom shows the stereoscopic 3D-rendered images. These are controlled by a 3D joystick installed on the console, and are updated in real time., Results: Five patients underwent a robotic augmented reality-enhanced procedure. The surgeon was able to switch between the classical endoscopic view and a combined virtual view during the procedure. Subjectively, the addition of the rendered images was considered to be an undeniable help during the dissection phase., Conclusion: With the rapid evolution of robotics, computer-aided surgery is receiving increasing interest. This paper details the authors' experience with 3D-rendered images projected inside the surgical console. The use of this intra-operative mixed reality technology is considered very useful by the surgeon. It has been shown that the usefulness of this technique is a step toward computer-aided surgery that will progress very quickly over the next few years., (Copyright © 2012 John Wiley & Sons, Ltd.)