Search

Background: The competency and skill development of aspiring surgeons heavily depend on surgical training. Traditionally, open surgery training has relied on cadavers, enabling trainees to actively participate in tutor-guided and evaluated interventions, which are crucial aspects of their education. Nonetheless, this approach places additional burdens and risks on patients, along with considerable time and financial constraints. Objective: Our goal is to develop an all-encompassing Learning Platform designed to offer standardized training modules for laparoscopy, arthroscopy, and open surgery. A pivotal element of this endeavor involves integrating a quantitative and systematic approach to assess surgical skills objectively, utilizing predefined criteria. This task would be challenging without the assistance of digital technologies. The introduction of this groundbreaking training method represents a remarkable leap forward in the field. The implementation of the learning platform encompasses three distinct sub-projects., AR Based Training: Our main objective is to create an AR-based simulator for open orthopedic surgery training, specifically focusing on Total Hip Arthroplasty. This simulator represents a significant advancement over conventional cadaver-based training as it integrates adaptive AR guidance and AI techniques. By harnessing the power of intuitive AR instructions and leveraging AI-driven analysis of human activity and behavior, our aim is to optimize the usability of surgical training simulators. Ultimately, this will reduce the strain on patients and minimize the requirement for constant instructor and technical support during surgical education.

Established surgical navigation systems for pedicle screw placement have been proven to be accurate, but still reveal limitations in registration or surgical guidance. Registration of preoperative data to the intraoperative anatomy remains a time-consuming, error-prone task that includes exposure to harmful radiation. Surgical guidance through conventional displays has well-known drawbacks, as information cannot be presented in-situ and from the surgeon's perspective. Consequently, radiation-free and more automatic registration methods with subsequent surgeon-centric navigation feedback are desirable. In this work, we present a marker-less approach that automatically solves the registration problem for lumbar spinal fusion surgery in a radiation-free manner. A deep neural network was trained to segment the lumbar spine and simultaneously predict its orientation, yielding an initial pose for preoperative models, which then is refined for each vertebra individually and updated in real-time with GPU acceleration while handling surgeon occlusions. An intuitive surgical guidance is provided thanks to the integration into an augmented reality based navigation system. The registration method was verified on a public dataset with a median of 100% successful registrations, a median target registration error of 2.7 mm, a median screw trajectory error of 1.6°and a median screw entry point error of 2.3 mm. Additionally, the whole pipeline was validated in an ex-vivo surgery, yielding a 100% screw accuracy and a median target registration error of 1.0 mm. Our results meet clinical demands and emphasize the potential of RGB-D data for fully automatic registration approaches in combination with augmented reality guidance.

INTRODUCTION The periacetabular osteotomy is a technically demanding procedure with the goal to improve the osseous containment of the femoral head. The options for controlled execution of the osteotomies and verification of the acetabular reorientation are limited. With the assistance of augmented reality, new possibilities are emerging to guide this intervention. However, the scientific knowledge regarding AR navigation for PAO is sparse. METHODS In this cadaveric study, we wanted to find out, if the execution of this complex procedure is feasible with AR guidance, quantify the accuracy of the execution of the three-dimensional plan, and find out what has to be done to proceed to real surgery. Therefore, an AR guidance for the PAO was developed and applied on 14 human hip cadavers. The guidance included performance of the four osteotomies and reorientation of the acetabular fragment. The osteotomy starting points, the orientation of the osteotomy planes, as well as the reorientation of the acetabular fragment were compared to the 3D planning. RESULTS The mean 3D distance between planned and performed starting points was between 9 and 17 mm. The mean angle between planned and performed osteotomies was between 6° and 7°. The mean reorientation error between the planned and performed rotation of the acetabular fragment was between 2° and 11°. CONCLUSION The planned correction can be achieved with promising accuracy and without serious errors. Further steps for a translation from the cadaver to the patient have been identified and must be addressed in future work.

Surgery digitalization is the process of creating a virtual replica of real-world surgery, also referred to as a surgical digital twin (SDT). It has significant applications in various fields such as education and training, surgical planning, and automation of surgical tasks. In addition, SDTs are an ideal foundation for machine learning methods, enabling the automatic generation of training data. In this paper, we present a proof of concept (PoC) for surgery digitalization that is applied to an ex-vivo spinal surgery. The proposed digitalization focuses on the acquisition and modelling of the geometry and appearance of the entire surgical scene. We employ five RGB-D cameras for dynamic 3D reconstruction of the surgeon, a high-end camera for 3D reconstruction of the anatomy, an infrared stereo camera for surgical instrument tracking, and a laser scanner for 3D reconstruction of the operating room and data fusion. We justify the proposed methodology, discuss the challenges faced and further extensions of our prototype. While our PoC partially relies on manual data curation, its high quality and great potential motivate the development of automated methods for the creation of SDTs., Comment: Accepted for the DCA in MI Workshop @ CVPR 2024. Project page: https://jonashein.github.io/surgerydigitization

This study tackles key obstacles in adopting surgical navigation in orthopedic surgeries, including time, cost, radiation, and workflow integration challenges. Recently, our work X23D showed an approach for generating 3D anatomical models of the spine from only a few intraoperative fluoroscopic images. This negates the need for conventional registration-based surgical navigation by creating a direct intraoperative 3D reconstruction of the anatomy. Despite these strides, the practical application of X23D has been limited by a domain gap between synthetic training data and real intraoperative images. In response, we devised a novel data collection protocol for a paired dataset consisting of synthetic and real fluoroscopic images from the same perspectives. Utilizing this dataset, we refined our deep learning model via transfer learning, effectively bridging the domain gap between synthetic and real X-ray data. A novel style transfer mechanism also allows us to convert real X-rays to mirror the synthetic domain, enabling our in-silico-trained X23D model to achieve high accuracy in real-world settings. Our results demonstrated that the refined model can rapidly generate accurate 3D reconstructions of the entire lumbar spine from as few as three intraoperative fluoroscopic shots. It achieved an 84% F1 score, matching the accuracy of our previous synthetic data-based research. Additionally, with a computational time of only 81.1 ms, our approach provides real-time capabilities essential for surgery integration. Through examining ideal imaging setups and view angle dependencies, we've further confirmed our system's practicality and dependability in clinical settings. Our research marks a significant step forward in intraoperative 3D reconstruction, offering enhancements to surgical planning, navigation, and robotics.