Search

A proposed solution to address increasing scarcity of operating room (OR) nurses is the deployment of robotic scrub nurses (RSNs). One of the primary responsibilities of these robots is the handover of surgical instruments to the surgeon. Despite this tasks’ apparent simplicity, it requires great precision and timing, while seamless execution is essential for the surgical workflow. This paper outlines the unique challenges of the instrument handover process between a RSN and a surgeon taking place during laparoscopic surgery. After customizing existing solutions from different domains to meet the specific needs of laparoscopic surgery, we introduced these solutions to medical professionals in a simulated OR scenario, and performed a utility analysis to identify the most suitable solutions.

The increase in staff shortage, particularly in urgently needed care professions, is becoming an immense problem in the German healthcare system. This critical issue is further intensified by the fact that the healthcare system is still a strongly human-dependent and often insufficiently interconnected environment. Automation, medical robotics and the concept of model-based medicine can play a key role not only in addressing these issues and supporting clinical staff, but also in enabling new applications to improve patient care. While many existing approaches aim to tackle these challenges, they are generally based on current network architectures and standards and are thus constrained by the limitations of these networks. This article highlights the need for a new network approach such as 6G, which offers flexible, dynamic structures, new communication classes and many other new features in order to leverage the true potential of these new technologies.

Swallowing problems (dysphagia) is associated with significant morbidity and mortality therefore diagnosis and treatment of dysphagia is important. Diagnostic tests include screening procedures, clinical swallowing examinations, and instrumental examination procedures. A non-invasive diagnostic option is auscultation of the swallowing act. However, there are different statements about the reliability and validity of the manual execution of this test. We developed a mobile hardware system to record cervical sounds using two microphones on the neck to acquire audio a data set. To generate ground truth data, fiberendoscopic swallow examinations were performed simultaneously to identify dysphagia. In order to diagnostically assess the swallowing sounds a spectrogram based classification pipeline was developed. In a first step this enabled us to identify different swallowing patterns in healthy individuals. With an accuracy of ~95%, we were able to reliably detect swallows within audio recordings, while the classification of types of swallow (dry, water, solid food) indicate the need for further improvements within the project ahead. In the future, we anticipate AI based analysis of auscultated swallowing sounds to detect swallowing disorders and aspirations.

To help with the critical nurse staffing shortages in hospitals worldwide, robotic assistants are designed to handle frequently required tasks in the digital operating room (DOR), such as the guidance of the laparoscopic camera. To enable fluent collaboration between robots and clinicians, an intuitive and efficient communication interface is needed to allow for interaction using natural language. However, the demanding requirements of the surgical domain make it challenging to develop suitable solutions. A variety of different vocabulary or phrases may be used for expressing the same command. At the same time, surgical workflows may be highly dynamic - especially in emergency situations - and thus the system must be able to grasp the user’s intent both quickly and with high accuracy. This is especially true as only some clinicians may be authorized to request certain tasks, depending on their rank or field of expertise. To solve these challenges, our proposed communication system uses the fine-tuned deep learning model to recognize the speaker information, and the robot assistant takes action only when it detects the commands from the responsible clinician. Also, our proposed conversational functions enable the finetuned large language models to understand the current natural language command given previous command history. In this work, we present a communication system to recognize the speaking person and understand the intent of conversational commands quickly and accurately.

Introduction: Application-related (hands-on) training and instruction is an essential component in the advanced education of physicians in residency. To teach and train required endoscopic examinations and interventions in a standardized manner, courses are offered by various societies. Within these courses the assistant endoscopists learn and train how to use the technical equipment and how to perform the examinations and related interventions. Similarly, for graduate biomedical engineering students with a job- or research-related interest in the field of endoand laparoscopy, an early involvement and practical introduction to the instrumentation and techniques - if possible already during their studies - can generate a fundamental understanding of the technical requirements as well needs of the physicians. Objective: As various such (hands-on) training-courses for flexible and rigid endoscopy specifically addressing young physicians, surgeons as well as engineers are offered by numerous institutes, this contribution tries do provide a structured overview on/over these possibilities. Material: Known cross-institutional courses in the field of endo- and laparoscopy currently offered in Germany in cooperation of physicians and engineers are listed and briefly described. Results: A total of n = 4 crossinstitutional courses for the introduction and training in flexible and rigid endoscopy have been identified. Discussion: The cross-institutional courses for physicians and technicians in the field of endo- and laparoscopy presented serve to provide a better understanding of the subject matter and the field of either discipline. Questions that arise in the field of endoscopy from physicians can be addressed at an early stage and, if necessary, worked on with little effort. Ideas for new instruments and even new interventions can arise and be pursued through an intensive exchange between technicians and physicians. Collaborations are established on a local basis which will foster national activities in this field and which allow to develop competencies relevant for the industrial sector of medical engineering.

A growing body of evidence supports Virtual Reality (VR) as an effective and safe strategy for management of pain and stress associated with medical procedures in both adults and children. We therefore initiated a feasibility study to investigate the effect of VR on pain, stress, and anxiety during elective surgery, e.g. implantation of a central-venous port catheter, hypothesizing that VR can reduce intraoperative pain, stress and anxiety of the patient. In this manuscript, the preliminary results of the first 20 (out of 6o planned) patients are presented. Baseline pain characteristics did not differ between the two study groups (VR group (n=10) and standard (no VR device) group (n=10)). System usability (“easy to use”, “easy to learn” and “safe”) was rated “good - very good” by the study participants. Selfassessment of anxiety components (Y-6 item questionnaire) revealed a calming (3.3 ± 0.5 vs. 2.4 ± 0.5, P= 0.009) and relaxing (2.7 ± 1.2 vs. 1.8 ± 0.4; P=0.09) effect of the VR device. Evaluation of pain level (Short form McGill questionnaire) during the procedure revealed a lower pain intensity (VAS) level (17.5 ± 12.1 vs. 19.5 ± 10.6; P= 0.834) and present pain intensity (PPI) score (0.9 ± 0.6 vs. 1.0 ± 0.5; P= 0.841) in the VR group Preliminary data of our feasibility study indicates a positive effect of VR towards reduction of pain and stress in patients undergoing minor surgery in local anaesthesia. However, further data is needed to substantiate these results.

Purpose The demand for telemedicine is continuously increasing. However, many medical examination techniques such as auscultation are based on manual execution and subjective assessment by each individual doctor. For the increasing trend of telemedicine, we have developed an electric percussion device that examines in a standardized way. Methods This study evaluated the new approach of using motorized percussion against the gold standard of manual percussion. An examination was carried out on a healthy volunteer using both techniques. Thereby, occurring sounds from both pulmonary and non-pulmonary areas were recorded as the primary goal of percussion is to assess the size of the lung. Recordings were cut into individual samples and randomized within each examination variant. Results When discriminating between ventral and dorsal, and manual and motorized percussion, the outcome of the 28-participant anonymous survey showed that the noisesuppressed ventral motorized percussion with 93.2% (10.2) correctly identified percussion points surpassed all other conducted percussion techniques. Conclusion Making a distinction between pulmonary volume and exterior areas is very reliable with the motorized percussion. In order to be able to describe the clinical picture precisely in the future, further evaluations are necessary.

Laparoscopic cholecystectomy (LCHE) is a widely employed model for surgical instrument and phase recognition in the field of machine learning (ML), with the latter being assigned to identify critical events and to avoid complications. Although ML algorithms have been proven to be effective for this instance and in selected patients, it is questionable whether patients receiving LCHE in daily clinical routine would actually benefit from adverse event prediction by ML applications. We believe, that the statistical problem of low prevalence (PREV) of potential adverse events in an unselected population and consequential low diagnostic yield was not considered adequately in recent research. Therefore, we performed a query to the G-DRG (German Diagnosis Related Groups) database of the German Federal Statistical Office with the aim to calculate prevalence of surgical and postoperative adverse events coming along with LCHE. The results enable an estimation of positive (PPV) and negative (NPV) predictive values hypothetically achievable by ML applications aiming to predict an adverse surgical course.

Navigation systems play an increasingly important role in minimally invasive surgery (MIS) by mitigating the problems rising from the decoupling of hand-eye movement of the surgeon. Many of these systems suffer from a high dependency on external optical tracking systems that require a constant line-of-sight to the optical markers being tracked. Simultaneous localization and mapping (SLAM) algorithms allow tracking the endoscope in cases where optical tracking fails due to the cluttered environment in the operating room. To ensure a correct camera pose estimate and to correct for drift, a recognition of previously visited locations (loop closures) is essential. We propose a method for location recognition in a minimally invasive scenario that only requires a stereo endoscope and an inertial measurement unit (IMU).We use a hierarchical bag-of-visual-words (BoW) algorithm that saves compact image representations and enables querying for matching images. A two-staged consistency check using a random sample consensus (RANSAC) and the data measured by the IMU ensure a high matching precision.

This work presents a sterile concept for 3D cameras within sterile surgical environments. In the digital operating room (OR), such cameras can serve as a valuable data source for cognitive workflow assistance systems, e.g decision or mechatronic support systems. One recent example are robotic assistants for instrument handling, such as the robotic scrub nurse currently developed in the framework of the SASHA-OR research project1. In this context, we detect laparoscopic instruments and the surgical environment with a 3D camera, whereby hygienic requirements need to be met. Using a Zivid Two sensor, we generated point clouds of the laparoscopic instruments located in an instrument holder and a drop zone. We compared the effect of using different pane types and thicknesses for the sterile camera enclosure and compared the performance with and without protective pane in terms of the point cloud accuracy. When analyzing multiple pane types, polymethyl methacrylate with 0.5 mm thickness (PMMA 0.5) provided the best results. At a scan distance of 560 mm to the surface center, which is required for the complete acquisition of a laparoscopic instrument, PMMA 0.5 achieved the smallest Chamfer distance (CD) values for both the scans with the laparoscopic instruments in the instrument holder (0.23 ± 1.52 mm) and in the drop zone (0.12 ± 0.25 mm).

Due to the coronavirus disease 2019 (Covid-19) pandemic, all scientific conferences in the year 2020 had to be adapted in their form of presence to accommodate for safety regulations, postponed, or canceled entirely. As organizers of the annual Computer Assisted Radiology & Surgery International Conference & Exhibition (CARS)-Conference 2020, we decided to hold a “hybrid” conference, i.e., a virtual conference with partial presence to mitigate the drawbacks of a purely virtual conference. It is the purpose of this paper to describe the results and experience gained by our first hybrid conference.

In context of the Corona pandemic, telemedicine acquired a new significance. Whereas previously the aim was to override given barriers, now, in the case of a pandemic, the main idea is to create an intentional distance between patients and healthcare professionals in order to avoid cross-infection. To meet the needs of a fully diagnostic examination, a robotic based system was designed. However, collaborative robotic systems bear new risks, that have to be dealt with. To prepare future physicians for telediagnostics, we developed a training curriculum for the telemedical examinations. It is based upon multiple stages including a skill trainer, healthy volunteers, supervised examinations of real patients and an exam. In a first proof of concept, we demonstrated the existence of a learning curve and significant better performance after the passed curriculum compared to an untrained collective.

The goal of our research work is the development of a novel endoscopic anastomosis device for the colon. One of the main challenges in this context is the application of forces at the endoscope tip to rejoin the two bowel endings. Thus, we focus on a magnetic two-part compression implant approach. The implant halves are detached from the applicator units by means of electromagnets. In this contribution we present the results of our experiments to determine the implant design with special focus on tissue compression forces and the resultant electromagnet dimensioning to estimate size requirements of the application/detachment system. To achieve the targeted compression forces derived from literature, we used cubic N52 magnetized neodymium magnets1 with a side length of 5 mm and mild steel screws. For these magnets, we evaluated a required electromagnetic repulsion force of 4.1 N. For the electromagnetic detachment system this led to the need for 166 windings for the coils on oral side, and 146 windings for the coils at the aboral side. Based on these requirements, a colonoscope diameter (~14 mm) increase of 10.6 mm on the oral side and of 12 mm on the aboral side due to the application device must be assumed. Nevertheless, this diameter still remains within the size range of other colonoscopic tools, such as e.g., circular staplers.

A core principle of modern health care is the compliance of hygienic and aseptic techniques in areas that are sensitive to contamination through bacteria, dust, aerosols, and fallout, primarily in operating theatres or around patients with contagious diseases. Keeping track of potentially contaminated surfaces in an environment is a major concern, especially when protecting from COVID-19. This work proposes a novel concept in using 3D sensing technology to track human movement within an indoor area and identifying high-risk contaminated surfaces in real-time. It combines recent Augmented Reality display technology, which allows keeping track of decontaminated surfaces during the cleaning process using an interactive visualization method. The proposed concept of Clean- AR is implemented in a clinical environment used for observation in COVID-19 scenarios. We discuss key challenges and outline further research direction in effectively reducing the risk of contamination using the proposed concept.

The basic concept of Diagnostic Laparoscopy (DL) is old but not old-fashioned. – Before the rise of tomography and ultrasound there was just the direct look into the abdomen and onto the affected organ available. As open surgery comes along with trauma, blood loss and infection, every effort have been made to improve the access strategies. Finally, due to innovation in optics, video technology and computer science, the look into the abdomen through a “keyhole” is a standard procedure today. – In this review we give an overview of history, implications and cost-effectiveness of DL, attempting an extrapolation of its future role.

Tracking of surgical instruments is an essential step towards the modernization of the surgical workflow by a comprehensive surgical landscape guidance system (COMPASS). Real-time tracking of a laparoscopic camera used in minimally-invasive surgery is required for applications in surgical workflow documentation, machine learning, image-localization, and intra-operative visualization. In our approach, an inertial measurement unit (IMU) assists the tool tracking in situations when no line-of-sight is available for infrared (IR) based tracking of the laparoscopic camera. The novelty of this approach lies in the localization method adjusted for the laparoscopic visceral surgery, particularly when the line-of-sight is lost. It is based on IMU tracking and the positioning of the trocar entry point. The trocar entry point is the remote center of motion (RCM), reducing degrees of freedom. We developed a method to tackle localization and a real-time tool for position and orientation estimation. The main error sources are given and evaluated in a test scenario. It reveals that for small changes in penetration length (e.g., pivoting), the IMU’s accuracy determines the error.

A common treatment for malignant hepatic tumors is radiofrequency thermal ablation (RFA); however this procedure is often exhausting and error-prone, when performed only with hand held devices. In this work, we present a new concept for a computer- and mechatronic-arm-assisted treatment of hepatic tumor with RFA. Challenging factors of state-of-the-art RFA procedures are tackled by utilizing a prototype, which combines state-of-the-art ultrasound image guidance with a mechatronic support arm system. Physicians can image and examine the human abdomen using an ultrasound capturing device which is guided by the mechatronic support arm. The mechatronic arm allows to track the ultrasound transducer via joint position encoders, so that the recorded images’ exact positions are acquired and the patient is automatically registered during imaging. Consequently, physicians can use the acquired ultrasound images to navigate in the imaged abdomen area. The established prototype enables preplanning and semi-automatic alignment of trajectories with sufficient accuracy. In addition, the arm system supports the usually lengthy, exhausting and error-prone holding of the needle during the ablation process. Based on the results of accuracy examination and expert assessments by surgeons, the established concept could be proven to be advantageous for the considered use case.

Different components of the newly defined field of surgical data science have been under research at our groups for more than a decade now. In this paper, we describe our sensor-driven approaches to workflow recognition without the need for explicit models, and our current aim is to apply this knowledge to enable context-aware surgical assistance systems, such as a unified surgical display and robotic assistance systems. The methods we evaluated over time include dynamic time warping, hidden Markov models, random forests, and recently deep neural networks, specifically convolutional neural networks.

Telemedicine has become a valuable asset in emergency responses for assisting paramedics in decision making and first contact treatment. Paramedics in unfamiliar environments or time-critical situations often encounter complications for which they require external advice. Modern ambulance vehicles are equipped with microphones, cameras, and vital sensors, which allow experts to remotely join the local team. However, the visual channels are rarely used since the statically installed cameras only allow broad views at the patient. They neither allow a close-up view nor a dynamic viewpoint controlled by the remote expert. In this paper, we present EyeRobot, a concept which enables dynamic viewpoints for telepresence using the intuitive control of the user’s head motion. In particular, EyeRobot utilizes the 6 degrees of freedom pose estimation capabilities of modern head-mounted displays and applies them in real-time to the pose of a robot arm. A stereo-camera, installed on the end-effector of the robot arm, serves as the eyes of the remote expert at the local site. We put forward an implementation of EyeRobot and present the results of our pilot study which indicates its intuitive control.