Search

In endoscopic submucosal dissection (ESD), the gastrointestinal (GI) tract warrants the surgical instruments to navigate through a long, narrow and tortuous endoscope. This poses a great challenge in developing ESD instruments with small dimensions, flexibility, and high distal dexterity. In this work, we propose the first Transendoscopic Flexible Parallel Continuum Robotic mechanism to develop a miniature dexterous flexible-stiff-balanced Wrist (FPCW). Besides, it can steer multifunctional instruments of diameters 2.5 mm to 3.5 mm, including the electrosurgical knife, injection needle, and forceps. Our FPCW instruments are adaptable to commercially available dual-channel endoscopes (diameter: <12 mm, channel width: 2.8 mm and around 3.8 mm). Furthermore, we develop a surgical telerobotic system, called DREAMS (Dual-arm Robotic Endoscopic Assistant for Minimally Invasive Surgery), by using our smallest FPCW instruments for bimanual ESD procedures. First, we conduct a series of experiments to determine the FPCW's design and kinematics parameters and to verify the mechanical properties of the FPCW instruments' prototypes, including workspace, stiffness, strength, and teleoperation accuracy. Second, we validate the functionality of the FPCW instruments through ex-vivo tests by performing ESD steps on porcine stomachs. Finally, we perform an invivo test on a live porcine model and showcase that our developed DREAMS can be teleoperated intuitively to perform bimanual ESD efficiently with an average dissection speed of 108.95 mm2/min at the greater curvature in gastric body, which demonstrates that our DREAMS has satisfactory maneuverability as well as accuracy and is more competitive than counterpart robotic systems. [ABSTRACT FROM AUTHOR]

Robot-assisted airway intubation application needs high accuracy in locating targets and organs. Two vital landmarks, nostrils and glottis, can be detected during the intubation to accommodate the stages of nasal intubation. Automated landmark detection can provide accurate localization and quantitative evaluation. The Detection Transformer (DeTR) leads object detectors to a new paradigm with long-range dependence. However, current DeTR requires long iterations to converge, and does not perform well in detecting small objects. This paper proposes a transformer-based landmark detection solution with deformable DeTR and the semantic-aligned-matching module for detecting landmarks in robot-assisted intubation. The semantics aligner can effectively align the semantics of object queries and image features in the same embedding space using the most discriminative features. To evaluate the performance of our solution, we utilize a publicly accessible glottis dataset and automatically annotate a nostril detection dataset. The experimental results demonstrate our competitive performance in detection accuracy. Our code can be accessible at https://github.com/ConorLTH/airway-intubation_landmarks-detection. [ABSTRACT FROM AUTHOR]

As one of the minimally invasive surgeries (MIS), transoral robotic surgery (TORS) contributes to excellent oncological and functional outcomes. This paper introduces a compliant and flexible robotic system for transoral surgery, consisting of an execution part with flexible parallel mechanisms and a positioning part with a continuum structure. A pilot cadaveric study that mimics the procedure of the TORS using an intact cadaveric human head was conducted to evaluate the feasibility and efficiency of this robotic system. Both the initial setup time and the time cost by the robot to safely access the deep surgical area in the upper aerodigestive tract are shortened due to the enlarged workspace, compact structure, and increased flexibility. The proposed surgical robotic system is preliminarily demonstrated to be feasible for TORS, especially for the in-depth surgical sites in the upper aerodigestive tract. [ABSTRACT FROM AUTHOR]

Accurately detecting tumors and estimating the depth of tumors is essential in the surgical removal of tumors. In robotic-assisted surgery, autonomous robotic palpation has the potential to provide more precise detection, tumors’ depth estimation, and less intrusion when normal tissues surround tumors. In this article, by mimicking the human finger touch, we propose a tactile sensing-based deep recurrent neural network (DRNN) with long short-term memory (LSTM) architecture to improve the accuracy of the detection and depth estimation of tumors embedded in soft tissue. In the experimental setup, the hard inclusions simulate the tumors, while the phantom tissue is fabricated by silicon to simulate the soft tissue. During the experiment, the data from the force sensor and displacement of the robot palpation probe are for detection and depth estimation purposes. The collected sequential data set of the force and the displacement of the probe during one completed palpation process will go through the proposed DRNN network with deep LSTM architecture, in which the temporal dependencies of the sequential data will be captured in the cell states in the deep LSTM layers. Subsequently, the softmax classifier is adopted to determine if there is any hard inclusion exists and offer the depth estimation of the hard inclusions. Experiments based on 396 real data sets demonstrate that the detection accuracy for the testing data set is 99.2% and the depth estimation accuracy for the testing data set is 95.8%. The accuracy of the proposed method is best when comparing with other widely used methods. Note to Practitioners—The palpation of tumors motivated this article in the robot-assisted surgical systems through tactile feedback. In order to mimic the human touch on the soft tissue, this article presents a deep-learning-based approach to estimate the depth of the hard inclusions in the phantom tissue through force information. The displacement of the palpation probe and the touch force during one palpation are recorded as data sequences to train the deep model, which aims to capture dynamics and long-term dependence of the palpation process. In this article, we made the first successful attempt to accurately estimate the depth of the hard inclusions buried at different locations of the phantom tissue using only force information. The proposed approach can work in different robot-assisted scenarios, such as master–slave robotic surgery. In the clinic applications, the force sensor will be integrated at the end-effector of the robotic manipulator. According to the specific requirements, the force sensor and the robotic manipulator might be different from those used in this article. For some applications, such as the laparoscopic interventions, the complete vertical contact tends to be difficult to obtain due to the laparoscopic port effects. The projection of the recorded force data and displacement can obtain the information in the normal direction. The future work is going to be extended to tissue environments with arbitrary surface and tumors with various shapes/depths for more complex and prospective clinical applications. [ABSTRACT FROM AUTHOR]

Robot-assisted technologies are overcoming the limitations of the current approaches for transoral surgeries, which are suffering from limited vision and workspace. As a result, we develop a novel teleoperated parallel continuum robot with variable stiffness for collision avoidance. This paper focuses on the feasibility study on a cadaveric model for the robotic system as a first trial. We introduce the configuration of the robotic system, the description of the processes of the trial, including the setting of the robotic system, the test of stiffness, and the action of the manipulation. The contact force between the manipulators with different stiffness and the surrounding tissues and a series of surgical operations of the manipulator, including grasping, cutting, pushing, and pulling tissues under the master-slave control mode, were recorded and analyzed. Experimental results suggest that the typical surgical procedure on a cadaveric model was successfully performed. Moreover, the efficacy and feasibility of the developed robotic system are verified to satisfy the requirements of transoral robotic surgery (TORS). [ABSTRACT FROM AUTHOR]

Force sensing in robotic-assisted minimally invasive surgery (RMIS) is crucial for performing dedicated surgical procedures, such as bilateral teleoperation and palpation. Due to the bio-compatibility and sterilization requirements, a specially designed surgical tool/shaft is normally attached to the sensor while contacting the organ targets. Through this design, the measured force from the sensor usually contains uncertainties, such as noise, inertial force etc., and thus cannot reflect the actual interaction force with the tissue environment. Motivated to provide the authentic contact force between a robotic tool and soft tissue, we proposed a data-driven force compensation scheme without intricate modeling to reduce the effects of force measurement uncertainties. In this paper, a neural-network-based approach is utilized to automatically model the inertial force subject to noise during the robotic palpation procedure, then the exact contact force can be obtained through the force compensation method which cancels the noise and inertial force. Following this approach, the genuine interaction force during the palpation task can be achieved furthermore to improve the appraisal of the tumor surrounded by the soft tissue. Experiments are conducted with robotic-assisted palpation tasks on a silicone-based soft tissue phantom and the results verify the effectiveness of the suggested method. [ABSTRACT FROM AUTHOR]

We investigate a new framework of surgical instrument detection based on power-varying ultrasound images with simple and efficient pixel-wise intensity processing. Without using complicated feature extraction methods, we identified the instrument with an estimated optimal power level and by comparing pixel values of varying transducer power level images. The proposed framework exploits the physics of ultrasound imaging system by varying the transducer power level to effectively distinguish metallic surgical instruments from tissue. This power-varying image-guidance is motivated from our observations that ultrasound imaging at different power levels exhibit different contrast enhancement capabilities between tissue and instruments in ultrasound-guided robotic beating-heart surgery. Using lower transducer power levels (ranging from 40 to 75% of the rated lowest ultrasound power levels of the two tested ultrasound scanners) can effectively suppress the strong imaging artifacts from metallic instruments and thus, can be utilized together with the images from normal transducer power levels to enhance the separability between instrument and tissue, improving intraoperative instrument tracking accuracy from the acquired noisy ultrasound volumetric images. We performed experiments in phantoms and ex vivo hearts in water tank environments. The proposed multi-level power-varying ultrasound imaging approach can identify robotic instruments of high acoustic impedance from low-signal-to-noise-ratio ultrasound images by power adjustments. [ABSTRACT FROM AUTHOR]

Traditional posterior nasopharyngeal biopsy using a flexible nasal endoscope has the risks of abrasion and injury to the nasal mucosa and thus causing trauma to the patient. Recently, a new class of robots known as continuum tubular robots (CTRs) provide a novel solution to the challenge with miniaturized size, curvilinear maneuverability, and capability of avoiding collision within the nasal environment. This paper presents a compact CTR which is 35 cm in total length, 10 cm in diameter, 2.15 kg in weight, and easy to be integrated with a robotic arm to perform more complicated operations. Structural design, end-effector design, and workspace analysis are described in detail. In addition, teleoperation of the CTR using a haptic input device is developed for position control in 3D space. Moreover, by integrating the robot with three electromagnetic tracking sensors, a navigation system together with a shape reconstruction algorithm is developed. Comprehensive experiments are conducted to test the functionality of the proposed prototype; experiment results show that under teleoperation, the system has an accuracy of 2.20 mm in following a linear path, an accuracy of 2.01 mm in following a circular path, and a latency time of 0.1 s. It is also found that the proposed shape reconstruction algorithm has a mean error of around 1 mm along the length of the tubes. Besides, the feasibility and effectiveness of the proposed robotic system being applied to posterior nasopharyngeal biopsy are demonstrated by a cadaver experiment. The proposed robotic system holds promise to enhance clinical operation in transnasal procedures. [ABSTRACT FROM AUTHOR]

Multirobot comanipulation shows great potential in surpassing the limitations of single-robot manipulation in complicated tasks such as robotic surgeries. However, a dynamic multirobot setup in unstructured environments poses great uncertainties in robot configurations. Therefore, the coordination relationships between the end-effectors and other devices, such as cameras (hand–eye calibration) and tools (tool–flange calibration), as well as the relationships among the base frames (robot–robot calibration) have to be determined timely to enable accurate robotic cooperation for the constantly changing configuration of the systems. We formulated the problem of hand–eye, tool–flange, and robot–robot calibration to a matrix equation \mathbfAXB=YCZ. A series of generic geometric properties and lemmas were presented, leading to the derivation of the final simultaneous algorithm. In addition to the accurate iterative solution, a closed-form solution was also introduced based on quaternions to give an initial value. To show the feasibility and superiority of the simultaneous method, two nonsimultaneous methods were compared through thorough simulations under various robot movements and noise levels. Comprehensive experiments on real robots were also performed to further validate the proposed methods. The comparison results from both simulations and experiments demonstrated the superior accuracy and efficiency of the proposed simultaneous calibration method. [ABSTRACT FROM PUBLISHER]

Natural orifice translumenal endoscopic surgery (NOTES) is the latest surgery paradigm in which the abdominal cavity is accessed via the body's natural orifice, e.g., vagina, mouth, etc. Compared with traditional laparoscopic surgery, NOTES completely eliminates the skin incision and therefore benefits the patients in several aspects such as less post-operative pain, shorter recovery period, fewer complications, etc. Due to the unique characteristics of NOTES, instruments for traditional laparoscopic surgery are not suitable for NOTES and hence novel hardware design is necessary for facilitating system development. This paper gives an overview of the state of the arts in the development of surgical instruments for NOTES, particularly with a focus on the promising robotic endoscopes. [ABSTRACT FROM AUTHOR]

The developments of medical practices and medical technologies have always progressed concurrently. The relatively recent developments in endoscopic technologies have allowed the realization of the "minimally invasive" formof surgeries. The advancements in robotics facilitate precise surgeries that are often integrated with medical image guidance capability. This in turn has driven the further development of technology to compensate for the unique complexities engendered by this new format and to improve the performance and broaden the scope of the procedures that can be performed. Medical robotics has been a central component of this development due to the highly suitable characteristics that a robotic system can purport, including highly optimizable mechanical conformation and the ability to program assistive functions inmedical robots for surgeons to performsafe and accurate minimally invasive surgeries. In addition, combining the robot-assisted interventions with touch-sensing and medical imaging technologies can greatly improve the available information and thus help to ensure that minimally invasive surgeries continue to gain popularity and stay at the focus of modern medical technology development. This paper presents a state-of-the-art review of robotic systems for minimally invasive and noninvasive surgeries, precise surgeries, diagnoses, and their corresponding technologies. [ABSTRACT FROM AUTHOR]

Off-the-shelf industrial robotic technologies have achieved significant advancements in the past several decades in terms of mechanics and automation performances. We are expecting to take advantage of the industrial robots for assisting surgeons in surgeries and quick prototyping a robotic surgery system. In precise computer-assisted surgeries (CASs), such as pelvic-acetabular surgery, eye surgery, or neurosurgery, it is extremely important to position the tools accurately and precisely for surgical operations. Some of the industrial robotics arms are able to achieve good repeatability and dexterity while positioning the surgical tools. To enable the application of industrial robots in the surgical rooms, there are several other essential modules to be integrated to the robotic surgery systems, such as real-time navigation system, surgical planning system, and surgeon-guidance system. In this paper, we review the existing studies on the medical robots including the ones using industrial robots, and then investigate the essentials for using industrial robots in computer-integrated surgery. [ABSTRACT FROM AUTHOR]

Oropharyngeal tumor is traditionally resected from an open approach, often necessitating the need of a midline mandibulotomy in order to remove tumor safely with oncologic margins. The limitations imposed by a transoral route include poor visualization of the inferior extent of the oropharynx, rigid instrumentation, and inability to resect tumor that extends caudally into the supraglottis. While visualization with angled endoscopes, coupled with flexible laser development and microscopic magnification may overcome some of these limitations, this technique suffers from linear trajectory of the instruments which hampers expedient surgical resection in a 3-dimensional fashion. With development of the Da Vinci Surgical System, the safety and oncologic feasibility of removing oropharyngeal tumors are made possible because it provides a 3-dimensional magnification of the surgical field and wristed maneuverability of the surgical instruments which enable surgeons to operate around tight anatomical confines. Nevertheless, this first-generation robot is continually being modified with more flexibility and maneuverability through the development of robots like the FLEX Robotic System and more recently the Da Vinci Single Port System (SP). In this review, we will discuss the historic developments of robots for transoral applications, present the current approved robotic systems, and highlight the upcoming robots for transoral robotic surgery (TORS). Finally, we will also propose an ideal TORS surgical robot by highlighting the engineering technologies to accomplish these challenges. [ABSTRACT FROM AUTHOR]

Medical visual question answering (VQA) bridges the gap between visual information and clinical decision-making, enabling doctors to extract understanding from clinical images and videos. In particular, surgical VQA can enhance the interpretation of surgical data, aiding in accurate diagnoses, effective education, and clinical interventions. However, the inability of VQA models to visually indicate the regions of interest corresponding to the given questions results in incomplete comprehension of the surgical scene. To tackle this, we propose the surgical visual question localized-answering (VQLA) for precise and context-aware responses to specific queries regarding surgical images. Furthermore, to address the strong demand for safety in surgical scenarios and potential corruptions in image acquisition and transmission, we propose a novel approach called Calibrated Co-Attention Gated Vision-Language (C 2 G-ViL) embedding to integrate and align multimodal information effectively. Additionally, we leverage the adversarial sample-based contrastive learning strategy to boost our performance and robustness. We also extend our EndoVis-18-VQLA and EndoVis-17-VQLA datasets to broaden the scope and application of our data. Extensive experiments on the aforementioned datasets demonstrate the remarkable performance and robustness of our solution. Our solution can effectively combat real-world image corruption. Thus, our proposed approach can serve as an effective tool for assisting surgical education, patient care, and enhancing surgical outcomes. Our code and data will be released at https://github.com/longbai1006/Surgical-VQLAPlus. • We propose a Surgical-VQLA++ framework to connect answering and localization. • We incorporate feature calibration and adversarial contrastive training techniques. • We expand our datasets by including additional queries related to surgical tools. • Extensive experiments prove the effectiveness and robustness of our solution. [ABSTRACT FROM AUTHOR]

• Propose a spatio-temporal MTL (ST-MTL) model with a weight-shared encoder and taskaware spatio-temporal decoders. • Introduce a novel way to train the proposed MTL model by using asynchronous spatiotemporal optimization (ASTO). • Novel design of decoders, skip competitive scSE unit, and ConvLSTM++ boost up the model performance. • Generate task-oriented instruments saliency and scanpath similar to the surgeon's visual perception to get the priority focus on surgical instruments. Representation learning of the task-oriented attention while tracking instrument holds vast potential in image-guided robotic surgery. Incorporating cognitive ability to automate the camera control enables the surgeon to concentrate more on dealing with surgical instruments. The objective is to reduce the operation time and facilitate the surgery for both surgeons and patients. We propose an end-to-end trainable Spatio-Temporal Multi-Task Learning (ST-MTL) model with a shared encoder and spatio-temporal decoders for the real-time surgical instrument segmentation and task-oriented saliency detection. In the MTL model of shared-parameters, optimizing multiple loss functions into a convergence point is still an open challenge. We tackle the problem with a novel asynchronous spatio-temporal optimization (ASTO) technique by calculating independent gradients for each decoder. We also design a competitive squeeze and excitation unit by casting a skip connection that retains weak features, excites strong features, and performs dynamic spatial and channel-wise feature recalibration. To capture better long term spatio-temporal dependencies, we enhance the long-short term memory (LSTM) module by concatenating high-level encoder features of consecutive frames. We also introduce Sinkhorn regularized loss to enhance task-oriented saliency detection by preserving computational efficiency. We generate the task-aware saliency maps and scanpath of the instruments on the dataset of the MICCAI 2017 robotic instrument segmentation challenge. Compared to the state-of-the-art segmentation and saliency methods, our model outperforms most of the evaluation metrics and produces an outstanding performance in the challenge. [ABSTRACT FROM AUTHOR]