Search

We investigate a new framework of surgical instrument detection based on power-varying ultrasound images with simple and efficient pixel-wise intensity processing. Without using complicated feature extraction methods, we identified the instrument with an estimated optimal power level and by comparing pixel values of varying transducer power level images. The proposed framework exploits the physics of ultrasound imaging system by varying the transducer power level to effectively distinguish metallic surgical instruments from tissue. This power-varying image-guidance is motivated from our observations that ultrasound imaging at different power levels exhibit different contrast enhancement capabilities between tissue and instruments in ultrasound-guided robotic beating-heart surgery. Using lower transducer power levels (ranging from 40 to 75% of the rated lowest ultrasound power levels of the two tested ultrasound scanners) can effectively suppress the strong imaging artifacts from metallic instruments and thus, can be utilized together with the images from normal transducer power levels to enhance the separability between instrument and tissue, improving intraoperative instrument tracking accuracy from the acquired noisy ultrasound volumetric images. We performed experiments in phantoms and ex vivo hearts in water tank environments. The proposed multi-level power-varying ultrasound imaging approach can identify robotic instruments of high acoustic impedance from low-signal-to-noise-ratio ultrasound images by power adjustments. [ABSTRACT FROM AUTHOR]

In endoscopic submucosal dissection (ESD), the gastrointestinal (GI) tract warrants the surgical instruments to navigate through a long, narrow and tortuous endoscope. This poses a great challenge in developing ESD instruments with small dimensions, flexibility, and high distal dexterity. In this work, we propose the first Transendoscopic Flexible Parallel Continuum Robotic mechanism to develop a miniature dexterous flexible-stiff-balanced Wrist (FPCW). Besides, it can steer multifunctional instruments of diameters 2.5 mm to 3.5 mm, including the electrosurgical knife, injection needle, and forceps. Our FPCW instruments are adaptable to commercially available dual-channel endoscopes (diameter: <12 mm, channel width: 2.8 mm and around 3.8 mm). Furthermore, we develop a surgical telerobotic system, called DREAMS (Dual-arm Robotic Endoscopic Assistant for Minimally Invasive Surgery), by using our smallest FPCW instruments for bimanual ESD procedures. First, we conduct a series of experiments to determine the FPCW's design and kinematics parameters and to verify the mechanical properties of the FPCW instruments' prototypes, including workspace, stiffness, strength, and teleoperation accuracy. Second, we validate the functionality of the FPCW instruments through ex-vivo tests by performing ESD steps on porcine stomachs. Finally, we perform an invivo test on a live porcine model and showcase that our developed DREAMS can be teleoperated intuitively to perform bimanual ESD efficiently with an average dissection speed of 108.95 mm2/min at the greater curvature in gastric body, which demonstrates that our DREAMS has satisfactory maneuverability as well as accuracy and is more competitive than counterpart robotic systems. [ABSTRACT FROM AUTHOR]

Accurately detecting tumors and estimating the depth of tumors is essential in the surgical removal of tumors. In robotic-assisted surgery, autonomous robotic palpation has the potential to provide more precise detection, tumors’ depth estimation, and less intrusion when normal tissues surround tumors. In this article, by mimicking the human finger touch, we propose a tactile sensing-based deep recurrent neural network (DRNN) with long short-term memory (LSTM) architecture to improve the accuracy of the detection and depth estimation of tumors embedded in soft tissue. In the experimental setup, the hard inclusions simulate the tumors, while the phantom tissue is fabricated by silicon to simulate the soft tissue. During the experiment, the data from the force sensor and displacement of the robot palpation probe are for detection and depth estimation purposes. The collected sequential data set of the force and the displacement of the probe during one completed palpation process will go through the proposed DRNN network with deep LSTM architecture, in which the temporal dependencies of the sequential data will be captured in the cell states in the deep LSTM layers. Subsequently, the softmax classifier is adopted to determine if there is any hard inclusion exists and offer the depth estimation of the hard inclusions. Experiments based on 396 real data sets demonstrate that the detection accuracy for the testing data set is 99.2% and the depth estimation accuracy for the testing data set is 95.8%. The accuracy of the proposed method is best when comparing with other widely used methods. Note to Practitioners—The palpation of tumors motivated this article in the robot-assisted surgical systems through tactile feedback. In order to mimic the human touch on the soft tissue, this article presents a deep-learning-based approach to estimate the depth of the hard inclusions in the phantom tissue through force information. The displacement of the palpation probe and the touch force during one palpation are recorded as data sequences to train the deep model, which aims to capture dynamics and long-term dependence of the palpation process. In this article, we made the first successful attempt to accurately estimate the depth of the hard inclusions buried at different locations of the phantom tissue using only force information. The proposed approach can work in different robot-assisted scenarios, such as master–slave robotic surgery. In the clinic applications, the force sensor will be integrated at the end-effector of the robotic manipulator. According to the specific requirements, the force sensor and the robotic manipulator might be different from those used in this article. For some applications, such as the laparoscopic interventions, the complete vertical contact tends to be difficult to obtain due to the laparoscopic port effects. The projection of the recorded force data and displacement can obtain the information in the normal direction. The future work is going to be extended to tissue environments with arbitrary surface and tumors with various shapes/depths for more complex and prospective clinical applications. [ABSTRACT FROM AUTHOR]

Robot-assisted technologies are overcoming the limitations of the current approaches for transoral surgeries, which are suffering from limited vision and workspace. As a result, we develop a novel teleoperated parallel continuum robot with variable stiffness for collision avoidance. This paper focuses on the feasibility study on a cadaveric model for the robotic system as a first trial. We introduce the configuration of the robotic system, the description of the processes of the trial, including the setting of the robotic system, the test of stiffness, and the action of the manipulation. The contact force between the manipulators with different stiffness and the surrounding tissues and a series of surgical operations of the manipulator, including grasping, cutting, pushing, and pulling tissues under the master-slave control mode, were recorded and analyzed. Experimental results suggest that the typical surgical procedure on a cadaveric model was successfully performed. Moreover, the efficacy and feasibility of the developed robotic system are verified to satisfy the requirements of transoral robotic surgery (TORS). [ABSTRACT FROM AUTHOR]

Oropharyngeal tumor is traditionally resected from an open approach, often necessitating the need of a midline mandibulotomy in order to remove tumor safely with oncologic margins. The limitations imposed by a transoral route include poor visualization of the inferior extent of the oropharynx, rigid instrumentation, and inability to resect tumor that extends caudally into the supraglottis. While visualization with angled endoscopes, coupled with flexible laser development and microscopic magnification may overcome some of these limitations, this technique suffers from linear trajectory of the instruments which hampers expedient surgical resection in a 3-dimensional fashion. With development of the Da Vinci Surgical System, the safety and oncologic feasibility of removing oropharyngeal tumors are made possible because it provides a 3-dimensional magnification of the surgical field and wristed maneuverability of the surgical instruments which enable surgeons to operate around tight anatomical confines. Nevertheless, this first-generation robot is continually being modified with more flexibility and maneuverability through the development of robots like the FLEX Robotic System and more recently the Da Vinci Single Port System (SP). In this review, we will discuss the historic developments of robots for transoral applications, present the current approved robotic systems, and highlight the upcoming robots for transoral robotic surgery (TORS). Finally, we will also propose an ideal TORS surgical robot by highlighting the engineering technologies to accomplish these challenges. [ABSTRACT FROM AUTHOR]

Traditional posterior nasopharyngeal biopsy using a flexible nasal endoscope has the risks of abrasion and injury to the nasal mucosa and thus causing trauma to the patient. Recently, a new class of robots known as continuum tubular robots (CTRs) provide a novel solution to the challenge with miniaturized size, curvilinear maneuverability, and capability of avoiding collision within the nasal environment. This paper presents a compact CTR which is 35 cm in total length, 10 cm in diameter, 2.15 kg in weight, and easy to be integrated with a robotic arm to perform more complicated operations. Structural design, end-effector design, and workspace analysis are described in detail. In addition, teleoperation of the CTR using a haptic input device is developed for position control in 3D space. Moreover, by integrating the robot with three electromagnetic tracking sensors, a navigation system together with a shape reconstruction algorithm is developed. Comprehensive experiments are conducted to test the functionality of the proposed prototype; experiment results show that under teleoperation, the system has an accuracy of 2.20 mm in following a linear path, an accuracy of 2.01 mm in following a circular path, and a latency time of 0.1 s. It is also found that the proposed shape reconstruction algorithm has a mean error of around 1 mm along the length of the tubes. Besides, the feasibility and effectiveness of the proposed robotic system being applied to posterior nasopharyngeal biopsy are demonstrated by a cadaver experiment. The proposed robotic system holds promise to enhance clinical operation in transnasal procedures. [ABSTRACT FROM AUTHOR]

Multirobot comanipulation shows great potential in surpassing the limitations of single-robot manipulation in complicated tasks such as robotic surgeries. However, a dynamic multirobot setup in unstructured environments poses great uncertainties in robot configurations. Therefore, the coordination relationships between the end-effectors and other devices, such as cameras (hand–eye calibration) and tools (tool–flange calibration), as well as the relationships among the base frames (robot–robot calibration) have to be determined timely to enable accurate robotic cooperation for the constantly changing configuration of the systems. We formulated the problem of hand–eye, tool–flange, and robot–robot calibration to a matrix equation \mathbfAXB=YCZ. A series of generic geometric properties and lemmas were presented, leading to the derivation of the final simultaneous algorithm. In addition to the accurate iterative solution, a closed-form solution was also introduced based on quaternions to give an initial value. To show the feasibility and superiority of the simultaneous method, two nonsimultaneous methods were compared through thorough simulations under various robot movements and noise levels. Comprehensive experiments on real robots were also performed to further validate the proposed methods. The comparison results from both simulations and experiments demonstrated the superior accuracy and efficiency of the proposed simultaneous calibration method. [ABSTRACT FROM PUBLISHER]

• Propose a spatio-temporal MTL (ST-MTL) model with a weight-shared encoder and taskaware spatio-temporal decoders. • Introduce a novel way to train the proposed MTL model by using asynchronous spatiotemporal optimization (ASTO). • Novel design of decoders, skip competitive scSE unit, and ConvLSTM++ boost up the model performance. • Generate task-oriented instruments saliency and scanpath similar to the surgeon's visual perception to get the priority focus on surgical instruments. Representation learning of the task-oriented attention while tracking instrument holds vast potential in image-guided robotic surgery. Incorporating cognitive ability to automate the camera control enables the surgeon to concentrate more on dealing with surgical instruments. The objective is to reduce the operation time and facilitate the surgery for both surgeons and patients. We propose an end-to-end trainable Spatio-Temporal Multi-Task Learning (ST-MTL) model with a shared encoder and spatio-temporal decoders for the real-time surgical instrument segmentation and task-oriented saliency detection. In the MTL model of shared-parameters, optimizing multiple loss functions into a convergence point is still an open challenge. We tackle the problem with a novel asynchronous spatio-temporal optimization (ASTO) technique by calculating independent gradients for each decoder. We also design a competitive squeeze and excitation unit by casting a skip connection that retains weak features, excites strong features, and performs dynamic spatial and channel-wise feature recalibration. To capture better long term spatio-temporal dependencies, we enhance the long-short term memory (LSTM) module by concatenating high-level encoder features of consecutive frames. We also introduce Sinkhorn regularized loss to enhance task-oriented saliency detection by preserving computational efficiency. We generate the task-aware saliency maps and scanpath of the instruments on the dataset of the MICCAI 2017 robotic instrument segmentation challenge. Compared to the state-of-the-art segmentation and saliency methods, our model outperforms most of the evaluation metrics and produces an outstanding performance in the challenge. [ABSTRACT FROM AUTHOR]