Search

Registration is an essential problem in image-guided surgery (IGS) since it brings different involved coordinate frames together. Nonrigid or deformable registration still faces many challenges, such as two point sets (PSs) are partially overlapped. To tackle the challenges in the nonrigid registration, we introduce a new two-step point-based registration pipeline that includes two steps. In the first step, the rigid transformation between the two spaces is recovered where the orientation vectors are adopted. In the second step, built on the nonrigid coherent point drift (CPD) approach, the anisotropic positional noise is also assumed. Registration results on the human liver verify the proposed approach’ great improvements over the other methods. First, the rotation and translation are recovered with smaller error values than the existing methods. Second, our registration method’s performance is much more robust to the partial overlapping between two PSs. Third, the two-step registration framework achieves the best performances in most test cases when there is a localization error in acquiring the intraoperative data. Note to Practitioners—A novel registration approach is presented for image-guided liver surgery (LGLS). Compared with existing nonrigid registration methods, two significant changes (or improvements) exist in the proposed registration framework: 1) the normal vectors are extracted and utilized in the rigid registration step and 2) the anisotropic positional uncertainties are considered. In both steps, the registration problems are formulated as a maximum likelihood (ML) problems and dealt with the expectation-maximization (EM) technique. In both steps, the matrix form of the updated positional covariance is provided and can speed up the computational process. The readers are reminded that with extra information and a more general positional error assumption, our approach demonstrates improved performances in the case of partial-to-full alignment. [ABSTRACT FROM AUTHOR]

The development and use of soft or flexible structural matters across various research domains have drastically increased in recent decades. Its flexible, compliant nature and interactive safety have made it a preferred candidate compared to its rigid bodied counterparts. However, the lack of robust soft robot detection and localization techniques has constrained its feedback control system, limiting its application. This paper proposes a novel depth sensor-based detection and tracking algorithm adaptive to shape morphing robots. The detection algorithm first employs optimal iterative threshold segmentation on the depth image to remove background and detect occlusions. Blob detection and polygon approximation using Fourier descriptor techniques are then utilized to detect and extract the contours of the shape morphing soft robots. Finally, using the pixel coordinates obtained from the detection algorithm, transformation is applied from the pixel coordinate system to the world coordinate system on the depth image to achieve motion tracking in 3D space. Qualitative and quantitative assessments prove that the detection algorithm is robust and accurate in tracking shape morphing soft robots. [ABSTRACT FROM AUTHOR]

This paper presents a sensor fusion-based surgical instrument tracking system which uses multiple monocular modules. The system is an optical tracking system, which has been widely utilized in the image-guide surgery because of its high accuracy and precision. However, the line-of-sight occlusion problem which remains unresolved in current systems frustrates surgeons during the operation. To address this challenge, we propose a surgical instrument tracking system based on multiple monocular modules. The rationale is to enable the system to track the surgical instruments inside the surgical site from different views. Three sensor fusion algorithms are proposed to integrate all sensor data from the multimodule system. In order to show the feasibility of the tracking system, simulations and comparison experiments have been carried out. The intensive investigation results give a practical instruction to the real implementation of the proposed system in image-guided interventions. Moreover, an image-guided surgical trial by using a cadaver head has been carried out to validate the feasibility of the proposed system and the tracking algorithms. The results from both the simulation and the cadaver trial have shown the effectiveness of the proposed robust fusion algorithm. Note to Practitioners—Optical tracking system (OTS) for image-guided surgery has suffered from line-of-sight (LoS) occlusion and field-of-view (FoV) limitation problems for a long time; nevertheless, no satisfactory solution has been given yet. In the real-world practice, surgeons and assistants have to manually adjust the position and orientation of the optical tracker during operation. This interrupts the surgery process and degrades the efficiency of the surgical procedure. Instead of using one tracker for the image-guided surgery, our approach is to utilize distributed modular trackers to avoid LoS occlusion and maintain FoV of the OTS. Based on our previous work, this paper is to provide an optimal fusion algorithm that takes the advantage of the multiple modules and validate the system in both the simulation and the real-world surgical environment. [ABSTRACT FROM AUTHOR]

To accurately navigate surgical instruments around in vivo environments in minimally invasive surgeries through flexible endoscopes or rigid scopes, real-time tracking of surgical instruments is required. It is even challenging to track regular-shaped rigid bodies on a less-distorted general-purpose monocular vision, while most of the surgical instruments or devices used are cylindrical and slim imaged with endoscope distortion in nature. Thus, more dedicated and accurate rigid-body tracking approach for cylindrical endoscopic instruments will be more helpful in intra-operative guidance. In this paper, we present an endoscopic instrument tracking approach with printable markers that consists of a green band and a square with four white and one black circle inscribed in it. The proposed approach can estimate the six degrees of freedom pose, i.e., X, Y, Z, Roll, Pitch, and Yaw, of the imaged instrument with respect to the endoscope. Detailed experiments justify that our proposed marker detection framework allows translation motions between 42 and 100 mm along depth direction and rotational motions around X and Y up to ±180º, with 90% detection rate in dynamic condition, 99% in static condition, and accurate pose estimation with a mean translation error of 1.286 mm with a standard deviation of 0.673 mm and a mean rotational error of 1.497° with a standard deviation of 0.873°. Our proposed marker can be easily printed and wrapped to the existing endoscopic instruments, which potentially can assist in the precise guidance in minimally invasive surgeries without changing the existing workflow or incurring significant cost burdens. [ABSTRACT FROM AUTHOR]

Optical flow (OF) measures the displacement of the projections of visual landmarks on the image plane and is widely used for motion and structure estimations. Fast rotational motion results in large image motion and poses a major challenge to many OF algorithms. To improve the performance of OF estimations, gyroscopes can be adopted to predict the image motion between adjacent frames when the camera motion is dominated by rotation. This paper first proposes an offline method for calibrating the camera focal length, gyroscope bias, the relative orientation and time offset between the camera and gyroscopes without using a man-made target. Then a simple and efficient method is proposed to aid the OF computation with the gyroscope readings. Our approach is evaluated using a low-cost endoscope-gyroscope system and shows improved visual tracking performance as compared to the stand-alone OF algorithm. The method is suited for use on lightweight platforms equipped with gyroscopes and an endoscopic camera, which potentially can provide efficient image guidance in endoscopic procedures. [ABSTRACT FROM PUBLISHER]

In this paper, the fault diagnosis (FD) problem in image-based visual servoing with eye-in-hand configurations is investigated. The potential failures are detected and isolated based on approximating parameters related. First, the failure scenarios of the visual servoing systems are reviewed and classified into the actuator and sensor faults. Second, a residual generator is proposed to detect the failure occurrences, based on the Kalman filter. Third, a decision table is proposed to isolate the fault type. Finally, simulation and experimental results are given to validate the efficacy and the efficiency of the proposed FD strategies. [ABSTRACT FROM PUBLISHER]

Multirobot comanipulation shows great potential in surpassing the limitations of single-robot manipulation in complicated tasks such as robotic surgeries. However, a dynamic multirobot setup in unstructured environments poses great uncertainties in robot configurations. Therefore, the coordination relationships between the end-effectors and other devices, such as cameras (hand–eye calibration) and tools (tool–flange calibration), as well as the relationships among the base frames (robot–robot calibration) have to be determined timely to enable accurate robotic cooperation for the constantly changing configuration of the systems. We formulated the problem of hand–eye, tool–flange, and robot–robot calibration to a matrix equation \mathbfAXB=YCZ. A series of generic geometric properties and lemmas were presented, leading to the derivation of the final simultaneous algorithm. In addition to the accurate iterative solution, a closed-form solution was also introduced based on quaternions to give an initial value. To show the feasibility and superiority of the simultaneous method, two nonsimultaneous methods were compared through thorough simulations under various robot movements and noise levels. Comprehensive experiments on real robots were also performed to further validate the proposed methods. The comparison results from both simulations and experiments demonstrated the superior accuracy and efficiency of the proposed simultaneous calibration method. [ABSTRACT FROM PUBLISHER]

Central catadioptric omnidirectional images exhibit serious nonlinear distortions due to the involved quadratic mirrors. Therefore, features based on the conventional pin-hole model are hard to achieve satisfactory performances when directly applied to the distorted omnidirectional images. This paper analyzes the catadioptric geometry to facilitate modeling the nonlinear distortions of omnidirectional images. Different to the conventional imaging model, the prior information is considered in catadioptric system. A parameterized neighborhood mapping model is proposed to efficiently calculate the neighborhood of an object based on its measurable radial distance in the image plane. On the basis of the parameterized nonlinear model, a distortion-invariant fragment-based joint-feature mixture model of Gaussian is presented for human target tracking in omnidirectional vision. Under the framework of Gaussian Mixture Model, the problem of feature matching is converted into the feature clustering. The joint probability distribution of a joint-feature class is modeled by a mixture of Gaussian. A weight contribution mechanism is designed to flexibly weight the fragments contribution based on their responses, which leads to a robust tracking even under serious partial occlusion. Finally, experiments validate the advantage of the proposed algorithm over other conventional approaches. [ABSTRACT FROM AUTHOR]

Optical means of instrument tracking has been widely used in image-guided interventions and considered the de facto standard for tracking rigid bodies with a direct line-of-sight. However, the occlusion problem which remains unresolved in current systems frustrates surgeons during the operation. To address this challenge, we propose a surgical instrument tracking system based on multiple reconfigurable monocular modules. The main approach is to enable the system to dynamically reconfigure the multiple monocular modules when occlusion occurs partially within the workspace. In this paper, we focus on the system architecture and an agile multicamera calibration method which only uses the customized tool for the surgical instrument tracking scenario. Additionally, two fast non-iterative algorithms are proposed and studied. In order to show the feasibility and superiority of the corresponding multicamera calibration algorithm, comparison experiments have carried out. The intensive investigation results give a practical instruction to the real implementation of the proposed system in image-guided interventions. [ABSTRACT FROM PUBLISHER]

This paper presents a passive-marker-based optical tracking system utilizing dual Kinect sensors and additional custom optical tracking components. To obtain sub-millimeter tracking accuracy, we introduce robust calibration of dual infrared sensors and point correspondence establishment in a stereo configuration. The 3D localization is subsequently accomplished using multiple back projection lines. The proposed system extends existing inexpensive consumer electronic devices, implements tracking algorithms, and shows the feasibility of applying the proposed low-cost system to surgical training for computer assisted surgeries. [ABSTRACT FROM AUTHOR]