Search

Highlights • It is difficult to visualize extra-hepatic biliary anatomy clearly because of long-presence of ICG in liver when administered intravenously. • Intracholecystic ICG injection illuminates extra-hepatic biliary tree preferentially thus reducing background hepatic noise. • Surgeons can experience more satisfaction with the use of fluorescent cholangiography during laparoscopic cholecystectomy when the intracystic route of ICG administration is utilized., Introduction The utility of intracystic administration of indocyanine green for near-infrared fluorescent cholangiography in acute calculous cholecystitis initially treated with percutaneous transhepatic gallbladder drainage (PTGBD) was described in this report. Presentation of case Two cases who underwent near-infrared fluorescent cholangiography guided interval laparoscopic cholecystectomy two weeks post-PTGBD were studied retrospectively. Both patients were diagnosed with moderate acute calculous cholecystitis based on diagnostic criteria of the Tokyo guidelines. Two routes of indocyanine green administration were utilized during surgery, first through direct intracystic administration through PTGBD tube (5 ml of 12.5 mg ICG) to achieve critical view of safety and then intravenous administration (1 ml of 2.5 mg ICG) to visualize cystic artery. Discussion Both patients had critical view of safety visualized clearly with ICG with the operation time of 84 and 125 min in cases 1 and 2, respectively without any intra or postoperative complications. Conclusion In comparison with intravenous ICG administration, trans-PTGBD ICG route can provide better signal-to-noise ratio by avoiding hepatic fluorescence and thus increasing the bile duct to liver contrast. However, ICG may enter the lymphatic system through necrotic and inflammatory gallbladder mucosa, of which lymph spillage during gallbladder dissection can obscure the fluorescent view.

Objective: The objective of this study was to evaluate augmented reality-assisted single-incision laparoscopic adrenalectomy (AR-SILA) and compare with pure SILA. Materials and Methods: Between January 2009 and March 2012, a total of 19 patients had received SILA. Eight cases were AR-SILA and the others were pure SILA. Results: Eight AR-SILA procedures and 11 SILA were successfully completed. There was no significant difference between the two groups in terms of age, body mass index, and tumor size. Operative times were shorter in the AR-SILA group (102.5 ± 12.8 vs. 150.9 ± 46.3, P = 0.005). The mean blood loss in the AR-SILA group was slightly less than that in the SILA group (60.0 ± 25.6 vs. 109.1 ± 80.1, P = 0.129). There was neither postoperative mortality nor morbidity at the time of discharge and during follow-up. Conclusion: AR-SILA provides important intraoperative information for surgeons to recognize where the pathological lesions and vessels are located beyond the surgeon's direct vision. It made SILA safer and easier.

Endoscopic surgery causes less tissue injury compared to open surgical techniques, thus promoting more rapid recuperation and reduced post-operative pain. Endoscopy, however, allows the surgeon to ...

In USG-guided interventional procedures, the surgeons need to mentally register the reference data set (computed tomography [CT] or magnetic resonance [MR] images), and the working data set of USG (real-time). This study presents a system to combine real time USG information with preoperative 3D CT virtual model using landmark registration. The future goal of this system is to apply for radiofrequency needle tracking during the surgery.

Disparity estimation for stereo endoscopic images is difficult due to their lack of distinct textures for matching. Many well-known algorithms fail in this kind of application to estimate a reliable dense disparity map. In this paper, we propose a strategy of using a sparse feature point set to estimate reliable disparity values, which are then propagated to other non-feature points to form the final dense disparity map. Our selected feature points include: SIFT, Canny-edge, Canny-edge-dilated, and grid points. The algorithms for disparity propagation are based on bilateral interpolation and refinement. Experiments show that our algorithm is successful in shaping the instruments in disparity map. This is helpful in 3D display for human perception. Disparity estimation for endoscopic images is still an open issue. Our preliminary result still presents some space for improvement.

Sub-surface information during a laparoscopic surgery is captured with a near infrared imaging where the tissue is highlighted with the help of an indocyanine dye injected to the patient before the surgery. The surgeons can use such system during the surgery, however, they need to swap the light source between infra-red and the white light to complete the surgery. Swapping the light source causes inconvenience and increase in the time of the surgery. The current study aims to develop a method for superimposing and tracking the infrared imagery on the laparoscopic video acquired under white light. In this study, the infrared imagery was captured and registered to one of the white light imagery from the endoscope video using a 2D image registration technique. The registration technique was based on maximization of mutual information between two images. The edges on the registered infrared image was detected using canny edge detection algorithm. A homography matrix between consecutive video frames under white light was calculated after characteristic feature point detection and matching. The homography matrix was applied on the registered infrared image (only edge) and a new registered edge image was generated. The process continued for every new frame of the video under white light. The system was applied with a video acquisition rate of 20 frames per second. The system needs to be evaluated for its influence on the time of surgery.

An expanding mosaic view is one of the possible solutions for narrow field of view during endoscopic surgery. The current work presents a system to create an expanding mosaic view of the video during laparoscopy where the laparoscope is held and moved by a customized robot arm.

Minimally Invasive Surgery (MIS) is a growing surgical technique comparing to traditional open surgery since it dramatically reduces patient trauma and costs but MIS technique is difficult because of looking at a 2D screen to perform the surgery. Depth perception is thus lost due to 2D endoscopic capturing and display. Therefore, the paper proposes real-time 2D to 3D conversion for enhancing MIS to provide depth perception on 3D display for increasing surgical quality, improving in safety and speed of procedures, and also to lower learning curve. The proposed method to convert an endoscopic 2D video frame into a depth map and consequently into a pair of right and left image for viewing on a stereoscopic display is based on the use of a bilateral filter approximated by a bilateral grid. The architecture is to make edges sharp to enhance moving instruments and to preserve chamber geometry segmentation. Results of verification by simulated surgery and questionnaire for surgeons proof the feasibility of 2D to 3D conversion on MIS.

Some new 3D image technologies including image capturing, digital data processing, 2D/3D video conversion, and naked eye 2D/3D compatible display have been developed in Taiwan for the future 3D world applications. An innovative 3D image capturing and displaying architecture is a cost effective system for 3D content generation, transmission, and display. The multiview images of fine art such as sculpture can be captured by using 3D scanner or camera. The obtained multiview images can be transmitted though popular streaming channel to the cloud computing center that handles huge tasks of 2D/3D image processing. Then, the multi-view images are fed into the new types of 2D/3D compatible display. This new type of display using a specially designed microretarder can present 2D and naked eye 3D images on the same display simultaneously. Some new 3D applications with real-time 2D to 3D conversion technology are also introduced.

Introduction: Endoscopic surgery causes reduced injury to tissues which helps in rapid recovery and less painful post-operative period of patients. However, its narrow field of view and loss of depth perception in endoscope image make surgeon's task difficult. Moreover, in endoscopic surgery surgeons can only see the surface of the surgical anatomy. In this study these limitations have been addressed with the development of an augmented reality system with 3D visualization. Method: The system is comprising of infrared based optical tracker, 2D endoscope system, computer and a 3D monitor. The system was calibrated to bring all the components to a single reference frame of the optical tracker. A phantom and its 3D CT model were used, respectively, to form the real and virtual part of the augmented reality. The endoscope camera position was tracked in 3D using a tracker tool mounted on it. The camera in the virtual environment was updated with the new positions of the endoscope camera. The endoscope video frame and the virtual model rendered image were superimposed to form augmented reality view. The depth map of the virtual scene was further used to make an stereo pair of the augmented reality. Results: The AR system was examined with endoscope video of phantom. It could produce well aligned real and virtual component. The contour different of the two components was 5 to 10 mm. Conclusion: An AR system for endoscopic surgery was successfully implemented on a phantom. It needs to include motion and deformation model to be applied on real patients.

Minimally invasive robotic surgery (MIRS) procedures are performed through small incisions on patient's body and provide the advantages of considerably reduced tissue trauma and shorter recovery time. However, loss of haptic feedback is one of the major drawbacks of minimally invasive robotic surgery. Surgeons may not always be able to control the force applied by the robotic arm because of lack of sense of tissue resistance. This study aims at determining stresses at the trocar/endoscope interface in order to limit the range of forces applied by robot on patient's skin and to avoid its inadvertent injury. Simulation of the robot movement was done using a computer aided design (CAD) software. Simulation results suggest that, due to mechanical advantage, contact forces calculated around the trocar are at least 10 times higher than the force applied on the robot arm. Therefore, surgeons need to be cautious to avoid too much pressure on patient's skin by applying high force to the robot arm.

Minimally invasive surgeries(MIS) possess obvious advantages for patients but require more specially trained personnel than common, open procedure. With modern advances in robotics, it becomes possible to ease these requirements by introducing robotized assistants into operation. However, for robot-assisted solo MIS, it is crucial to have appropriate methods for robot control. This work presents a design of flag language as a novel human-to-robot communication method based on image processing of the endoscope video during real-time intervention. The proposed system comprises autonomous positioning of endoscope holder combined with recognition of surgeon intention by analyzing surgical instrument postures. The proposed algorithm has been evaluated using experimental setup as well as video clips from laparoscopic operation. Experimental results show that the flag posture can be detected satisfactorily with speed of 12 frames per second, as well as system robustness under moderate lighting and noise conditions.

The current study describes a technique for reconstructing a three dimensional surface from multiple images of the (single camera) endoscopic video. A 3D shape of the scene in each image frame of the endoscopic video was reconstructed using shape from shading technique. The characteristic feature points on the 2D images were detected using SURF algorithm and the features were matched using BRIEF and Hamming distance criteria. The matched feature points between the consecutive frames were used to find the transformation matrix to align the 3D surfaces of the consecutive video frames. Using the transformation matrix, the 3D surfaces were stitched (registered) together. The method was applied on a recorded video of the laparoscopic surgery. The reconstructed surface provides a wider view and depth information to the viewers.

This paper presents a technique that generates rough depth maps from laparoscopy camera using feature tracking techniques for depth sampling. The method is applicable to single laparoscopy video images through local motion cues to improve the inferred depth maps. The feature tracking is used to ensure temporal depth consistency, such as SURF. The results show that the proposed methods significantly improve the depth estimation for laparoscopy surgery. The technique can be used to convert a 2D laparoscopy video into stereo for 3D visualization, and it demonstrates this through a laparoscopy surgery video, including results of surgical instruments.

Introduction: Endoscopy is widely used in the surgical world. Reduced surgical injury during endoscopic surgery makes the patient’s life easy. However, the endoscopic surgery becomes a challenge for the novice surgeons because of the narrow field of view and the lack of 3D perception in the 2D endoscope image. Such limitations of the endoscope may be addressed with a 3D panorama created with the endoscopic 2D images. Few studies have been reported to create a 3D panorama from endoscopic images. However, to our knowledge, no study has reported a 3D panorama system which uses only a single camera image from the conventional endoscope to make a 3D shape of the organs, and further stitch those 3D shapes to create a 3D panorama of the surgical scene. Such system would enable surgeons to have an extended field of view of the surgical scene with a 3D perception. Method: Images from the endoscopic surgery video were used in this study. A shape from shading (SfS) algorithm was applied to create a 3D shape of the organs in the images. Characteristic feature points were identified on the images using a feature detection algorithm (SURF). The matching feature points in the consecutive images were found using a feature matching algorithm (BRIEF). An iterative closest point (ICP) algorithm was applied to stitch 3D shapes from the consecutive images to synthesize the 3D panorama. Results: The method was applied on 100 consecutive video frames from an endoscopic video of a patient. The root mean square error for the registration of the consecutive image feature points was

We propose a novel framework for reconstructing 3Dliver vessel model from CT images. The proposed algorithm consists of vessel detection, vessel tree reconstruction and vessel radius estimation. First, we employ the tubular-filter based approach to detect vessel structure and construct the minimum spanning tree to bridge all the gaps between vessels. Then, we propose an approach to estimate the radius of the vessel at all vessel centerline voxels based on the local patch descriptors. Using the proposed 3D vessel reconstruction system can provide detailed 3D liver vessel model very efficiently. Our experimental results demonstrate the accuracy of the proposed system for 3D liver vessel reconstruction from 3D CT images.

Pulmonary edema, i.e. excess of extravascular fluid in lungs, is a common manifestation of various clinical conditions. Although the etiology of the pulmonary edema is deduced with the help of history, physical examination and various biochemical and radiological investigations, computer aided evaluation of pulmonary edema will be helpful for physicians in determining the course of management for the condition. In this study we present texture analysis of chest x-ray, using Gabor filter and one of the machine learning techniques, Support Vector Machine (SVM), to distinguish the normal chest x-ray from the chest x-ray of pulmonary edema. This is an initial step towards computer aided quantitative assessment of the pulmonary edema using chest x-ray.

In this paper, we present a quality enhancement scheme for endoscopic images. Traditional algorithms might be able to enhance the image contrast, but possible over-enhancement also lead to bad overall visual quality which prevents surgeons from accurate examination or operations of instruments in Minimal Invasive Surgery (MIS). Our proposed scheme integrates the well-known retinex algorithm with a pseudo-HDR (High Dynamic Range) synthesis process, designed to compose of three parts: multiscale retinex with gamma correction (MSR-G), local brightness range expansion (brightness diversity), and bilateral-filter-based HDR image fusion. Experiment results demonstrate that the proposed scheme is able to enhance image details and keep the overall visual quality good as well, with respect to other existing methods.

Laparoscopic surgery is indispensable from the current surgical procedures. It uses an endoscope system of camera and light source, and surgical instruments which pass through the small incisions on the abdomen of the patients undergoing laparoscopic surgery. Conventional laparoscope (endoscope) systems produce 2D colored video images which do not provide surgeons an actual depth perception of the scene. In this work, the problem was formulated as synthesizing a stereo image of the monocular (conventional) laparoscope image by incorporating into them the depth information from a 3D CT model. Various algorithms of the computer vision including the algorithms for the feature detection, matching and tracking in the video frames, and for the reconstruction of 3D shape from shading in the 2D laparoscope image were combined for making the system. The current method was applied to the laparoscope video at the rate of up to 5 frames per second to visualize its stereo video. A correlation was investigated between the depth maps calculated with our method with those from the shape from shading algorithm. The correlation coefficients between the depth maps were within the range of 0.70-0.95 (P

This paper proposed a technique for skin curve registration based on landmark detection and calibrated camera-projector system for medical purpose. The technique applies the algorithm which registers a precalculated 3D model for the 2D images of the video frames. The algorithm first calculates the 3D locations of landmarks based upon the calibrated camera projector system and the matching the landmarks in the 3D model using CT-scan. We then generate the image to be projected on the skin curve of a patient from the registered 3D spinal model, and it is created for projecting on the pork or real patient for surgeons to process surgical procedure by utilizing their natural eyes system. The experiments demonstrate the proposed registration method on both animal and real patient and evaluated by several surgeons for spinal surgery.

Operation in minimally invasive surgery is more difficult since the surgeons perform operations without haptic feedback or depth perception. Moreover, the field of view perceived by the surgeons through endoscopy is usually quite limited. The goal of this paper is to allow surgeons to see wide-angle images from endoscopy without the drawback of lens distortion. The proposed distortion correction process consists of lens calibration and real-time image warping. The calibration step is to estimate the parameters in the lens distortion model. We propose a fully automatic Hough-entropy-based calibration algorithm, which provides calibration results comparable to the previous manual calibration method. To achieve real-time correction, we use graphics processing unit to warp the image in parallel. In addition, surgeons may adjust the focal length of a lens during the operation. Real-time distortion correction of a zoomable lens is impossible by using traditional calibration methods because the tedious calibration process has to repeat again if focal length is changed. We derive a formula to describe the relationship between the distortion parameter, focal length, and image boundary. Hence, we can estimate the focal length for a zoomable lens from endoscopic images online and achieve real-time lens distortion correction.

This paper presents an advanced augmented reality system for spinal surgery assistance, and develops entry-point guidance prior to vertebroplasty spinal surgery. Based on image-based marker detection and tracking, the proposed camera-projector system superimposes pre-operative 3-D images onto patients. The patients' preoperative 3-D image model is registered by projecting it onto the patient such that the synthetic 3-D model merges with the real patient image, enabling the surgeon to see through the patients' anatomy. The proposed method is much simpler than heavy and computationally challenging navigation systems, and also reduces radiation exposure. The system is experimentally tested on a preoperative 3D model, dummy patient model and animal cadaver model. The feasibility and accuracy of the proposed system is verified on three patients undergoing spinal surgery in the operating theater. The results of these clinical trials are extremely promising, with surgeons reporting favorably on the reduced time of finding a suitable entry point and reduced radiation dose to patients.

In this paper, we present a system for spinal cord and nerves segmentation from STIR-MRI. We propose an user interactive segmentation method for 3D images, which is extended from the 2D random walker algorithm and implemented with a slice-section strategy. After obtaining the 3D segmentation result, we build the 3D spinal cord and nerves model for each view using VTK, which is an open-source, freely available software. Then we obtain the point cloud of the spinal cord and nerves surface by registering the three surface models constructed from three STIR-MRI images of different directions. In the experimental results, we show the 3D segmentation results of spinal cord and nerves from the STIR-MRI (Short Tau Inversion Recovery - Magnetic Resonance Imaging)images in three different views, and also display the reconstructed 3D surface model.

This paper aims to be an outlet for speeding up workflow of orthopedics surgery and be helpful for surgeons to simply coordinate transformations between several imaging displays. Based on epipolar geometry and simple feature landmarks, a 3-D superimposed imaging approach is developed via the construction of the camera-projector system. The superimposed approach is to rectify the perspective, X-ray and projected image pair using the perspective projection model. The proposed method not only simplifies the computation between surgical instruments and patient for surgeons, but also reduces the radiation exposure. Experimental results for both the synthetic spinal image on dummy and real patient testing have demonstrated the feasibility of our approach.

In the paper, we proposed a mass detection method based on texture analysis and neural classifier. The proposed mass detection method is composed of two parts: ROI selection, feature extraction, and neural classifier. ROI selection is used to reduce the computational complexity of the proposed scheme. In the texture analysis, the intensity and texture information extracted from spatial and wavelet domains are utilized to find the candidates of mass regions. These texture features are extracted and combined with a supervised neural network to be classifier. The experimental result shows that the average recall rate of our proposed scheme is more than 93%. The result demonstrates that our proposed method can achieve mass detection.

In this paper, we present a system for segmenting the 3D liver region from CT images and reconstructing its 3D model. The segmentation is accomplished in 3-D space which is extended from the user controlled 2-D random walker technique and implemented by a slice-section method. After obtaining the 3D liver segmentation result, we apply the surface reconstruction algorithm based on bipartite polar classification to build the 3-D liver surface model. In the experimental results, we apply the proposed algorithm to ten test CT datasets to evaluate its accuracy and we demonstrate the good accuracy compared with the human labeled results.