Search

Purpose Automatic identification of interventional devices in X-ray (XR) fluoroscopy offers the potential of improved navigation during transcatheter endovascular procedures. This paper presents a prototype implementation of fully automatic 3D reconstruction of a cryo-balloon catheter during pulmonary vein isolation (PVI) procedures by deep learning approaches. Methods We employ convolutional neural networks (CNN) to automatically identify the cryo-balloon XR marker and catheter shaft in 2D fluoroscopy during PVI. Training data are generated exploiting established semiautomatic techniques, including template-matching and analytical graph building. A first network of U-net architecture uses a single grayscale XR image as input and yields the mask of the XR marker. A second network of the similar architecture is trained using the mask of the XR marker as additional input to the grayscale XR image for the segmentation of the cryo-balloon catheter shaft mask. The structures automatically identified in two 2D images with different angulations are then used to reconstruct the cryo-balloon in 3D. Results Automatic identification of the XR marker was successful in 78% of test cases and in 100% for the catheter shaft. Training of the model for prediction of the XR marker mask was successful with 3426 training samples. Incorporation of the XR marker mask as additional input for the model predicting the catheter shaft allowed to achieve good training result with only 805 training samples. The average prediction time per frame was 14.47 ms for the XR marker and 78.22 ms for the catheter shaft. Localization accuracy for the XR marker yielded on average 1.52 pixels or 0.56 mm. Conclusions In this paper, we report a novel method for automatic detection and 3D reconstruction of the cryo-balloon catheter shaft and marker from 2D fluoroscopic images. Initial evaluation yields promising results thus indicating the high potential of CNNs as alternatives to the current state-of-the-art solutions.

Augmenting X-ray (XR) fluoroscopy with 3D anatomic overlays is an essential technique to improve the guidance of the catheterization procedures. Unfortunately, cardiac and respiratory motion compromises the augmented fluoroscopy. Motion compensation methods can be applied to update the overlay of a static model with regard to respiratory and cardiac motion. We investigate the feasibility of motion detection between two fluoroscopic frames by applying a convolutional neural network (CNN). Its integration in the existing open-source software framework 3D-XGuide is demonstrated, such extending its functionality to automatic motion detection and compensation.The CNN is trained on reference data generated from tracking of the rapid pacing catheter tip by applying template matching with normalized cross-correlation (CC). The developed CNN motion compensation model is packaged in a standalone web service, allowing for independent use via a REST API. For testing and demonstration purposes, we have extended the functionality of 3D-XGuide navigation framework by an additional motion compensation module, which uses the displacement predictions of the standalone CNN model service for motion compensation of the static 3D model overlay. We provide the source code on GitHub under BSD license.The performance of the CNN motion compensation model was evaluated on a total of 1690 fluoroscopic image pairs from ten clinical datasets. The CNN model-based motion compensation method clearly overperformed the tracking of the rapid pacing catheter tip with CC with prediction frame rates suitable for live application in the clinical setting.A novel CNN model-based method for automatic motion compensation during fusion of 3D anatomic models with XR fluoroscopy is introduced and its integration with a real software application demonstrated. Automatic motion extraction from 2D XR images using a CNN model appears as a substantial improvement for reliable augmentation during catheter interventions.

PurposeWith the growing availability and variety of imaging modalities, new methods of intraoperative support have become available for all kinds of interventions. The basic principles of image fusion and image guidance have been widely adopted and are commercialized through a number of platforms. Although multimodal systems have been found to be useful for guiding interventional procedures, they all have their limitations. The integration of more advanced guidance techniques into the product functionality is, however, not easy due to the proprietary solutions of the vendors. Therefore, the purpose of this work is to introduce a software system for image fusion, real-time navigation, and working points documentation during transcatheter interventions performed under X-ray (XR) guidance.MethodsAn interactive software system for cross-modal registration and image fusion of XR fluoroscopy with CT or MRI-derived anatomic 3D models is implemented using Qt application framework and VTK visualization pipeline. DICOM data can be imported in retrospective mode. Live XR data input is realized by a video capture card application interface.ResultsThe actual software release offers a graphical user interface with basic functionality including data import and handling, calculation of projection geometry and transformations between related coordinate systems, rigid 3D-3D registration, and template matching-based tracking and motion compensation algorithms in 2D and 3D. The link to the actual software release on GitHub including source code and executable is provided to support independent research and development in the field of intervention guidance.ConclusionThe introduced system provides a common foundation for the rapid prototyping of new approaches in the field of XR fluoroscopic guidance. As a pure software solution, the developed system is potentially vendor-independent and can be easily extended to be used with the XR systems of different manufacturers.