Search

Objective. In computer-assisted minimally invasive surgery, the intraoperative x-ray image is enhanced by overlapping it with a preoperative CT volume to improve visualization of vital anatomical structures. Therefore, accurate and robust 3D/2D registration of CT volume and x-ray image is highly desired in clinical practices. However, previous registration methods were prone to initial misalignments and struggled with local minima, leading to issues of low accuracy and vulnerability. Approach. To improve registration performance, we propose a novel CT/x-ray image registration agent (CT2X-IRA) within a task-driven deep reinforcement learning framework, which contains three key strategies: (1) a multi-scale-stride learning mechanism provides multi-scale feature representation and flexible action step size, establishing fast and globally optimal convergence of the registration task. (2) A domain adaptation module reduces the domain gap between the x-ray image and digitally reconstructed radiograph projected from the CT volume, decreasing the sensitivity and uncertainty of the similarity measurement. (3) A weighted reward function facilitates CT2X-IRA in searching for the optimal transformation parameters, improving the estimation accuracy of out-of-plane transformation parameters under large initial misalignments. Main results. We evaluate the proposed CT2X-IRA on both the public and private clinical datasets, achieving target registration errors of 2.13 mm and 2.33 mm with the computation time of 1.5 s and 1.1 s, respectively, showing an accurate and fast workflow for CT/x-ray image rigid registration. Significance. The proposed CT2X-IRA obtains the accurate and robust 3D/2D registration of CT and x-ray images, suggesting its potential significance in clinical applications., (© 2023 Institute of Physics and Engineering in Medicine.)

The classic N-wire phantom has been widely used in the calibration of freehand ultrasound probes. One of the main challenges of the phantom is accurately identifying N-fiducials in ultrasound images, especially with multiple N-wire structures. In this study, a method using a multilayer N-wire phantom for the automatic spatial calibration of ultrasound images is proposed. All dots in the ultrasound image are segmented, scored, and classified according to the unique geometric features of the multilayer N-wire phantom. A recognition method for identifying N-fiducials from the dots is proposed for calibrating the spatial transformation of the ultrasound probe. At depths of 9, 11, 13, and 15 cm, the reconstruction error of 50 points is 1.24 ± 0.16, 1.09 ± 0.06, 0.95 ± 0.08, 1.02 ± 0.05 mm, respectively. The reconstruction mockup test shows that the distance accuracy is 1.11 ± 0.82 mm at a depth of 15 cm., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2022 Elsevier B.V. All rights reserved.)

Background and objective Fusion of the preoperative computed tomography angiography and intraoperative X-ray angiography images can considerably enhance the visual perception of physicians during percutaneous coronary interventions. This technique can provide 3D information of the arteries and reduce the uncertainty of 2D guidance images. For this purpose, 3D/2D vascular registration with high accuracy and robustness is crucial for performing accurate surgery. Methods In this study, we propose an iterative closest graph matching (ICGM) method that utilizes an alternative iteration framework including correspondence and transformation phases. A coarse-to-fine matching approach based on redundant graph matching is proposed for the correspondence phase. The transformation phase involves rigid and non-rigid transformations, in which rigid transformation is calculated using a closed-form solution, and non-rigid transformation is achieved using a statistical shape model established from a synthetic deformation dataset. Results The proposed method is evaluated and compared with nine state-of-the-art methods on simulated data and clinical datasets. Experiments demonstrate that our method is insensitive to the pose of data and robust to noise and deformation. Moreover, it outperforms other methods in terms of registering real data. Conclusions Given its high capture range, the proposed method can register 3D vessels without prior initialization in clinical practice., Competing Interests: Declaration of Competing Interest The authors have declared that no conflict of interest., (Copyright © 2020. Published by Elsevier B.V.)

Background and Objective: Feature matching of endoscopic images is of crucial importance in many clinical applications, such as object tracking and surface reconstruction. However, with the presence of low texture, specular reflections and deformations, the feature matching methods of natural scene are facing great challenges in minimally invasive surgery (MIS) scenarios. We propose a novel motion consensus-based method for endoscopic image feature matching to address these problems., Methods: Our method starts by correcting the radial distortion with the spherical projection model and removing the specular reflection regions with an adaptive detection method, which helps to eliminate the image distortion and to reduce the quantity of outliers. We solve the matching problem with a two-stage strategy that progressively estimates a consensus of inliers; the result is a precisely smoothed motion field. First, we construct a spatial motion field from candidate feature matches and estimate its maximum posterior with expectation maximization algorithm, which is computationally efficient and able to obtain smoothed motion field quickly. Second, we extend the smoothed motion field to the affine domain and refine it with bilateral regression to preserve locally subtle motions. The true matches can be identified by checking the difference of feature motion against the estimated field., Results: Evaluations are implemented on two simulation datasets of deformation (218 images) and four different types of endoscopic datasets (1032 images). Our method is compared with three other state-of-the-art methods and achieves the best performance on affine transformation and nonrigid deformation simulations, with inlier ratio of 86.7% and 94.3%, sensitivity of 90.0% and 96.2%, precision of 88.2% and 93.9%, and F1-Score of 89.1% and 95.0%, respectively. On clinical datasets evaluations, the proposed method achieves an average reprojection error of 3.7 pixels and a consistent performance in multi-image correspondence of sequential images. Furthermore, we also present a surface reconstruction result from rhinoscopic images to validate the reliability of our method, which shows high-quality feature matching results., Conclusions: The proposed motion consensus-based feature matching method is proved effective and robust for endoscopic images correspondence. This demonstrates its capability to generate reliable feature matches for surface reconstruction and other meaningful applications in MIS scenarios., Competing Interests: Declaration of Competing Interest The authors declare that there are no conflicts of interest related to this article., (Copyright © 2020 Elsevier B.V. All rights reserved.)

The 3D/2D registration of pre-operative computed tomography angiography (CTA) and intra-operative x-ray angiography (XRA) images in vascular intervention is imperative for guiding surgical instruments and reducing the dosage of toxic contrast agents. In this study, 3D/2D vascular registration is formulated as a search tree problem on the basis of the topological continuity of vessels and the fact that matching can be decomposed into continuous states. In each node of the tree, a closed-solution of 3D/2D transformation is used to obtain the registration results based on the dense correspondences of vessel points, and the results of matching and registration are calculated and recorded. Then, a hand-crafted score that quantifies the qualities of matching and registration of vessels is used, and the remaining problem focuses on finding the highest score in the search tree. An improved heuristic tree search strategy is also proposed to find the best registration. The proposed method is evaluated and compared with four state-of-the-art methods. Experiments on simulated data demonstrate that our method is insensitive to initial pose and robust to noise and deformation. It outperforms other methods in terms of registering real model data and clinical coronary data. In the 3D/2D registration of uninitialized and initialized coronary arteries, the average registration errors are 1.85 and 1.79 mm, respectively. Given that the proposed method is independent of the initial pose, it can be used to navigate vascular intervention for clinical practice.

A graph-based groupwise shape registration algorithm for building statistical shape model (SSM) is proposed, which has been successfully applied to shape prediction of foot scans. Establishing unbiased and effective shape correspondences of large-scale data sets is extremely challenging, for the inappropriate selection of initial mean shape and non-rigid registration of shape with large-scale deformation. To address these issues, first, we use a simplified graph to model the shape distribution in metric space and an edge-guided graph shrinkage to deform the shapes. Then, the groupwise registration is performed by iteratively performing the graph shrinkage until the shape converges. And, the correspondences of training shapes are obtained by propagating the converged shape to the original data along each shrinkage path. Compared with traditional forward and backward models of groupwise registration, the proposed method is data-driven without initial mean shape as input. Moreover, under the constraint of the established graph, the non-rigid registration can perform more accurately by restricting shape register to its neighbors. Based on the shape correspondence, the SSM of foot shapes is constructed and applied to shape prediction by taking the collected anthropometric information as predictor. Experiments demonstrate that the proposed method can obtain robust shape correspondences and SSM capability with respect to model generalization, specificity, and compactness. The application of shape prediction model shows an average prediction error lower than 1% for general foot size. Graphical abstract The graphical abstract of unbiased groupwise registration for foot prediction.

In the diagnosis and interventional treatment of coronary artery disease, the 3D[Formula: see text]time reconstruction of the coronary artery on the basis of x-ray angiographic image sequences can provide dynamic structural information. The synchronization of cardiac phases in the sequences is essential for minimizing the influence of cardiorespiratory motion and realizing precise 3D[Formula: see text]time reconstruction. Key points are initially extracted from the first image of a sequence. Matching grid points between consecutive images in the sequence are extracted by a multi-layer matching strategy. Then deep motion tracking (DMT) of key points is achieved by local deformation based on the neighboring grid points of key points. The local deformation is optimized by the Random sample consensus (RANSAC) algorithm. Then, a simple harmonic motion (SHM) model is utilized to distinguish cardiac motion from other motion sources (e.g. respiratory, patient movement, etc). Next, the signal which is composed of cardiac motions is filtered by a band-pass filter to reconstruct the cardiac phases. Finally, the synchronization of cardiac phases from different imaging angles is realized by a piece-wise linear transformation. The proposed method was evaluated using clinical x-ray angiographic image sequences from 13 patients. [Formula: see text] matching points can be accurately computed by the DMT method. The mean peak temporal distance (MPTD) between the reconstructed cardiac phases and the electrocardiograph signal is [Formula: see text] s. The correlation between the cardiac phases of the same patient is over [Formula: see text]. Compared with three other state-of-the-art methods, the proposed method accurately reconstructs and synchronizes the cardiac phases from different sequences of the same patient. The proposed DMT method is robust and highly effective in synchronizing cardiac phases of angiographic image sequences captured from different imaging angles.

Background and Objective: The development of biometric identification technology and intelligent medication has enabled researchers to analyze subcutaneous veins from near-infrared images. However, the stereo reconstruction of subcutaneous veins has not been well studied, and the results are difficult to utilize in clinical practice., Methods: We present a hybrid constraint optimization (HCO) matching algorithm for vein reconstruction to solve the matching failure problems caused by the incomplete segmentation of vein structures captured from different views. This algorithm initially introduces the existence of the epipolar and homography constraints in the subcutaneous vein matching. Then, the HCO matching algorithm of the vascular centerline is established by homography point-to-point matching, homography matrix optimization, and vascular section matching. Finally, the 3D subcutaneous vein is reconstructed on the basis of the principle of triangulation and system calibration parameters., Results: To validate the performance of the proposed matching method, we designed a series of experiments to evaluate the effectiveness of the hybrid constraint optimization method. The experiments were performed on simulated and real datasets. 42 real vascular images were analyzed on the basis of different matching strategies. Experimental result shows that the matching accuracy increased significantly with the proposed optimization matching method. To calculate the reconstruction accuracy, we reconstructed seven simulated cardboards and measured 10 vascular distances in each simulated cardboard. The average vascular distance error of each simulated image was within 1.0 mm, and the distance errors of 75% feature points were less than 1.5 mm. Also, we printed a 3D simulated vein model to improve the illustration of this system. The reconstruction error extends from -3.58 mm to 1.94 mm with a standard deviation of 0.68 mm and a mean of 0.07 mm., Conclusions: The algorithm is validated in terms of homography optimization, matching efficiency, and simulated vascular reconstruction error. The experimental results demonstrate that the veins captured from the left and right views can be accurately matched through the proposed algorithm., (Copyright © 2018 Elsevier B.V. All rights reserved.)

Statistical shape models are capable of extracting shape prior information, and are usually utilized to assist the task of segmentation of medical images. However, such models require large training datasets in the case of multi-object structures, and it also is difficult to achieve satisfactory results for complex shapes. This study proposed a novel statistical model for cervical vertebra segmentation, called sparse intervertebral fence composition (SiFC), which can reconstruct the boundary between adjacent vertebrae by modeling intervertebral fences. The complex shape of the cervical spine is replaced by a simple intervertebral fence, which considerably reduces the difficulty of cervical segmentation. The final segmentation results are obtained by using a 3D active contour deformation model without shape constraint, which substantially enhances the recognition capability of the proposed method for objects with complex shapes. The proposed segmentation framework is tested on a dataset with CT images from 20 patients. A quantitative comparison against corresponding reference vertebral segmentation yields an overall mean absolute surface distance of 0.70 mm and a dice similarity index of 95.47% for cervical vertebral segmentation. The experimental results show that the SiFC method achieves competitive cervical vertebral segmentation performances, and completely eliminates inter-process overlap.

Background: 3D ultrasound volume reconstruction from B-model ultrasound slices can provide more clearly and intuitive structure of tissue and lesion for the clinician., Methods: This paper proposes a novel Global Path Matching method for the 3D reconstruction of freehand ultrasound images. The proposed method composes of two main steps: bin-filling scheme and hole-filling strategy. For the bin-filling scheme, this study introduces two operators, including the median absolute deviation and the inter-quartile range absolute deviation, to calculate the invariant features of each voxel in the 3D ultrasound volume. And the best contribution range for each voxel is obtained by calculating the Euclidian distance between current voxel and the voxel with the minimum invariant features. Hence, the intensity of the filling vacant voxel can be obtained by weighted combination of the intensity distribution of pixels in the best contribution range. For the hole-filling strategy, three conditions, including the confidence term, the data term and the gradient term, are designed to calculate the weighting coefficient of the matching patch of the vacant voxel. While the matching patch is obtained by finding patches with the best similarity measure that defined by the three conditions in the whole 3D volume data., Results: Compared with VNN, PNN, DW, FMM, BI and KR methods, the proposed Global Path Matching method can restore the 3D ultrasound volume with minimum difference., Conclusions: Experimental results on phantom and clinical data sets demonstrate the effectiveness and robustness of the proposed method for the reconstruction of ultrasound volume.

This study proposes a novel geometrical force constraint method for 3-D vasculature modeling and angiographic image simulation. For this method, space filling force, gravitational force, and topological preserving force are proposed and combined for the optimization of the topology of the vascular structure. The surface covering force and surface adhesion force are constructed to drive the growth of the vasculature on any surface. According to the combination effects of the topological and surface adhering forces, a realistic vasculature can be effectively simulated on any surface. The image projection of the generated 3-D vascular structures is simulated according to the perspective projection and energy attenuation principles of X-rays. Finally, the simulated projection vasculature is fused with a predefined angiographic mask image to generate a realistic angiogram. The proposed method is evaluated on a CT image and three generally utilized surfaces. The results fully demonstrate the effectiveness and robustness of the proposed method.

Three-dimensional (3-D) reconstruction of the coronary artery is important for the diagnosis and interventional treatment of cardiovascular diseases. In this paper, a novel mean composited external force back-projective composition model is proposed and integrated into the deformable model framework for the 3-D reconstruction of coronary arteries from multiple angiograms. The parametric snake evolves toward the real vascular centerline in 3-D space based on the integrated internal energy and composited external energy. In addition, a polynomial function is constructed to determine the diameter of the cross section of the vascular segments, which fully utilizes the back-projection information of multiple angiograms. The deformable and proposed methods are comparatively validated using phantom datasets and routine angiographic images with respect to space and reprojection Euclidean distance errors. The experimental results demonstrate the effectiveness and robustness of the proposed model, which can achieve a mean space error of 0.570 mm and a mean reprojection error of 0.351 mm. In addition, the influence of the angle difference to the reconstruction accuracy is discussed and validated on phantom datasets, which demonstrate that an angle difference of for any two angiograms is suitable for the 3-D reconstruction process.

This study proposes a novel hierarchical pyramid strategy for 3D registration of multimodality medical images. The surfaces of the source and target volume data are first extracted, and the surface point clouds are then aligned roughly using convex hull matching. The convex hull matching registration procedure could align images with large-scale transformations. The original images are divided into blocks and the corresponding blocks in the two images are registered by affine and non-rigid registration procedures. The sub-blocks are iteratively smoothed by the Gaussian kernel with different sizes during the registration procedure. The registration result of the large kernel is taken as the input of the small kernel registration. The fine registration of the two volume data sets is achieved by iteratively increasing the number of blocks, in which increase in similarity measure is taken as a criterion for acceptation of each iteration level. Results demonstrate the effectiveness and robustness of the proposed method in registering the multiple modalities of medical images.