Search

Image registration is a primary task in various medical image analysis applications. However, cardiac image registration is difficult due to the large non-rigid deformation of the heart and the complex anatomical structure. This paper proposes a structure-aware independently trained multi-scale registration network (SIMReg) to address this challenge. Using image pairs of different resolutions, independently train each registration network to extract image features of large deformation image pairs at different resolutions. In the testing stage, the large deformation registration is decomposed into a multi-scale registration process, and the deformation fields of different resolutions are fused by a step-by-step deformation method, thus solving the difficulty of directly processing large deformation. Meanwhile, the targeted introduction of MIND (modality independent neighborhood descriptor) structural features to guide network training enhances the registration of cardiac structural contours and improves the registration effect of local details. Experiments were carried out on the open cardiac dataset ACDC (automated cardiac diagnosis challenge), and the average Dice value of the experimental results of the proposed method was 0.833. Comparative experiments showed that the proposed SIMReg could better solve the problem of heart image registration and achieve a better registration effect on cardiac images., (© 2024. International Federation for Medical and Biological Engineering.)

To investigate the impact of sagging correction calibration errors in radiotherapy software on image matching. Three software applications were used, with and without a polymethyl methacrylate rod supporting the ball bearings (BB). The calibration error for sagging correction across nine flex maps (FMs) was determined by shifting the BB positions along the Left-Right (LR), Gun-Target (GT), and Up-Down (UD) directions from the reference point. Lucy and pelvic phantom cone-beam computed tomography (CBCT) images underwent auto-matching after modifying each FM. Image deformation was assessed in orthogonal CBCT planes, and the correlations among BB shift magnitude, deformation vector value, and differences in auto-matching were analyzed. The average difference in analysis results among the three softwares for the Winston-Lutz test was within 0.1 mm. The determination coefficients (R 2 ) between the BB shift amount and Lucy phantom matching error in each FM were 0.99, 0.99, and 1.00 in the LR-, GT-, and UD-directions, respectively. The pelvis phantom demonstrated no cross-correlation in the GT direction during auto-matching error evaluation using each FM. The correlation coefficient (r) between the BB shift and the deformation vector value was 0.95 on average for all image planes. Slight differences were observed among software in the evaluation of the Winston-Lutz test. The sagging correction calibration error in the radiotherapy imaging system was caused by an auto-matching error of the phantom and deformation of CBCT images., (© 2024. The Author(s).)

Lung image registration is more challenging than other organs. This is because the breath of the human body causes large deformations in the lung parenchyma and small deformations in tissues such as the pulmonary vascular. Many studies have recently used multi-resolution networks to solve the lung registration problem. However, they use the same structure of registration modules on each level, which makes it difficult to handle complex and small deformations. We propose an unsupervised heterogeneous multi-resolution network (UHMR-Net) to overcome the above problem. The image detail registration module (IDRM) is designed on the highest resolution level. Within this module, the cascaded network is used on the same resolution image to continuously learn the "remaining" detail deformation fields. The shallow shrinkage loss (SS-Loss) is designed to supervise the cascaded network, thus further improving the ability of the network to handle small deformations. Moreover, with the lightweight feature local correlation layer we proposed, the image boundary registration module (IBRM), on multiple low-resolution levels, can better solve the large deformation registration problem. The target registration error on the public DIR-Lab 4DCT dataset was 1.56 ± 1.39 mm, which was significantly better than the classic conventional methods and advanced deep-based methods., (© 2023. International Federation for Medical and Biological Engineering.)

Accurate registration of lung X-rays is an important task in medical image analysis. However, the conventional methods usually cost a lot in running time, and the existing deep learning methods are hard to deal with the large deformation caused by respiratory and cardiac motion. In this paper, we attempt to use deep learning methods to deal with large deformation and enable it to achieve the accuracy of conventional methods. We proposed the cascading affine and B-spline network (CABN), which consists of convolutional cross-stitch affine block (CCAB) and B-splines U-net-like block (BUB) for large lung motion. CCAB makes use of the convolutional cross-stitch model to learn global features among images. And BUB adopts the idea of cubic B-splines which is suitable for large deformation. We separately demonstrated CCAB, BUB, and CABN on two chest X-ray datasets. The experimental results indicate that our methods are highly competitive both in accuracy and runtime when compared to both other deep learning methods and iterative conventional approaches. Moreover, CCAB also can be used for the preprocessing of non-rigid registration methods, replacing affine in conventional methods., (© 2023. The Author(s) under exclusive licence to Society for Imaging Informatics in Medicine.)

Free-breathing external beam radiotherapy remains challenging due to the complex elastic or irregular motion of abdominal organs, as imaging moving organs leads to the creation of motion blurring artifacts. In this paper, we propose a radial-based MRI reconstruction method from 3D free-breathing abdominal data using spatio-temporal geodesic trajectories, to quantify motion during radiotherapy. The prospective study was approved by the institutional review board and consent was obtained from all participants. A total of 25 healthy volunteers, 12 women and 13 men (38 years ± 12 [standard deviation]), and 11 liver cancer patients underwent imaging using a 3.0 T clinical MRI system. The radial acquisition based on golden-angle sparse sampling was performed using a 3D stack-of-stars gradient-echo sequence and reconstructed using a discretized piecewise spatio-temporal trajectory defined in a low-dimensional embedding, which tracks the inhale and exhale phases, allowing the separation between distinct motion phases. Liver displacement between phases as measured with the proposed radial approach based on the deformation vector fields was compared to a navigator-based approach. Images reconstructed with the proposed technique with 20 motion states and registered with the multiscale B-spline approach received on average the highest Likert scores for the overall image quality and visual SNR score 3.2 ± 0.3 (mean ± standard deviation), with liver displacement errors varying between 0.1 and 2.0 mm (mean 0.8 ± 0.6 mm). When compared to navigator-based approaches, the proposed method yields similar deformation vector field magnitudes and angle distributions, and with improved reconstruction accuracy based on mean squared errors. Schematic illustration of the proposed 4D-MRI reconstruction method based on radial golden-angle acquisitions and a respiration motion model from a manifold embedding used for motion tracking. First, data is extracted from the center of k-space using golden-angle sampling, which is then mapped onto a low-dimensional embedding, describing the relationship between neighboring samples in the breathing cycle. The trained model is then used to extract the respiratory motion signal for slice re-ordering. The process then improves the image quality through deformable image registration. Using a reference volume, the deformation vector field (DVF) of sequential motion states are extracted, followed by deformable registrations. The output is a 4DMRI which allows to visualize and quantify motion during free-breathing., (© 2021. International Federation for Medical and Biological Engineering.)

Purpose: Image registration is a fundamental task in the area of image processing, and it is critical to many clinical applications, e.g., computer-assisted surgery. In this work, we attempt to design an effective framework that gains higher accuracy at a minimal cost of the invertibility of registration field., Methods: A hierarchically aggregated transformation (HAT) module is proposed. Within each HAT module, we connect multiple convolutions in a hierarchical manner to capture the multi-scale context, enabling small and large displacements between a pair of images to be taken into account simultaneously during the registration process. Besides, an adaptive feature scaling (AFS) mechanism is presented to refine the multi-scale feature maps derived from the HAT module by rescaling channel-wise features in the global receptive field. Based on the HAT module and AFS mechanism, we establish an efficacious and efficient unsupervised deformable registration framework., Results: The devised framework is validated on the dataset of SCARED and MICCAI Instrument Segmentation and Tracking Challenge 2015, and the experimental results demonstrate that our method achieves better registration accuracy with fewer number of folding pixels than three widely used baseline approaches of SyN, NiftyReg and VoxelMorph., Conclusion: We develop a novel method for unsupervised deformable image registration by incorporating the HAT module and AFS mechanism into the framework, which provides a new way to obtain a desirable registration field between a pair of images., (© 2021. CARS.)

Nonlinear inter-modality registration is often challenging due to the lack of objective functions that are good proxies for alignment. Here we propose a synthesis-by-registration method to convert this problem into an easier intra-modality task. We introduce a registration loss for weakly supervised image translation between domains that does not require perfectly aligned training data. This loss capitalises on a registration U-Net with frozen weights, to drive a synthesis CNN towards the desired translation. We complement this loss with a structure preserving constraint based on contrastive learning, which prevents blurring and content shifts due to overfitting. We apply this method to the registration of histological sections to MRI slices, a key step in 3D histology reconstruction. Results on two public datasets show improvements over registration based on mutual information (13% reduction in landmark error) and synthesis-based algorithms such as CycleGAN (11% reduction), and are comparable to registration with label supervision. Code and data are publicly available at https://github.com/acasamitjana/SynthByReg.

Purpose: Computational surgical planning tools could help develop novel skull base surgical approaches that improve safety and patient outcomes. This defines a need for automated skull base segmentation to improve the usability of surgical planning software. The objective of this work was to design and validate an algorithm for atlas-based automated segmentation of skull base structures in individual image sets for skull base surgical planning., Methods: Advanced Normalization Tools software was used to construct a synthetic CT template from 6 subjects, and skull base structures were manually segmented to create a reference atlas. Landmark registration followed by Elastix deformable registration was applied to the template to register it to each of the 30 trusted reference image sets. Dice coefficient, average Hausdorff distance, and clinical usability scoring were used to compare the atlas segmentations to those of the trusted reference image sets., Results: The mean for average Hausdorff distance for all structures was less than 2 mm (mean for 95th percentile Hausdorff distance was less than 5 mm). For structures greater than 2.5 mL in volume, the average Dice coefficient was 0.73 (range 0.59-0.82), and for structures less than 2.5 mL in volume the Dice coefficient was less than 0.7. The usability scoring survey was completed by three experts, and all structures met the criteria for acceptable effort except for the foramen spinosum, rotundum, and carotid artery, which required more than minor corrections., Conclusion: Currently available open-source algorithms, such as the Elastix deformable algorithm, can be used for automated atlas-based segmentation of skull base structures with acceptable clinical accuracy and minimal corrections with the use of the proposed atlas. The first publicly available CT template and anterior skull base segmentation atlas being released (available at this link: http://hdl.handle.net/1773/46259 ) with this paper will allow for general use of automated atlas-based segmentation of the skull base.

We quantitatively investigate the influence of image registration, using open-source software (3DSlicer), on kinetic analysis (Tofts model) of dynamic contrast enhanced MRI of early-stage breast cancer patients. We also show that registration computation time can be reduced by reducing the percent sampling (PS) of voxels used for estimation of the cost function. DCE-MRI breast images were acquired on a 3T-PET/MRI system in 13 patients with early-stage breast cancer who were scanned in a prone radiotherapy position. Images were registered using a BSpline transformation with a 2 cm isotropic grid at 100, 20, 5, 1, and 0.5PS (BRAINSFit in 3DSlicer). Signal enhancement curves were analyzed voxel-by-voxel using the Tofts kinetic model. Comparing unregistered with registered groups, we found a significant change in the 90th percentile of the voxel-wise distribution of K trans . We also found a significant reduction in the following: (1) in the standard error (uncertainty) of the parameter value estimation, (2) the number of voxel fits providing unphysical values for the extracellular-extravascular volume fraction (v e  > 1), and (3) goodness of fit. We found no significant differences in the median of parameter value distributions (K trans , v e ) between unregistered and registered images. Differences between parameters and uncertainties obtained using 100PS versus 20PS were small and statistically insignificant. As such, computation time can be reduced by a factor of 2, on average, by using 20PS while not affecting the kinetic fit. The methods outlined here are important for studies including a large number of post-contrast images or number of patient images.

Survival of pediatric patients with brain tumor has increased over the past 20 years, and increasing evidence of iatrogenic toxicities has been reported. In follow-ups, images are acquired at different time points where substantial changes of brain morphology occur, due to childhood physiological development and treatment effects. To address the image registration complexity, we propose two multi-metric approaches (M plus , M dot ), combining mutual information (MI) and normalized gradient field filter (NGF). The registration performance of the proposed metrics was assessed on a simulated dataset (Brainweb) and compared with those obtained by MI and NGF separately, using mean magnitude and mean angular errors. The most promising metric (M plus ) was then selected and tested on a retrospective dataset comprising 45 pediatric patients who underwent focal radiotherapy for brain cancer. The quality of the realignment was scored by a radiation oncologist using a perceived misalignment metric (PM). All patients but one were assessed as PM ≤ 2 (good alignment), but the remaining one, severely affected by hydrocephalus and pneumocephalus at the first MRI acquisition, scored PM = 5 (unacceptable). These preliminary findings suggest that M plus might improve the registration accuracy in complex applications such as pediatric oncology, when data are acquired throughout the years of follow-up, and is worth investigating. Graphical abstract Graphical abstract showing the clinical workflow of the overall registration procedure including the three rigid steps, the fourth deformable step, the reference MRI and the registered MRI as well as the contoured ROIs. The registration performance is assessed by means of the Perceived Misalignment score (PM).

Purpose: Clinical examinations that involve endoscopic exploration of the nasal cavity and sinuses often do not have a reference preoperative image, like a computed tomography (CT) scan, to provide structural context to the clinician. The aim of this work is to provide structural context during clinical exploration without requiring additional CT acquisition., Methods: We present a method for registration during clinical endoscopy in the absence of CT scans by making use of shape statistics from past CT scans. Using a deformable registration algorithm that uses these shape statistics along with dense point clouds from video, we simultaneously achieve two goals: (1) register the statistically mean shape of the target anatomy with the video point cloud, and (2) estimate patient shape by deforming the mean shape to fit the video point cloud. Finally, we use statistical tests to assign confidence to the computed registration., Results: We are able to achieve submillimeter errors in registrations and patient shape reconstructions using simulated data. We establish and evaluate the confidence criteria for our registrations using simulated data. Finally, we evaluate our registration method on in vivo clinical data and assign confidence to these registrations using the criteria established in simulation. All registrations that are not rejected by our criteria produce submillimeter residual errors., Conclusion: Our deformable registration method can produce submillimeter registrations and reconstructions as well as statistical scores that can be used to assign confidence to the registrations.

Purpose Video see-through augmented reality (VST-AR) navigation for laparoscopic partial nephrectomy (LPN) can enhance intraoperative perception of surgeons by visualizing surgical targets and critical structures of the kidney tissue. Image registration is the main challenge in the procedure. Existing registration methods in laparoscopic navigation systems suffer from limitations such as manual alignment, invasive external marker fixation, relying on external tracking devices with bulky tracking sensors and lack of deformation compensation. To address these issues, we present a markerless automatic deformable registration framework for LPN VST-AR navigation., Method: Dense stereo matching and 3D reconstruction, automatic segmentation and surface stitching are combined to obtain a larger dense intraoperative point cloud of the renal surface. A coarse-to-fine deformable registration is performed to achieve a precise automatic registration between the intraoperative point cloud and the preoperative model using the iterative closest point algorithm followed by the coherent point drift algorithm. Kidney phantom experiments and in vivo experiments were performed to evaluate the accuracy and effectiveness of our approach., Results: The average segmentation accuracy rate of the automatic segmentation was 94.9%. The mean target registration error of the phantom experiments was found to be 1.28 ± 0.68 mm (root mean square error). In vivo experiments showed that tumor location was identified successfully by superimposing the tumor model on the laparoscopic view., Conclusion: Experimental results have demonstrated that the proposed framework could accurately overlay comprehensive preoperative models on deformable soft organs automatically in a manner of VST-AR without using extra intraoperative imaging modalities and external tracking devices, as well as its potential clinical use.

Purpose: The registration of preoperative 3D images to intra-operative laparoscopic 2D images is one of the main concerns for augmented reality in computer-assisted surgery. For laparoscopic liver surgery, while several algorithms have been proposed, there is neither a public dataset nor a systematic evaluation methodology to quantitatively evaluate registration accuracy., Method: Our main contribution is to provide such a dataset with an in vivo porcine model. It is used to evaluate a state-of-the-art registration algorithm that is capable of simultaneous registration and soft-body collision reasoning., Results: The dataset consists of 13 deformed liver states, with corresponding exploration videos and interventional CT acquisitions with 60 small artificial fiducials located on the surface of the liver and distributed within the parenchyma, where a precise registration is crucial for augmented reality. This dataset will be made public. Using this dataset, we show that collision reasoning improves performance of registration for strong deformation and independent lobe motion., Conclusion: This dataset addresses the lack of public datasets in this field. As an example of use, we present and evaluate a state-of-the-art energy-based approach and a novel extension that handles self-collisions.

Purpose: In interstitial high-dose rate brachytherapy, liver cancer is treated by internal radiation, requiring percutaneous placement of applicators within or close to the tumor. To maximize utility, the optimal applicator configuration is pre-planned on magnetic resonance images. The pre-planned configuration is then implemented via a magnetic resonance-guided intervention. Mapping the pre-planning information onto interventional data would reduce the radiologist's cognitive load during the intervention and could possibly minimize discrepancies between optimally pre-planned and actually placed applicators., Methods: We propose a fast and robust two-step registration framework suitable for interventional settings: first, we utilize a multi-resolution rigid registration to correct for differences in patient positioning (rotation and translation). Second, we employ a novel iterative approach alternating between bias field correction and Markov random field deformable registration in a multi-resolution framework to compensate for non-rigid movements of the liver, the tumors and the organs at risk. In contrast to existing pre-correction methods, our multi-resolution scheme can recover bias field artifacts of different extents at marginal computational costs., Results: We compared our approach to deformable registration via B-splines, demons and the SyN method on 22 registration tasks from eleven patients. Results showed that our approach is more accurate than the contenders for liver as well as for tumor tissues. We yield average liver volume overlaps of 94.0 ± 2.7% and average surface-to-surface distances of 2.02 ± 0.87 mm and 3.55 ± 2.19 mm for liver and tumor tissue, respectively. The reported distances are close to (or even below) the slice spacing (2.5 - 3.0 mm) of our data. Our approach is also the fastest, taking 35.8 ± 12.8 s per task., Conclusion: The presented approach is sufficiently accurate to map information available from brachytherapy pre-planning onto interventional data. It is also reasonably fast, providing a starting point for computer-aidance during intervention.

A 3D-2D image registration method is reported for guiding the placement of surgical devices (e.g., K-wires). The solution registers preoperative CT (and planning data therein) to intraoperative radiographs and computes the pose, shape, and deformation parameters of devices (termed "components") known to be in the radiographic scene. The deformable known-component registration (dKC-Reg) method was applied in experiments emulating spine surgery to register devices (K-wires and spinal fixation rods) undergoing realistic deformation. A two-stage registration process (i) resolves patient pose from individual radiographs and (ii) registers components represented as polygonal meshes based on a B-spline model. The registration result can be visualized as overlay of the component in CT analogous to surgical navigation but without conventional trackers or fiducials. Target registration error in the tip and orientation of deformable K-wires was ( 1.5 ± 0.9 ) mm and 0.6 ∘ ± 0.2 ∘ , respectively. For spinal fixation rods, the registered components achieved Hausdorff distance of 3.4 mm. Future work includes testing in cadaver and clinical data and extension to more generalized deformation and component models.

Purpose: Accurate preoperative planning is crucial for the outcome of total hip arthroplasty. Recently, 2D pelvic X-ray radiographs have been replaced by 3D CT. However, CT suffers from relatively high radiation dosage and cost. An alternative is to reconstruct a 3D patient-specific volume data from 2D X-ray images., Methods: In this paper, based on a fully automatic image segmentation algorithm, we propose a new control point-based 2D-3D registration approach for a deformable registration of a 3D volumetric template to a limited number of 2D calibrated X-ray images and show its application to personalized reconstruction of 3D volumes of the proximal femur. The 2D-3D registration is done with a hierarchical two-stage strategy: the scaled-rigid 2D-3D registration stage followed by a regularized deformable B-spline 2D-3D registration stage. In both stages, a set of control points with uniform spacing are placed over the domain of the 3D volumetric template first. The registration is then driven by computing updated positions of these control points with intensity-based 2D-2D image registrations of the input X-ray images with the associated digitally reconstructed radiographs, which allows computing the associated registration transformation at each stage., Results: Evaluated on datasets of 44 patients, our method achieved an overall surface reconstruction accuracy of [Formula: see text] and an average Dice coefficient of [Formula: see text]. We further investigated the cortical bone region reconstruction accuracy, which is important for planning cementless total hip arthroplasty. An average cortical bone region Dice coefficient of [Formula: see text] and an inner cortical bone surface reconstruction accuracy of [Formula: see text] were found., Conclusions: In summary, we developed a new approach for reconstruction of 3D personalized volumes of the proximal femur from 2D X-ray images. Comprehensive experiments demonstrated the efficacy of the present approach.

Purpose: Computer-assisted intervention often depends on multimodal deformable registration to provide complementary information. However, multimodal deformable registration remains a challenging task., Methods: This paper introduces a novel robust and fast modality-independent 3D binary descriptor, called miLBP, which integrates the principle of local self-similarity with a form of local binary pattern and can robustly extract the similar geometry features from 3D volumes across different modalities. miLBP is a bit string that can be computed by simply thresholding the voxel distance. Furthermore, the descriptor similarity can be evaluated efficiently using the Hamming distance., Results: miLBP was compared to vector-valued self-similarity context (SSC) in artificial image and clinical settings. The results show that miLBP is more robust than SSC in extracting local geometry features across modalities and achieved higher registration accuracy in different registration scenarios. Furthermore, in the most challenging registration between preoperative magnetic resonance imaging and intra-operative ultrasound images, our approach significantly outperforms the state-of-the-art methods in terms of both accuracy ([Formula: see text]) and speed (29.2 s for one case)., Conclusions: Registration performance and speed indicate that miLBP has the potential of being applied to the time-sensitive intra-operative computer-assisted intervention.

This paper presents the development and evaluation of video augmentation on the stereoscopic da Vinci S system with intraoperative image guidance for base of tongue tumor resection in transoral robotic surgery (TORS). Proposed workflow for image-guided TORS begins by identifying and segmenting critical oropharyngeal structures (e.g., the tumor and adjacent arteries and nerves) from preoperative computed tomography (CT) and/or magnetic resonance (MR) imaging. These preoperative planned data can be deformably registered to the intraoperative endoscopic view using mobile C-arm cone-beam computed tomography (CBCT) [1, 2]. Augmentation of TORS endoscopic video defining surgical targets and critical structures has the potential to improve navigation, spatial orientation, and confidence in tumor resection. Experiments in animal specimens achieved statistically significant improvement in target localization error when comparing the proposed image guidance system to simulated current practice.

Objective: A deformable registration technique was developed and evaluated to track and quantify tumor response to radiofrequency ablation for patients with liver malignancies., Materials and Methods: The method uses the combined power of global and local alignment of pre- and post-treatment computed tomography image data sets. The strategy of the algorithm is to infer volumetric deformation based upon surface displacements using a linearly elastic finite element model (FEM). Using this framework, the major challenge for tracking tumor location is not the tissue mechanical properties for FEM modeling but rather the evaluation of boundary conditions. Three different methods were systematically investigated to automatically determine the boundary conditions defined by the correspondences on liver surfaces., Results: Using both 2D synthetic phantoms and imaged 3D beef liver data we performed gold standard registration while measuring the accuracy of non-rigid deformation. The fact that the algorithms could support mean displacement error of tumor deformation up to 2 mm indicates that this technique may serve as a useful tool for surgical interventions. The method was further demonstrated and evaluated using consecutive imaging studies for three liver cancer patients., Conclusion: The FEM-based surface registration technique provides accurate tracking and monitoring of tumor and surrounding tissue during the course of treatment and follow-up.