Search

Producing spatial transformations that are diffeomorphic is a key goal in deformable image registration. As a diffeomorphic transformation should have positive Jacobian determinant | J | everywhere, the number of pixels (2D) or voxels (3D) with | J | < 0 has been used to test for diffeomorphism and also to measure the irregularity of the transformation. For digital transformations, | J | is commonly approximated using a central difference, but this strategy can yield positive | J | 's for transformations that are clearly not diffeomorphic—even at the pixel or voxel resolution level. To show this, we first investigate the geometric meaning of different finite difference approximations of | J | . We show that to determine if a deformation is diffeomorphic for digital images, the use of any individual finite difference approximation of | J | is insufficient. We further demonstrate that for a 2D transformation, four unique finite difference approximations of | J | 's must be positive to ensure that the entire domain is invertible and free of folding at the pixel level. For a 3D transformation, ten unique finite differences approximations of | J | 's are required to be positive. Our proposed digital diffeomorphism criteria solves several errors inherent in the central difference approximation of | J | and accurately detects non-diffeomorphic digital transformations. The source code of this work is available at https://github.com/yihao6/digital_diffeomorphism. [ABSTRACT FROM AUTHOR]

Background: Image registration is a challenging problem in many clinical tasks, but deep learning has made significant progress in this area over the past few years. Real‐time and robust registration has been made possible by supervised transformation estimation. However, the quality of registrations using this framework depends on the quality of ground truth labels such as displacement field. Purpose: To propose a simple and reliable method for registering medical images based on image structure similarity in a completely unsupervised manner. Methods: We proposed a deep cascade unsupervised deformable registration approach to align images without reliable clinical data labels. Our basic network was composed of a displacement estimation module (ResUnet) and a deformation module (spatial transformer layers). We adopted l2$l_2$‐norm to regularize the deformation field instead of the traditional l1$l_1$‐norm regularization. Additionally, we utilized structural similarity (ssim) estimation during the training stage to enhance the structural consistency between the deformed images and the reference images. Results: Experiments results indicated that by incorporating ssim loss, our cascaded methods not only achieved higher dice score of 0.9873, ssim score of 0.9559, normalized cross‐correlation (NCC) score of 0.9950, and lower relative sum of squared difference (SSD) error of 0.0313 on CT images, but also outperformed the comparative methods on ultrasound dataset. The statistical t$t$‐test results also proved that these improvements of our method have statistical significance. Conclusions: In this study, the promising results based on diverse evaluation metrics have demonstrated that our model is simple and effective in deformable image registration (DIR). The generalization ability of the model was also verified through experiments on liver CT images and cardiac ultrasound images. [ABSTRACT FROM AUTHOR]

Multi-modal histological image registration tasks pose significant challenges due to tissue staining operations causing partial loss and folding of tissue. Convolutional neural network (CNN) and generative adversarial network (GAN) are pivotal in medical image registration. However, existing methods often struggle with severe interference and deformation, as seen in histological images of conditions like Cushing's disease. We argue that the failure of current approaches lies in underutilizing the feature extraction capability of the discriminator in GAN. In this study, we propose a novel multi-modal registration approach GAN-DIRNet based on GAN for deformable histological image registration. To begin with, the discriminators of two GANs are embedded as a new dual parallel feature extraction module into the unsupervised registration networks, characterized by implicitly extracting feature descriptors of specific modalities. Additionally, modal feature description layers and registration layers collaborate in unsupervised optimization, facilitating faster convergence and more precise results. Lastly, experiments and evaluations were conducted on the registration of the Mixed National Institute of Standards and Technology database (MNIST), eight publicly available datasets of histological sections and the Clustering-Registration-Classification-Segmentation (CRCS) dataset on the Cushing's disease. Experimental results demonstrate that our proposed GAN-DIRNet method surpasses existing approaches like DIRNet in terms of both registration accuracy and time efficiency, while also exhibiting robustness across different image types. [ABSTRACT FROM AUTHOR]

Image registration is a primary task in various medical image analysis applications. However, cardiac image registration is difficult due to the large non-rigid deformation of the heart and the complex anatomical structure. This paper proposes a structure-aware independently trained multi-scale registration network (SIMReg) to address this challenge. Using image pairs of different resolutions, independently train each registration network to extract image features of large deformation image pairs at different resolutions. In the testing stage, the large deformation registration is decomposed into a multi-scale registration process, and the deformation fields of different resolutions are fused by a step-by-step deformation method, thus solving the difficulty of directly processing large deformation. Meanwhile, the targeted introduction of MIND (modality independent neighborhood descriptor) structural features to guide network training enhances the registration of cardiac structural contours and improves the registration effect of local details. Experiments were carried out on the open cardiac dataset ACDC (automated cardiac diagnosis challenge), and the average Dice value of the experimental results of the proposed method was 0.833. Comparative experiments showed that the proposed SIMReg could better solve the problem of heart image registration and achieve a better registration effect on cardiac images. [ABSTRACT FROM AUTHOR]

To investigate the impact of sagging correction calibration errors in radiotherapy software on image matching. Three software applications were used, with and without a polymethyl methacrylate rod supporting the ball bearings (BB). The calibration error for sagging correction across nine flex maps (FMs) was determined by shifting the BB positions along the Left–Right (LR), Gun–Target (GT), and Up–Down (UD) directions from the reference point. Lucy and pelvic phantom cone-beam computed tomography (CBCT) images underwent auto-matching after modifying each FM. Image deformation was assessed in orthogonal CBCT planes, and the correlations among BB shift magnitude, deformation vector value, and differences in auto-matching were analyzed. The average difference in analysis results among the three softwares for the Winston–Lutz test was within 0.1 mm. The determination coefficients (R2) between the BB shift amount and Lucy phantom matching error in each FM were 0.99, 0.99, and 1.00 in the LR-, GT-, and UD-directions, respectively. The pelvis phantom demonstrated no cross-correlation in the GT direction during auto-matching error evaluation using each FM. The correlation coefficient (r) between the BB shift and the deformation vector value was 0.95 on average for all image planes. Slight differences were observed among software in the evaluation of the Winston–Lutz test. The sagging correction calibration error in the radiotherapy imaging system was caused by an auto-matching error of the phantom and deformation of CBCT images. [ABSTRACT FROM AUTHOR]

Background Purpose Methods Results Conclusion Deformable registration is required to generate a time‐integrated activity (TIA) map which is essential for voxel‐based dosimetry. The conventional iterative registration algorithm using anatomical images (e.g., computed tomography (CT)) could result in registration errors in functional images (e.g., single photon emission computed tomography (SPECT) or positron emission tomography (PET)). Various deep learning‐based registration tools have been proposed, but studies specifically focused on the registration of serial hybrid images were not found.In this study, we introduce CoRX‐NET, a novel unsupervised deep learning network designed for deformable registration of hybrid medical images. The CoRX‐NET structure is based on the Swin‐transformer (ST), allowing for the representation of complex spatial connections in images. Its self‐attention mechanism aids in the effective exchange and integration of information across diverse image regions. To augment the amalgamation of SPECT and CT features, cross‐stitch layers have been integrated into the network.Two different 177 Lu DOTATATE SPECT/CT datasets were acquired at different medical centers. 22 sets from Seoul National University and 14 sets from Sunway Medical Centre are used for training/internal validation and external validation respectively. The CoRX‐NET architecture builds upon the ST, enabling the modeling of intricate spatial relationships within images. To further enhance the fusion of SPECT and CT features, cross‐stitch layers have been incorporated within the network. The network takes a pair of SPECT/CT images (e.g., fixed and moving images) and generates a deformed SPECT/CT image. The performance of the network was compared with Elastix and TransMorph using L1 loss and structural similarity index measure (SSIM) of CT, SSIM of normalized SPECT, and local normalized cross correlation (LNCC) of SPECT as metrics. The voxel‐wise root mean square errors (RMSE) of TIA were compared among the different methods.The ablation study revealed that cross‐stitch layers improved SPECT/CT registration performance. The cross‐stitch layers notably enhance SSIM (internal validation: 0.9614 vs. 0.9653, external validation: 0.9159 vs. 0.9189) and LNCC of normalized SPECT images (internal validation: 0.7512 vs. 0.7670, external validation: 0.8027 vs. 0.8027). CoRX‐NET with the cross‐stitch layer achieved superior performance metrics compared to Elastix and TransMorph, except for CT SSIM in the external dataset. When qualitatively analyzed for both internal and external validation cases, CoRX‐NET consistently demonstrated superior SPECT registration results. In addition, CoRX‐NET accomplished SPECT/CT image registration in less than 6 s, whereas Elastix required approximately 50 s using the same PC's CPU. When employing CoRX‐NET, it was observed that the voxel‐wise RMSE values for TIA were approximately 27% lower for the kidney and 33% lower for the tumor, compared to when Elastix was used.This study represents a major advancement in achieving precise SPECT/CT registration using an unsupervised deep learning network. It outperforms conventional methods like Elastix and TransMorph, reducing uncertainties in TIA maps for more accurate dose assessments. [ABSTRACT FROM AUTHOR]

In recent years, deep learning has ushered in significant development in medical image registration, and the method of non-rigid registration using deep neural networks to generate a deformation field has higher accuracy. However, unlike monomodal medical image registration, multimodal medical image registration is a more complex and challenging task. This paper proposes a new linear-to-nonlinear framework (L2NLF) for multimodal medical image registration. The first linear stage is essentially image conversion, which can reduce the difference between two images without changing the authenticity of medical images, thus transforming multimodal registration into monomodal registration. The second nonlinear stage is essentially unsupervised deformable registration based on the deep neural network. In this paper, a brand-new registration network, CrossMorph, is designed, a deep neural network similar to the U-net structure. As the backbone of the encoder, the volume CrossFormer block can better extract local and global information. Booster promotes the reduction of more deep features and shallow features. The qualitative and quantitative experimental results on T1 and T2 data of 240 patients' brains show that L2NLF can achieve excellent registration effect in the image conversion part with very low computation, and it will not change the authenticity of the converted image at all. Compared with the current state-of-the-art registration method, CrossMorph can effectively reduce average surface distance, improve dice score, and improve the deformation field's smoothness. The proposed methods have potential value in clinical application. [ABSTRACT FROM AUTHOR]

Abstract Background To evaluate a novel CBCT conversion algorithm for dose calculation implemented in a research version of a treatment planning system (TPS). Methods The algorithm was implemented in a research version of RayStation (v. 11B-DTK, RaySearch, Stockholm, Sweden). CBCTs acquired for each ten head and neck (HN), gynecology (GYN) and lung cancer (LNG) patients were collected and converted using the new algorithm (CBCTc). A bulk density overriding technique implemented in the same version of the TPS was used for comparison (CBCTb). A deformed CT (dCT) was created by using deformable image registration of the planning CT (pCT) to the CBCT to reduce anatomical changes. All treatment plans were recalculated on the pCT, dCT, CBCTc and the CBCTb. The resulting dose distributions were analyzed using the MICE toolkit (NONPIMedical AB Sweden, Umeå) with local gamma analysis, with 1% dose difference and 1 mm distance to agreement criteria. A Wilcoxon paired rank sum test was applied to test the differences in gamma pass rates (GPRs). A p value smaller than 0.05 considered statistically significant. Results The GPRs for the CBCTb method were systematically lower compared to the CBCTc method. Using the 10% dose threshold and the dCT as reference the median GPRs were for the CBCTc method were 100% and 99.8% for the HN and GYN cases, respectively. Compared to that the GPRs of the CBCTb method were lower with values of 99.8% and 98.0%, for the HN and GYN cases, respectively. The GPRs of the LNG cases were 99.9% and 97.5% for the CBCTc and CBCTb method, respectively. These differences were statistically significant. The main differences between the dose calculated on the CBCTs and the pCTs were found in regions near air/tissue interfaces, which are also subject to anatomical variations. Conclusion The dose distribution calculated using the new CBCTc method showed excellent agreement with the dose calculated using dCT and pCT and was superior to the CBCTb method. The main reasons for deviations of the calculated dose distribution were caused by anatomical variations between the pCT and the corrected CBCT.

Background: To evaluate a novel CBCT conversion algorithm for dose calculation implemented in a research version of a treatment planning system (TPS). Methods: The algorithm was implemented in a research version of RayStation (v. 11B-DTK, RaySearch, Stockholm, Sweden). CBCTs acquired for each ten head and neck (HN), gynecology (GYN) and lung cancer (LNG) patients were collected and converted using the new algorithm (CBCTc). A bulk density overriding technique implemented in the same version of the TPS was used for comparison (CBCTb). A deformed CT (dCT) was created by using deformable image registration of the planning CT (pCT) to the CBCT to reduce anatomical changes. All treatment plans were recalculated on the pCT, dCT, CBCTc and the CBCTb. The resulting dose distributions were analyzed using the MICE toolkit (NONPIMedical AB Sweden, Umeå) with local gamma analysis, with 1% dose difference and 1 mm distance to agreement criteria. A Wilcoxon paired rank sum test was applied to test the differences in gamma pass rates (GPRs). A p value smaller than 0.05 considered statistically significant. Results: The GPRs for the CBCTb method were systematically lower compared to the CBCTc method. Using the 10% dose threshold and the dCT as reference the median GPRs were for the CBCTc method were 100% and 99.8% for the HN and GYN cases, respectively. Compared to that the GPRs of the CBCTb method were lower with values of 99.8% and 98.0%, for the HN and GYN cases, respectively. The GPRs of the LNG cases were 99.9% and 97.5% for the CBCTc and CBCTb method, respectively. These differences were statistically significant. The main differences between the dose calculated on the CBCTs and the pCTs were found in regions near air/tissue interfaces, which are also subject to anatomical variations. Conclusion: The dose distribution calculated using the new CBCTc method showed excellent agreement with the dose calculated using dCT and pCT and was superior to the CBCTb method. The main reasons for deviations of the calculated dose distribution were caused by anatomical variations between the pCT and the corrected CBCT. [ABSTRACT FROM AUTHOR]

PurposeTo develop a convolutional neural network (CNN)-based deformable lung registration algorithm to reduce computation time and assess its potential for lobar air trapping quantification.Materials and methodsIn this retrospective study, a CNN algorithm was developed to perform deformable registration of lung CT (LungReg) using data on 9118 patients from the COPDGene Study (data collected between 2007 and 2012). Loss function constraints included cross-correlation, displacement field regularization, lobar segmentation overlap, and the Jacobian determinant. LungReg was compared with a standard diffeomorphic registration (SyN) for lobar Dice overlap, percentage voxels with nonpositive Jacobian determinants, and inference runtime using paired t tests. Landmark colocalization error (LCE) across 10 patients was compared using a random effects model. Agreement between LungReg and SyN air trapping measurements was assessed using intraclass correlation coefficient. The ability of LungReg versus SyN emphysema and air trapping measurements to predict Global Initiative for Chronic Obstructive Lung Disease (GOLD) stages was compared using area under the receiver operating characteristic curves.ResultsAverage performance of LungReg versus SyN showed lobar Dice overlap score of 0.91-0.97 versus 0.89-0.95, respectively (P < .001); percentage voxels with nonpositive Jacobian determinant of 0.04 versus 0.10, respectively (P < .001); inference run time of 0.99 second (graphics processing unit) and 2.27 seconds (central processing unit) versus 418.46 seconds (central processing unit) (P < .001); and LCE of 7.21 mm versus 6.93 mm (P < .001). LungReg and SyN whole-lung and lobar air trapping measurements achieved excellent agreement (intraclass correlation coefficients > 0.98). LungReg versus SyN area under the receiver operating characteristic curves for predicting GOLD stage were not statistically different (range, 0.88-0.95 vs 0.88-0.95, respectively; P = .31-.95).ConclusionCNN-based deformable lung registration is accurate and fully automated, with runtime feasible for clinical lobar air trapping quantification, and has potential to improve diagnosis of small airway diseases.Keywords: Air Trapping, Convolutional Neural Network, Deformable Registration, Small Airway Disease, CT, Lung, Semisupervised Learning, Unsupervised Learning Supplemental material is available for this article. © RSNA, 2021 An earlier incorrect version of this article appeared online. This article was corrected on December 22, 2021.

Abstract Deformable multimodal image registration plays a key role in medical image analysis. It remains a challenge to find accurate dense correspondences between multimodal images due to the significant intensity distortion and the large deformation. macJNet is proposed to align the multimodal medical images, which is a weakly-supervised multimodal image deformable registration method using a joint learning framework and multi-sampling cascaded modality independent neighborhood descriptor (macMIND). The joint learning framework consists of a multimodal image registration network and two segmentation networks. The proposed macMIND is a modality-independent image structure descriptor to provide dense correspondence for registration, which incorporates multi-orientation and multi-scale sampling patterns to build self-similarity context. It greatly enhances the representation ability of cross-modal features in the registration network. The semi-supervised segmentation networks generate anatomical labels to provide semantics correspondence for registration, and the registration network helps to improve the performance of multimodal image segmentation by providing the consistency of anatomical labels. 3D CT-MR liver image dataset with 118 samples is built for evaluation, and comprehensive experiments have been conducted to demonstrate that macJNet achieves superior performance over state-of-the-art multi-modality medical image registration methods.

PurposeMissing or discrepant imaging volume is a common challenge in deformable image registration (DIR). To minimize the adverse impact, we train a neural network to synthesize cropped portions of head and neck CT's and then test its use in DIR.MethodsUsing a training dataset of 409 head and neck CT's, we trained a generative adversarial network to take in a cropped 3D image and output an image with synthesized anatomy in the cropped region. The network used a 3D U-Net generator along with Visual Geometry Group (VGG) deep feature losses. To test our technique, for each of the 53 test volumes, we used Elastix to deformably register combinations of a randomly cropped, full, and synthetically full volume to a single cropped, full, and synthetically full target volume. We additionally tested our method's robustness to crop extent by progressively increasing the amount of cropping, synthesizing the missing anatomy using our network, and then performing the same registration combinations. Registration performance was measured using 95% Hausdorff distance across 16 contours.ResultsWe successfully trained a network to synthesize missing anatomy in superiorly and inferiorly cropped images. The network can estimate large regions in an incomplete image, far from the cropping boundary. Registration using our estimated full images was not significantly different from registration using the original full images. The average contour matching error for full image registration was 9.9 mm, whereas our method was 11.6, 12.1, and 13.6 mm for synthesized-to-full, full-to-synthesized, and synthesized-to-synthesized registrations, respectively. In comparison, registration using the cropped images had errors of 31.7 mm and higher. Plotting the registered image contour error as a function of initial preregistered error shows that our method is robust to registration difficulty. Synthesized-to-full registration was statistically independent of cropping extent up to 18.7 cm superiorly cropped. Synthesized-to-synthesized registration was nearly independent, with a -0.04 mm of change in average contour error for every additional millimeter of cropping.ConclusionsDifferent or inadequate in scan extent is a major cause of DIR inaccuracies. We address this challenge by training a neural network to complete cropped 3D images. We show that with image completion, the source of DIR inaccuracy is eliminated, and the method is robust to varying crop extent.

Deformable multimodal image registration plays a key role in medical image analysis. It remains a challenge to find accurate dense correspondences between multimodal images due to the significant intensity distortion and the large deformation. macJNet is proposed to align the multimodal medical images, which is a weakly-supervised multimodal image deformable registration method using a joint learning framework and multi-sampling cascaded modality independent neighborhood descriptor (macMIND). The joint learning framework consists of a multimodal image registration network and two segmentation networks. The proposed macMIND is a modality-independent image structure descriptor to provide dense correspondence for registration, which incorporates multi-orientation and multi-scale sampling patterns to build self-similarity context. It greatly enhances the representation ability of cross-modal features in the registration network. The semi-supervised segmentation networks generate anatomical labels to provide semantics correspondence for registration, and the registration network helps to improve the performance of multimodal image segmentation by providing the consistency of anatomical labels. 3D CT-MR liver image dataset with 118 samples is built for evaluation, and comprehensive experiments have been conducted to demonstrate that macJNet achieves superior performance over state-of-the-art multi-modality medical image registration methods. [ABSTRACT FROM AUTHOR]

Lung image registration is more challenging than other organs. This is because the breath of the human body causes large deformations in the lung parenchyma and small deformations in tissues such as the pulmonary vascular. Many studies have recently used multi-resolution networks to solve the lung registration problem. However, they use the same structure of registration modules on each level, which makes it difficult to handle complex and small deformations. We propose an unsupervised heterogeneous multi-resolution network (UHMR-Net) to overcome the above problem. The image detail registration module (IDRM) is designed on the highest resolution level. Within this module, the cascaded network is used on the same resolution image to continuously learn the "remaining" detail deformation fields. The shallow shrinkage loss (SS-Loss) is designed to supervise the cascaded network, thus further improving the ability of the network to handle small deformations. Moreover, with the lightweight feature local correlation layer we proposed, the image boundary registration module (IBRM), on multiple low-resolution levels, can better solve the large deformation registration problem. The target registration error on the public DIR-Lab 4DCT dataset was 1.56 ± 1.39 mm, which was significantly better than the classic conventional methods and advanced deep-based methods. [ABSTRACT FROM AUTHOR]

This study was to investigate the new analysis manner of dental hard tissue change using in vivo micro-computed tomography (CT) in rat. Scanning, registration, analyzing, and presenting method to track longitudinal in vivo micro-CT data on dental hard tissues were validated in murine models: formative, dentin thickness after direct pulp capping with mineral trioxide aggregate; resorptive, development of apical bone rarefaction in apical periodontitis model. Serial in vivo micro-CT scans were analyzed through rigidregistration, active-contouring, deformable-registration, and motion vector-based quantitative analyses. The rate and direction of hard tissue formation after direct pulp capping was datafied by tracing coordinate shift of fiducial points on pulp chamber outline in formative model. The development of apical periodontitis could be monitored with voxel counts, and quantitatively analyzed in terms of lesion size, bone loss, and mineral density in resorptive model. This study supports the application of longitudinal in vivo micro-CT for resorptive- and formative-phase specific monitoring of dental hard tissues. [ABSTRACT FROM AUTHOR]

Purpose: To evaluate deep learning (DL)‐based deformable image registration (DIR) for dose accumulation during radiotherapy of prostate cancer patients. Methods and Materials: Data including 341 CBCTs (209 daily, 132 weekly) and 23 planning CTs from 23 patients was retrospectively analyzed. Anatomical deformation during treatment was estimated using free‐form deformation (FFD) method from Elastix and DL‐based VoxelMorph approaches. The VoxelMorph method was investigated using anatomical scans (VMorph_Sc) or label images (VMorph_Msk), or the combination of both (VMorph_Sc_Msk). Accumulated doses were compared with the planning dose. Results: The DSC ranges, averaged for prostate, rectum and bladder, were 0.60–0.71, 0.67–0.79, 0.93–0.98, and 0.89–0.96 for the FFD, VMorph_Sc, VMorph_Msk, and VMorph_Sc_Msk methods, respectively. When including both anatomical and label images, VoxelMorph estimated more complex deformations resulting in heterogeneous determinant of Jacobian and higher percentage of deformation vector field (DVF) folding (up to a mean value of 1.90% in the prostate). Large differences were observed between DL‐based methods regarding estimation of the accumulated dose, showing systematic overdosage and underdosage of the bladder and rectum, respectively. The difference between planned mean dose and accumulated mean dose with VMorph_Sc_Msk reached a median value of +6.3 Gy for the bladder and −5.1 Gy for the rectum. Conclusion: The estimation of the deformations using DL‐based approach is feasible for male pelvic anatomy but requires the inclusion of anatomical contours to improve organ correspondence. High variability in the estimation of the accumulated dose depending on the deformable strategy suggests further investigation of DL‐based techniques before clinical deployment. [ABSTRACT FROM AUTHOR]

Deformable image registration plays a crucial role in estimating cardiac deformation from a sequence of images. However, existing registration methods primarily process images as pairs instead of processing all images in a sequence together. This study proposes a novel end-to-end learning-free generative multi-resolution convolutional neural network (GMCNet) with the primary focus of registering images in a sequence. Even though learning-based methods have yielded high performance for image registration, their performance depends on their ability to learn information from a large number of samples which are difficult to obtain and might bias the framework to the specific domain of data. The proposed learning-free method eliminates the need for a dedicated training set while exploiting the capabilities of neural networks to achieve accurate deformation fields. Due to its capability of parameter sharing through the architecture, the GMCNet can be used as a groupwise registration as well as pairwise registration. The proposed method was evaluated on three different clinical cardiac magnetic resonance imaging datasets and compared quantitatively against nine other state-of-the-art learning and optimization-based algorithms. The proposed method outperformed other methods in all comparisons and yielded average Dice metric values ranging from 0.85 to 0.88 for the datasets. Different aspects of the GMCNet are also explored by assessing 1) the robustness; 2) performance on pairwise registration; 3) the influence of spatial transformation in a controlled environment; and 4) the impact of different multi-resolution structures. The results demonstrate that using temporal information to estimate the deformation fields leads to more accurate registration results and improved robustness under different noise levels. Moreover, the proposed method does not need images for training, and therefore, its prediction is not domain-specific and can be applied to any sequence of images.

3d reconstruction of deformable objects in dynamic scenes forms the fundamental basis of many robotic applications. Existing mesh-based approaches compromise registration accuracy, and lose important details due to interpolation and smoothing. Additionally, existing non-rigid registration techniques struggle with unindexed points and disconnected manifolds. We propose a novel non-rigid registration framework for raw, unstructured, deformable point clouds purely based on geometric features. The global non-rigid deformation of an object is formulated as an aggregation of locally rigid transformations. The concept of locality is embodied in soft patches described by geometrical properties based on SHOT descriptor and its neighborhood. By considering the confidence score of pairwise association between soft patches of two scans (not necessarily consecutive), a computed similarity matrix serves as the seed to grow a correspondence graph which leverages rigidity terms defined in As-Rigid-As-Possible for pruning and optimization. Experiments on simulated and publicly available datasets demonstrate the capability of the proposed approach to cope with large deformations blended with numerous missing parts in the scan process.

Accurate registration of lung X-rays is an important task in medical image analysis. However, the conventional methods usually cost a lot in running time, and the existing deep learning methods are hard to deal with the large deformation caused by respiratory and cardiac motion. In this paper, we attempt to use deep learning methods to deal with large deformation and enable it to achieve the accuracy of conventional methods. We proposed the cascading affine and B-spline network (CABN), which consists of convolutional cross-stitch affine block (CCAB) and B-splines U-net-like block (BUB) for large lung motion. CCAB makes use of the convolutional cross-stitch model to learn global features among images. And BUB adopts the idea of cubic B-splines which is suitable for large deformation. We separately demonstrated CCAB, BUB, and CABN on two chest X-ray datasets. The experimental results indicate that our methods are highly competitive both in accuracy and runtime when compared to both other deep learning methods and iterative conventional approaches. Moreover, CCAB also can be used for the preprocessing of non-rigid registration methods, replacing affine in conventional methods. [ABSTRACT FROM AUTHOR]

To address the low precision of pairwise registration method based on the deep learning and the time-consuming nature of traditional registration algorithm, this paper presents a method of unsupervised end-to-end groupwise registration based on variational inference, as well as a registration framework based on normalized cross correlation (NCC) and prior knowledge. The framework can warp all images in the group into a common space and effectively control the deformation field of the regularization, and it doesn't need a real deformation field or a reference image. The estimation of deformation field by this method can be modeled as a probability generation model and solved by variational inference. Then unsupervised training is implemented with the help of spatial transformer network and loss function. The registration results of 3D brain magnetic resonance image from the public data set LPBA40 show that: compared with the baseline method, the proposed method has better Dice score, less running time, better diffeomorphisms domain, and is robust to noise.

Introduction: This study aimed to assess and compare the capabilities of two commercially available deformable image registration algorithms implemented in Raystation 9A (A1) and Velocity AI (A2) for possible usage in adaptive prostate radiotherapy based on the propagation of anatomical contours from computed tomography (CT) images to cone-beam CT (CBCT) images. Material and methods: Ten patients were retrospectively selected from a group treated for localized prostate cancer. The propagation of rectum contours was analyzed in a set of CT-CBCT pairs. Two independent observers carried out qualitative analysis using the two-level descriptive scale (meet/fail). Quantitative analysis was done using landmark points distances based on implanted markers as navigation points and differently obtained contours (manually and automatically using DIR algorithms). Quantitative analysis was taken on sets preselected by qualitative analysis. Results: Qualitative analysis shows that 83.7% of the rectum contours were scored identically (meet or fail) for both algorithms, from which 53.5% and 55.8% are failed results for A1 and A2, respectively. For the rectum size (RWD parameter), differences between referenced and deformation-based values were 5.5 and 5.8 mm, and for the rectum wall, the prostate marker distance (WMD parameter) was 4.5 and 5.5 mm for A1 and A2, respectively. The differences between the WMD parameters were statistically significant (p = 0.007). Conclusions: In both tested algorithms, neither effectiveness nor measured uncertainties in the propagation of rectum contour process in prostate patient cases were satisfactory. Careful selection of input images followed by case/set-based verification of every deformable registration is a substantial step to avoid inappropriate results.

Organ shape reconstruction based on a single-projection image during treatment has wide clinical scope, e.g., in image-guided radiotherapy and surgical guidance. We propose an image-to-graph convolutional network that achieves deformable registration of a three-dimensional (3D) organ mesh for a low-contrast two-dimensional (2D) projection image. This framework enables simultaneous training of two types of transformation: from the 2D projection image to a displacement map, and from the sampled per-vertex feature to a 3D displacement that satisfies the geometrical constraint of the mesh structure. Assuming application to radiation therapy, the 2D/3D deformable registration performance is verified for multiple abdominal organs that have not been targeted to date, i.e., the liver, stomach, duodenum, and kidney, and for pancreatic cancer. The experimental results show shape prediction considering relationships among multiple organs can be used to predict respiratory motion and deformation from digitally reconstructed radiographs with clinically acceptable accuracy. [ABSTRACT FROM AUTHOR]

Purpose: This investigation compares three commercial methods of cone‑beam computed tomography (CBCT)‑based dosimetric analysis to a method based on repeat computed tomography (CT). Materials and Methods: Seventeen head‑and‑neck patients treated in 2020, and with a repeat CT, were included in the analyses. The planning CT was deformed to anatomy in repeat CT to generate a reference plan. Two of the CBCT‑based methods generated test plans by deforming the planning CT to CBCT of fraction N using VelocityAI™ and SmartAdapt®. The third method compared directly calculated doses on the CBCT for fraction 1 and fraction N, using PerFraction™. Maximum dose to spinal cord (Cord_dmax) and dose to 95% volume (D95) of planning target volumes (PTVs) were used to assess “need to replan” criteria. Results: The VelocityAI™ method provided results that most accurately matched the reference plan in “need to replan” criteria using either Cord_dmax or PTV D95. SmartAdapt® method overestimated the change in Cord_dmax (6.77% vs. 3.85%, P < 0.01) and change in cord volume (9.56% vs. 0.67%, P < 0.01) resulting in increased false positives in “need to replan” criteria, and performed similarly to VelocityAI™ for D95, but yielded more false negatives. PerFraction™ method underestimated Cord_dmax, did not perform any volume deformation, and missed all “need to replan” cases based on cord dose. It also yielded high false negatives using the D95 PTV criteria. Conclusions: The VelocityAI™‑based method using fraction N CBCT is most similar to the reference plan using repeat CT; the other two methods had significant differences. [ABSTRACT FROM AUTHOR]

To determine the relationship between local relapse following radical radiotherapy for muscle-invasive bladder cancer (MIBC) and radiation dose. Patients with T2-4N0-3M0 MIBC were recruited to a phase II study assessing the feasibility of intensity-modulated radiotherapy to the bladder and pelvic lymph nodes. Patients were planned to receive 64 Gy/32 fractions to the bladder tumour, 60 Gy/32 fractions to the involved pelvic nodes and 52 Gy/32 fractions to the uninvolved bladder and pelvic nodes. Pre-treatment set-up was informed by cone-beam CT. For patients who experienced local relapse, cystoscopy and imaging (CT/MRI) was used to reconstruct the relapse gross tumour volume (GTV relapse) on the original planning CT. GTV relapse D98% and D95% was determined by co-registering the relapse image to the planning CT utilising deformable image registration (DIR) and rigid image registration (RIR). Failure was classified into five types based on spatial and dosimetric criteria as follows: A (central high-dose failure), B (peripheral high-dose failure), C (central elective dose failure), D (peripheral elective dose failure) and E (extraneous dose failure). Between June 2009 and November 2012, 38 patients were recruited. Following treatment, 18/38 (47%) patients experienced local relapse within the bladder. The median time to local relapse was 9.0 months (95% confidence interval 6.3–11.7). Seventeen of 18 patients were evaluable based on the availability of cross-sectional relapse imaging. A significant difference between DIR and RIR methods was seen. With the DIR approach, the median GTV relapse D98% and D95% was 97% and 98% of prescribed dose, respectively. Eleven of 17 (65%) patients experienced type A failure and 6/17 (35%) patients type B failure. No patients had type C, D or E failure. MIBC failure occurred in 10/17 (59%) relapsed patients; of those, 7/11 (64%) had type A failure and 3/6 (50%) had type B failure. Non-MIBC failure occurred in 7/17 (41%) patients; 4/11 (36%) with type A failure and 3/6 (50%) with type B failure. Relapse following radiotherapy occurred within close proximity to the original bladder tumour volume and within the planned high-dose region, suggesting possible biological causes for failure. We advise caution when considering margin reduction for future reduced high-dose radiation volume or partial bladder radiotherapy protocols. • Relapse following radical bladder radiotherapy most often occurs within the bladder. • Local relapse occurs near the original MIBC tumour site and within the planned high dose region. • In this high risk MIBC patient population, muscle invasive relapse was more common than non-muscle invasive relapse. • Tumour dose escalation may be an appropriate future consideration; trial outcome data is awaited. • Margin reduction should be considered with caution for reduced high-dose volume or partial bladder radiotherapy. [ABSTRACT FROM AUTHOR]

Purpose: This investigation compares three commercial methods of cone-beam computed tomography (CBCT)-based dosimetric analysis to a method based on repeat computed tomography (CT). Materials and Methods: Seventeen head-and-neck patients treated in 2020, and with a repeat CT, were included in the analyses. The planning CT was deformed to anatomy in repeat CT to generate a reference plan. Two of the CBCT-based methods generated test plans by deforming the planning CT to CBCT of fraction N using VelocityAI™ and SmartAdapt®. The third method compared directly calculated doses on the CBCT for fraction 1 and fraction N, using PerFraction™. Maximum dose to spinal cord (Cord_dmax) and dose to 95% volume (D95) of planning target volumes (PTVs) were used to assess “need to replan” criteria. Results: The VelocityAI™ method provided results that most accurately matched the reference plan in “need to replan” criteria using either Cord_dmax or PTV D95. SmartAdapt® method overestimated the change in Cord_dmax (6.77% vs. 3.85%, P < 0.01) and change in cord volume (9.56% vs. 0.67%, P < 0.01) resulting in increased false positives in “need to replan” criteria, and performed similarly to VelocityAI™ for D95, but yielded more false negatives. PerFraction™ method underestimated Cord_dmax, did not perform any volume deformation, and missed all “need to replan” cases based on cord dose. It also yielded high false negatives using the D95 PTV criteria. Conclusions: The VelocityAI™-based method using fraction N CBCT is most similar to the reference plan using repeat CT; the other two methods had significant differences.

The performance and speed of medical image registration have been greatly boosted by advanced deep-learning based methods. However, most current methods are challenged by large deformations between input images, which necessitate a compromise in computational cost to enhance the model's receptive field and its ability to model long-range spatial relationships for improving registration performance. In order to enhance the performance of registration for images with large deformations at a lower computational cost, in this paper, we propose a light-weight registration model with the ability to model large receptive fields and long-range spatial relationships, named LL-Net. The core components of LL-Net consist of a Rectangular Decomposition Large Kernel Attention (RD-LKA) layer and a Spatial and Channel Fusion Attention (SC-Fusion) layer. The RD-LKA layer utilizes anisotropic depth-wise large kernel convolutions to capture large receptive fields with an extremely low parameter count while modeling long-range spatial relationships. Moreover, the SC-Fusion layer enhances the model's feature fusion capability and strengthens feature representations at critical locations. Our LL-Net exhibits state-of-the-art performance across multiple datasets. Specifically, it achieves a Dice score of 76.7% and an HD95 of 2.983 mm on the IXI dataset, and a Dice score of 87.8% and an HD95 of 1.042 mm on the OASIS dataset. Experimental results substantiate the efficacy of LL-Net in capturing large receptive fields and modeling long-range spatial relationships. The code for LL-Net is available at https://github.com/BoyOfChu/LL_Net. • A light-weight registration network that models large receptive fields. • RD-LKA block extracts global semantic information and detailed local information. • SC-Fusion block decouples the interaction of spatial and channel dimensions. • Experimental results on two datasets validate the effectiveness of our network. [ABSTRACT FROM AUTHOR]

We address the problem of lung CT image registration, which underpins various diagnoses and treatments for lung diseases. The main crux of the problem is the large deformation that the lungs undergo during respiration. This physiological process imposes several challenges from a learning point of view. In this paper, we propose a novel training scheme, called stochastic decomposition, which enables deep networks to effectively learn such a difficult deformation field during lung CT image registration. The key idea is to stochastically decompose the deformation field, and supervise the registration by synthetic data that have the corresponding appearance discrepancy. The stochastic decomposition allows for revealing all possible decompositions of the deformation field. At the learning level, these decompositions can be seen as a prior to reduce the ill-posedness of the registration yielding to boost the performance. We demonstrate the effectiveness of our framework on Lung CT data. We show, through extensive numerical and visual results, that our technique outperforms existing methods. • A novel and effective training algorithm for deformable lung CT registration. • Stochastic decomposition of large deformation and supervision with surrogate images. • The surrogate image has corresponding intensity discrepancy at decomposition phase. [ABSTRACT FROM AUTHOR]

PURPOSE:The deformable registration of pulmonary computed tomography images before and after radiation therapy is challenging due to anatomic changes from radiation fibrosis. We hypothesize that a line-enhanced registration algorithm can reduce landmark error over the entire lung, including the irradiated regions, when compared to an intensity-based deformable registration algorithm. MATERIALS:Two intensity-based B-spline deformable registration algorithms of pre-radiation therapy and post-radiation therapy images were compared. The first was a control intensity-based algorithm that utilized computed tomography images without modification. The second was a line enhancement algorithm that incorporated a Hessian-based line enhancement filter prior to deformable image registration. Registrations were evaluated based on the landmark error between user-identified landmark pairs and the overlap ratio. RESULTS:Twenty-one patients with pre-radiation therapy and post-radiation therapy scans were included. The median time interval between scans was 1.2 years (range: 0.3-3.3 years). Median landmark errors for the line enhancement algorithm were significantly lower than those for the control algorithm over the entire lung (1.67 vs 1.83 mm; P < .01), as well as within the 0 to 5 Gy (1.40 vs 1.57; P < .01) and >5 Gy (2.25 vs 3.31; P < .01) dose intervals. The median lung mask overlap ratio for the line enhancement algorithm (96.2%) was greater than that for the control algorithm (95.8%; P < .01). Landmark error within the >5 Gy dose interval demonstrated a significant inverse relationship with post-radiation therapy fibrosis enhancement after line enhancement filtration (Pearson correlation coefficient = -0.48; P = .03). CONCLUSION:The line enhancement registration algorithm is a promising method for registering images before and after radiation therapy.

Abstract Background Image-guided adaptive brachytherapy shows the ability to deliver high doses to tumors while sparing normal tissues. However, interfraction dose delivery introduces uncertainties to high dose estimation, which relates to normal tissue toxicity. The purpose of this study was to investigate the high-dose regions of two applicator approaches in brachytherapy. Method For 32 cervical cancer patients, the CT images from each fraction were wrapped to a reference image, and the displacement vector field (DVF) was calculated with a hybrid intensity-based deformable registration algorithm. The fractional dose was then accumulated to calculate the position and the overlap of high dose (D2cc) during multiple fractions. Result The overall Dice similarity coefficient (DSC) of the deformation algorithm for the bladder and the rectum was (0.97 and 0.91). No significant difference was observed between the two applicators. However, the location of the intracavitary brachytherapy (ICBT) high-dose region was relatively concentrated. The overlap volume of bladder and rectum D2cc was 0.42 and 0.71, respectively, which was higher than that of interstitial brachytherapy (ISBT) (0.26 and 0.31). The cumulative dose was overestimated in ISBT cases when using the GEC-recommended method. The ratio of bladder and rectum D2cc to the GEC method was 0.99 and 1, respectively, which was higher than that of the ISBT method (0.96 and 0.94). Conclusion High-dose regions for brachytherapy based on different applicator types were different. The 3D-printed ICBT has better high-dose region consistency than freehand ISBT and hence is more predictable.

Background: Spinal deformation during surgical intervention (caused by patient positioning and/or the correction of malalignment) confounds conventional navigation due to the assumptions of rigid transformation. Moreover, the ability to accurately quantify spinal alignment in the operating room would provide an assessment of the surgical product via metrics that correlate with clinical outcomes. Purpose: A method for deformable 3D–2D registration of preoperative CT to intraoperative long‐length tomosynthesis images is reported for an accurate 3D evaluation of device placement in the presence of spinal deformation and automated evaluation of global spinal alignment (GSA). Methods: Long‐length tomosynthesis ("Long Film," LF) images were acquired using an O‐arm imaging system (Medtronic, Minneapolis USA). A deformable 3D–2D patient registration was developed using multi‐scale masking (proceeding from the full‐length image to local subvolumes about each vertebra) to transform vertebral labels and planning information from preoperative CT to the LF images. Automatic measurement of GSA (main thoracic kyphosis [MThK] and lumbar lordosis [LL]) was obtained using a spline fit to registered labels. The "Known‐Component Registration" method for device registration was adapted to the multi‐scale process for 3D device localization from orthogonal LF images. The multi‐scale framework was evaluated using a deformable spine phantom in which pedicle screws were inserted, and deformations were induced over a range in LL ∼25°–80°. Further validation was carried out in a cadaver study with implanted pedicle screws and a similar range of spinal deformation. The accuracy of patient and device registration was evaluated in terms of 3D translational error and target registration error, respectively, and the accuracies of automatic GSA measurements were compared to manual annotation. Results: Phantom studies demonstrated accurate registration via the multi‐scale framework for all vertebral levels in both the neutral and deformed spine: median (interquartile range, IQR) patient registration error was 1.1 mm (0.7–1.9 mm IQR). Automatic measures of MThK and LL agreed with manual delineation within −1.1° ± 2.2° and 0.7° ± 2.0° (mean and standard deviation), respectively. Device registration error was 0.7 mm (0.4–1.0 mm IQR) at the screw tip and 0.9° (1.0°–1.5°) about the screw trajectory. Deformable 3D–2D registration significantly outperformed conventional rigid registration (p < 0.05), which exhibited device registration errors of 2.1 mm (0.8–4.1 mm) and 4.1° (1.2°–9.5°). Cadaver studies verified performance under realistic conditions, demonstrating patient registration error of 1.6 mm (0.9–2.1 mm); MThK within −4.2° ± 6.8° and LL within 1.7° ± 3.5°; and device registration error of 0.8 mm (0.5–1.9 mm) and 0.7° (0.4°–1.2°) for the multi‐scale deformable method, compared to 2.5 mm (1.0–7.9 mm) and 2.3° (1.6°–8.1°) for rigid registration (p < 0.05). Conclusion: The deformable 3D–2D registration framework leverages long‐length intraoperative imaging to achieve accurate patient and device registration over the extended lengths of the spine (up to 64 cm) even with strong anatomical deformation. The method offers a new means for the quantitative validation of spinal correction (intraoperative GSA measurement) and the 3D verification of device placement in comparison to preoperative images and planning data. [ABSTRACT FROM AUTHOR]

Diabetic retinopathy is one of the most dangerous complications of diabetes. It affects the eyes causing damage to the blood vessels of the retina. Eventually, as the disease develops, it is possible to lose sight. The main cure for this pathology is based on the early detection which plays a crucial role in slowing the progress of the underlying disease and protecting many patients from losing their sight. However, the detection of diabetic retinopathy at its early stages remains an arduous task that requires human expert interpretation of fundus images in order to vigilantly follow-up the patient. In this paper, we shall propose a new automatic diabetic retinopathy detection method that based on deep-learning. The aforementioned approach is composed of two main steps: an initial pre-processing step where the deformable registration is applied on the retina to occupy the entire image and eliminate the effect of the background on the classification process. The second step is the classification phase in which we train four convolutional neural networks (CNN) models (Densenet-121, Xception, Inception-v3, Resnet-50) to detect the stage of diabetic retinopathy. The performance of our proposed architecture has been tested on the APTOS 2019 dataset. As the latter is relatively small, a transfer learning is adopted by pre-training the mentioned CNNs on the ImageNet dataset and fine-tuning them on the APTOS dataset. In the testing phase, the final prediction is obtained by a system of voting based on the output of the four convolutional neural networks. Our model has performed an accuracy of 85.28% in the testing phase. [ABSTRACT FROM AUTHOR]

This study aimed to assess and compare the capabilities of two commercially available deformable image registration algorithms implemented in Raystation 9A (A 1) and Velocity AI (A 2) for possible usage in adaptive prostate radiotherapy based on the propagation of anatomical contours from computed tomography (CT) images to cone-beam CT (CBCT) images. Ten patients were retrospectively selected from a group treated for localized prostate cancer. The propagation of rectum contours was analyzed in a set of CT-CBCT pairs. Two independent observers carried out qualitative analysis using the two-level descriptive scale (meet/fail). Quantitative analysis was done using landmark points distances based on implanted markers as navigation points and differently obtained contours (manually and automatically using DIR algorithms). Quantitative analysis was taken on sets preselected by qualitative analysis. Qualitative analysis shows that 83.7% of the rectum contours were scored identically (meet or fail) for both algorithms, from which 53.5% and 55.8% are failed results for A 1 and A 2 , respectively. For the rectum size (RWD parameter), differences between referenced and deformation-based values were 5.5 and 5.8 mm, and for the rectum wall, the prostate marker distance (WMD parameter) was 4.5 and 5.5 mm for A 1 and A 2 , respectively. The differences between the WMD parameters were statistically significant (p = 0.007). In both tested algorithms, neither effectiveness nor measured uncertainties in the propagation of rectum contour process in prostate patient cases were satisfactory. Careful selection of input images followed by case/set-based verification of every deformable registration is a substantial step to avoid inappropriate results. [ABSTRACT FROM AUTHOR]

Medical image segmentation and registration are two tasks to analyze the anatomical structures in clinical research. Still, deep-learning solutions utilizing the connections between segmentation and registration remain underdiscovered. This article designs a joint learning framework named RSegNet that can realize concurrent deformable registration and segmentation by minimizing an integrated loss function, including three parts: diffeomorphic registration loss, segmentation similarity loss, and dual-consistency supervision loss. The probabilistic diffeomorphic registration branch could benefit from the auxiliary segmentations available from the segmentation branch to achieve anatomical consistency and better deformation regularity by dual-consistency supervision. Simultaneously, the segmentation performance could also be improved by data augmentation based on the registration with well-behaved diffeomorphic guarantees. Experiments on the human brain 3-D magnetic resonance images have been implemented to demonstrate the effectiveness of our approach. We trained and validated RSegNet with 1000 images and tested its performances on four public datasets, which shows that our method successfully yields concurrent improvements of both segmentation and registration compared with separately trained networks. Specifically, our method can increase the accuracy of segmentation and registration by 7.0% and 1.4%, respectively, in terms of Dice scores. Note to Practitioners—Registration and segmentation of medical images are two significant tasks in medical research and clinical application. However, most existing approaches consider these two tasks independently while neglecting the potential association between them. Therefore, we suggest a new approach that combines these two tasks into one joint deep learning framework, boosting registration, and segmentation performance by introducing dual-consistency supervision. Besides, our framework could generate outputs within 1 s by taking an affinely aligned medical image pair as input, which is suitable for time-critical requirements in a clinic. We tested it on four public datasets and achieved state-of-the-art performance to demonstrate the proposed method’s feasibility and robustness. Furthermore, our proposed RSegNet is a general learning framework suitable for various image modalities and anatomical structures. Hence, we expect our framework to serve as a practical clinical tool to speed up medical image analysis procedures and improve diagnostic accuracy. [ABSTRACT FROM AUTHOR]

The registration of preoperative and follow-up brain MRI, which is crucial in illustrating patients' responses to treatments and providing guidance for postoperative therapy, presents significant challenges. These challenges stem from the considerable deformation of brain tissue and the areas of non-correspondence due to surgical intervention and postoperative changes. We propose a stepwise corrected attention registration network grounded in convolutional neural networks (CNNs). This methodology leverages preoperative and follow-up MRI scans as fixed images and moving images, respectively, and employs a multi-level registration strategy that establishes a precise and holistic correspondence between images, from coarse to fine. Furthermore, our model introduces a corrected attention module into the multi-level registration network that can generate an attention map at the local level through the deformation fields of the upper-level registration network and pathological areas of preoperative images segmented by a mature algorithm in BraTS, serving to strengthen the registration accuracy of non-correspondence areas. A comparison between our scheme and the leading approach identified in the MICCAI's BraTS-Reg challenge indicates a 7.5% enhancement in the target registration error (TRE) metric and improved visualization of non-correspondence areas. These results illustrate the better performance of our stepwise corrected attention registration network in not only enhancing the registration accuracy but also achieving a more logical representation of non-correspondence areas. Thus, this work contributes significantly to the optimization of the registration of brain MRI between preoperative and follow-up scans.

Background: The use of magnetic resonance (MR) imaging for proton therapy treatment planning is gaining attention as a highly effective method for guidance. At the core of this approach is the generation of computed tomography (CT) images from MR scans. However, the critical issue in this process is accurately aligning the MR and CT images, a task that becomes particularly challenging in frequently moving body areas, such as the head-and-neck. Misalignments in these images can result in blurred synthetic CT (sCT) images, adversely affecting the precision and effectiveness of the treatment planning., Purpose: This study introduces a novel network that cohesively unifies image generation and registration processes to enhance the quality and anatomical fidelity of sCTs derived from better-aligned MR images., Methods: The approach synergizes a generation network (G) with a deformable registration network (R), optimizing them jointly in MR-to-CT synthesis. This goal is achieved by alternately minimizing the discrepancies between the generated/registered CT images and their corresponding reference CT counterparts. The generation network employs a UNet architecture, while the registration network leverages an implicit neural representation (INR) of the displacement vector fields (DVFs). We validated this method on a dataset comprising 60 head-and-neck patients, reserving 12 cases for holdout testing., Results: Compared to the baseline Pix2Pix method with MAE 124.95 ± $\pm$ 30.74 HU, the proposed technique demonstrated 80.98 ± $\pm$ 7.55 HU. The unified translation-registration network produced sharper and more anatomically congruent outputs, showing superior efficacy in converting MR images to sCTs. Additionally, from a dosimetric perspective, the plan recalculated on the resulting sCTs resulted in a remarkably reduced discrepancy to the reference proton plans., Conclusions: This study conclusively demonstrates that a holistic MR-based CT synthesis approach, integrating both image-to-image translation and deformable registration, significantly improves the precision and quality of sCT generation, particularly for the challenging body area with varied anatomic changes between corresponding MR and CT., (© 2024 The Author(s). Medical Physics published by Wiley Periodicals LLC on behalf of American Association of Physicists in Medicine.)

Objective . Existing registration networks based on cross-attention design usually divide the image pairs to be registered into patches for input. The division and merging operations of a series of patches are difficult to maintain the topology of the deformation field and reduce the interpretability of the network. Therefore, our goal is to develop a new network architecture based on a cross-attention mechanism combined with a multi-resolution strategy to improve the accuracy and interpretability of medical image registration. Approach . We propose a new deformable image registration network NCNet based on neighborhood cross-attention combined with multi-resolution strategy. The network structure mainly consists of a multi-resolution feature encoder, a multi-head neighborhood cross-attention module and a registration decoder. The hierarchical feature extraction capability of our encoder is improved by introducing large kernel parallel convolution blocks; the cross-attention module based on neighborhood calculation is used to reduce the impact on the topology of the deformation field and double normalization is used to reduce its computational complexity. Main result . We performed atlas-based registration and inter-subject registration tasks on the public 3D brain magnetic resonance imaging datasets LPBA40 and IXI respectively. Compared with the popular VoxelMorph method, our method improves the average DSC value by 7.9% and 3.6% on LPBA40 and IXI. Compared with the popular TransMorph method, our method improves the average DSC value by 4.9% and 1.3% on LPBA40 and IXI. Significance . We proved the advantages of the neighborhood attention calculation method compared to the window attention calculation method based on partitioning patches, and analyzed the impact of the pyramid feature encoder and double normalization on network performance. This has made a valuable contribution to promoting the further development of medical image registration methods., (© 2024 IOP Publishing Ltd.)