Search

Multimodal image registration is applied in medical image analysis as it allows the integration of complementary data from multiple imaging modalities. In recent years, various neural network-based approaches for medical image registration have been presented in papers, but due to the use of different datasets, a fair comparison is not possible. In this research 20 different neural networks for an affine registration of medical images were implemented. The networks’ performance and the networks’ generalizability to new datasets were evaluated using two multimodal datasets - a synthetic and a real patient dataset - of three-dimensional CT and MR images of the liver. The networks were first trained semi-supervised using the synthetic dataset and then evaluated on the synthetic dataset and the unseen patient dataset. Afterwards, the networks were finetuned on the patient dataset and subsequently evaluated on the patient dataset. The networks were compared using our own developed CNN as benchmark and a conventional affine registration with SimpleElastix as baseline. Six networks improved the pre-registration Dice coefficient of the synthetic dataset significantly (p-value

We proposed neural networks for deformable multimodal medical image registration that use multiple steps and varying resolutions. The networks were trained jointly in an unsupervised manner with Mutual Information and Gradient L2 loss. By comparing the multistep neural networks to each other and to a monostep/monoresolution network as a benchmark and the classical registration methods SimpleElastix and NiftyReg as a baseline, we investigated the impact of using multiple resolutions on the registration result. To assess the performance of the multistep networks, we used four three-dimensional multimodal datasets (a synthetic and an in-vivo liver dataset with CT and T1-weighted MR scans, an in-vivo kidney MR dataset with T1-weighted and T2-weighted MR scans and an in-vivo prostate MR dataset with T2-weighted and DWI MR scans). Experimental results showed that incorporating multiple steps and resolutions in a neural network leads to registration results with high structural similarity (NMI up to 0.33 ± 0.02, Dice up to 90.8 ± 3.1) and minimal image folding (|J $\vert \le 0$ : less than 0.5%), resulting in a medically plausible transformation, while maintaining a low registration time (

Multimodal image registration is an important component of medical image processing, allowing the integration of complementary information from various imaging modalities to improve clinical applications like diagnosis and treatment planning. We proposed a novel multistage neural network for three-dimensional multimodal medical image registration, which addresses the challenge of larger rigid deformations commonly present in medical images due to variations in patient positioning in different scanners and rigid anatomical structures. This multistage network combines rigid, affine and deformable transformations in three stages. The network was trained unsupervised with Mutual Information and Gradient L2 loss. We compared the results of our proposed multistage network with a rigid-affine-deformable registration with the classical registration method NiftyReg as a baseline and a multistage network, which combines affine and deformable transformation, as a benchmark. To evaluate the performance of the proposed multistage network, we used four three-dimensional multimodal in vivo datasets: three renal MR datasets consisting of T1-weighted and T2-weighted MR scans and one liver dataset containing CT and T1-weighted MR scans. Experimental results showed that combining rigid, affine and deformable transformations in a multistage network leads to registration results with a high structural similarity, overlap of the corresponding structures (Dice: 76.7 ± 12.5, 61.1 ± 14.0, 64.8 ± 16.2, 68.1 ± 24.6 for the four datasets) and a low level of image folding (|J| ≤ 0: less than or equal to 1.1%), resulting in a medical plausible registration result.