Search

Purpose : Augmented reality (AR) may allow vitreoretinal surgeons to leverage microscope-integrated digital imaging systems to analyze and highlight key retinal anatomic features in real time, possibly improving safety and precision during surgery. By employing convolutional neural networks (CNNs) for retina vessel segmentation, a retinal coordinate system can be created that allows pre-operative images of capillary non-perfusion or retinal breaks to be digitally aligned and overlayed upon the surgical field in real time. Such technology may be useful in assuring thorough laser treatment of capillary non-perfusion or in using pre-operative optical coherence tomography (OCT) to guide macular surgery when microscope-integrated OCT (MIOCT) is not available. Methods : This study is a retrospective analysis involving the development and testing of a novel image-registration algorithm for vitreoretinal surgery. Fifteen anonymized cases of pars plana vitrectomy with epiretinal membrane peeling, along with corresponding preoperative fundus photographs and optical coherence tomography (OCT) images, were retrospectively collected from the Mayo Clinic database. We developed a TPU (Tensor-Processing Unit)-accelerated CNN for semantic segmentation of retinal vessels from fundus photographs and subsequent real-time image registration in surgical video streams. An iterative patch-wise cross-correlation (IPCC) algorithm was developed for image registration, with a focus on optimizing processing speeds and maintaining high spatial accuracy. The primary outcomes measured were processing speed in frames per second (FPS) and the spatial accuracy of image registration, quantified by the Dice coefficient between registered and manually aligned images. Results : When deployed on an Edge TPU, the CNN model combined with our image-registration algorithm processed video streams at a rate of 14 FPS, which is superior to processing rates achieved on other standard hardware configurations. The IPCC algorithm efficiently aligned pre-operative and intraoperative images, showing high accuracy in comparison to manual registration. Conclusions : This study demonstrates the feasibility of using TPU-accelerated CNNs for enhanced AR in vitreoretinal surgery.

Background: In neuroscience research about functional magnetic resonance imaging (fMRI), accurate inter-subject image registration is the basis for effective statistical analysis. Traditional fMRI registration methods are usually based on high-resolution structural MRI with clear anatomical structure features. However, this registration method based on structural information cannot achieve accurate functional consistency between subjects since the functional regions do not necessarily correspond to anatomical structures. In recent years, fMRI registration methods based on functional information have emerged, which usually ignore the importance of structural MRI information., Methods: In this study, we proposed a non-rigid cycle consistent bidirectional network with Transformer for unsupervised deformable functional MRI registration. The work achieves fMRI registration through structural MRI registration, and functional information is introduced to improve registration performance. Specifically, we employ a bidirectional registration network that implements forward and reverse registration between image pairs and apply Transformer in the registration network to establish remote spatial mapping between image voxels. Functional and structural information are integrated by introducing the local functional connectivity pattern, the local functional connectivity features of the whole brain are extracted as functional information. The proposed registration method was experimented on real fMRI datasets, and qualitative and quantitative evaluations of the quality of the registration method were implemented on the test dataset using relevant evaluation metrics. We implemented group ICA analysis in brain functional networks after registration. Functional consistency was evaluated on the resulting t-maps., Results: Compared with non-learning-based methods (Affine, Syn) and learning-based methods (Transmorph-tiny, Cyclemorph, VoxelMorph x2), our method improves the peak t-value of t-maps on DMN, VN, CEN, and SMN to 18.7, 16.5, 16.6, and 17.3 and the mean number of suprathreshold voxels ( p < 0.05, t > 5.01) on the four networks to 2596.25, and there is an average improvement in peak t-value of 23.79%, 12.74%, 12.27%, 7.32%, and 5.43%., Conclusions: The experimental results show that the registration method of this study improves the structural and functional consistency between fMRI with superior registration performance.

Tertiary Lymphoid Structures (TLS) are considered as genuine markers of inflammation. Their presence within inflamed tissues or the tumor microenvironment has been associated with the local development of an active immune response. While high densities of TLS are correlated with disease severity in autoimmune diseases or during graft rejection, it has been associated with longer patient survival in many cancer types and more recently with positive responses to anti-PD-1 immunotherapy. Their efficient visualization and quantification within human tissues may represent new tools for helping clinicians in adjusting their therapeutic strategy. In clinical settings, the use of single-marker immunohistochemistry (IHC) protocols prevails in immune cell infiltration in formalin-fixed, paraffin-embedded (FFPE) tissues. In contrast, the development of automated multiplex immunofluorescence markings, i.e., 40-plex, requires very costly investments in equipment and analysis stations. Yet, employing two or more markers can enhance the characterization of immune infiltrates, particularly in the context of TLS. Besides the growing development of multiplex labeling approaches, imaging can also be used to overcome some technical difficulties encountered during the immunolabeling of tissues with several markers.This chapter describes IHC methods to visualize in human tissue (tumoral or not) the presence of TLS. These methods are based on the immunostaining of four TLS-associated immune cell populations, namely, follicular B cells, follicular dendritic Cells (FDCs), mature Dendritic Cells (mDCs), and Follicular Helper T cells (T FH ), together with non-T FH T cells. Methodologies for subsequent quantification of TLS density are also proposed, as well as a virtual multiplexing method based on image registration using the open-source software ImageJ (IJ), aiming at co-localizing several immune cell populations from different IHC stainings performed on serial tissue sections., (© 2025. The Author(s), under exclusive license to Springer Science+Business Media, LLC, part of Springer Nature.)

Objective . Previous work has that deep learning (DL)-enhanced 4D cone beam computed tomography (4D-CBCT) images improve motion modeling and subsequent motion-compensated (MoCo) reconstruction for 4D-CBCT. However, building the motion model at treatment time via conventional deformable image registration (DIR) methods is not temporally feasible. This work aims to improve the efficiency of 4D-CBCT MoCo reconstruction using DL-based registration for the rapid generation of a motion model prior to treatment. Approach. An artifact-reduction DL model was first used to improve the initial 4D-CBCT reconstruction by reducing streaking artifacts. Based on the artifact-reduced phase images, a groupwise DIR employing DL was used to estimate the inter-phase motion model. Two DL DIR models using different learning strategies were employed: (1) a patient-specific one-shot DIR model which was trained from scratch only using the images to be registered, and (2) a population DIR model which was pre-trained using collected 4D-CT images from 35 patients. The registration accuracy of two DL DIR models was assessed and compared to a conventional groupwise DIR approach implemented in the Elastix toolbox using the publicly available DIR-Lab dataset, a Monte Carlo simulation dataset from the SPARE challenge, and two clinical cases. Main results. The patient-specific DIR model and the population DIR model demonstrated registration accuracy comparable to the conventional state-of-the-art methods on the DIR-Lab dataset. No significant difference in image quality was observed between the final MoCo reconstructions using the patient-specific model and population model for motion modeling, compared to using the conventional approach. The average runtime (hh:mm:ss) of the entire MoCo reconstruction on SPARE dataset was reduced from 01:37:26 using conventional DIR method to 00:10:59 using patient-specific model and 00:01:05 using the pre-trained population model. Significance. DL-based registration methods can improve the efficiency in generating motion models for 4D-CBCT without compromising the performance of final MoCo reconstruction., (Creative Commons Attribution license.)

Multispectral transmission imaging has emerged as a promising technique for imaging breast tissue with high resolution. However, the method encounters challenges such as low grayscale, noisy transmission images with weak signals, primarily due to the strong absorption and scattering of light in breast tissue. A common approach to improve the signal-to-noise ratio (SNR) and overall image quality is frame accumulation. However, factors such as camera jitter and respiratory motion during image acquisition can cause frame misalignment, degrading the quality of the accumulated image. To address these issues, this study proposes a novel image registration method. A hybrid approach combining a genetic algorithm (GA) and a constriction factor-based particle swarm optimization (CPSO), referred to as GA-CPSO, is applied for image registration before frame accumulation. The efficiency of this hybrid method is enhanced by incorporating a squared constriction factor (SCF), which speeds up the registration process and improves convergence towards optimal solutions. The GA identifies potential solutions, which are then refined by CPSO to expedite convergence. This methodology was validated on the sequence of breast frames taken at 600 nm, 620 nm, 670 nm, and 760 nm wavelength of light and proved the enhancement of accuracy by various mathematical assessments. It demonstrated high accuracy (99.93%) and reduced registration time. As a result, the GA-CPSO approach significantly improves the effectiveness of frame accumulation and enhances overall image quality. This study explored the groundwork for precise multispectral transmission image segmentation and classification.

Image registration is a crucial post-processing technique in biomedical imaging, enabling the alignment and integration of images from various sources to facilitate accurate diagnosis, treatment planning, and longitudinal studies. This paper explores the application of Scale Invariant Feature Transform (SIFT), a robust feature-based method for the alignment of biomedical images. SIFT is particularly advantageous due to its invariance to scale, rotation, and affine transformations, making it well-suited for handling the diverse and complex nature of biomedical images. However, SIFT was not initially developed specifically for medical imaging applications, so it is necessary to adapt the algorithm to those kinds of images. In particular, this work was focused on images obtained with Cone-Beam Computed Tomography (CBCT) technology. Besides fine-tuning SIFT parameters on a case-by-case basis, the novelty of this work consists of finding the optimal SIFT parameters on the basis of the keypoints stability. A statistical analysis throughout a dataset of images obtained with CBCT technology was performed to find the best SIFT parameters setting, in terms of computational cost and result quality, compared to default presets.

Background: For a single exposure in radiography, a dual-layer flat-panel detector (DFD) can provide spectral images and efficiently utilize the transmitted X-ray photons to improve the detective quantum efficiency (DQE) performance. In this paper, to acquire high DQE performance, we present a registration method for X-ray images acquired from a DFD, considering only spatial translations and scale factors. The conventional registration methods have inconsistent estimate accuracies depending on the captured object scene, even when using entire pixels, and have deteriorated frequency performance because of the interpolation method employed., Methods: The proposed method consists of two steps; the first step is conducting a spatial translation according to the Fourier shift theorem with a subpixel registration, and the second step is conducting a scale transformation using cubic interpolation to process the X-ray projections. To estimate the subpixel spatial translation, a maximum-amplitude method using a small portion of the slant-edge phantom is used., Results: The performance of the proposed two-step method is first theoretically analyzed and then observed by conducting extensive experiments and measuring the noise power spectrum and DQE. An example for registering chest images is also shown. For a DFD, the proposed method shows a better registration result than the conventional one-step registration. The DQE improvement was more than 56% under RQA 9 compared to the single flat-panel detector case., Conclusions: The proposed two-step registration method can efficiently provide aligned image pairs from the DFD to improve the DQE performance at low doses and, thus, increase the accuracy of clinical diagnosis.

Introduction : Prostate-specific membrane antigen positron emission tomography/computed tomography (PSMA PET/CT), in combination with magnetic resonance imaging (MRI), may enhance the diagnosis and staging of prostate cancer. Image fusion of separately acquired PET/CT and MRI images serve to facilitate clinical integration and treatment planning. This study aimed to investigate different PSMA PET/CT and MRI image fusion workflows for prostate cancer visualisation. Methods : Eighteen patients with prostate cancer who underwent PSMA PET/CT and MRI prior to radical prostatectomy were retrospectively selected. Alignment of the prostate was performed between PET/CT and MRI via three techniques: semi-automatic rigid, automatic rigid, and automatic non-rigid. Image fusion accuracy was evaluated through boundary and volume agreement, quantified by the Dice Similarity Coefficient (DSC), 95% Hausdorff Distance (HD), and Mean Surface Distance (MSD), with comparison against reconstructed histopathology slices. Results : Image fusion using all techniques resulted in clear lesion visualisation from PSMA PET/CT overlay and anatomical detail afforded by the MRI base and was consistent with histopathology tumour location. Image fusion accuracy was within the recommended range based on a DSC of 0.8-0.9. The automatic non-rigid registration method had the highest volume agreement (DSC: 0.96 ± <0.01) and boundary agreement (HD: 1.17 ± 0.35 mm) when compared to automatic rigid (DSC 0.88 ± 0.02, HD 3.18 ± 0.29 mm) and semi-automatic rigid (DSC 0.80 ± 0.06, HD 5.25 ± 1.68 mm). Conclusions : Image fusion of clinically obtained PET/CT and MRI is feasible and clinically acceptable for use in prostate cancer diagnosis and surgical management. While the best accuracy was observed with the automatic non-rigid technique, which requires further validation, image fusion with clinically accessible methods (semi-automatic rigid) may currently aid patient education, pre-operative planning, and intra-operative guidance.

An image guided robot only becomes fully useful with integrated software leveraging image fusion. Image fusion is the process of registering and superimposing imaging data in the same coordinate space and can be helpful to image-guided robotic interventions. Effective percutaneous robotic procedures can utilize real-time image guidance and navigation which are powered by fusion technologies. By integrating information from multiple imaging modalities, fusion technologies provide insights into anatomic features and procedural targets that may not be apparent through traditional positional tracking or single-modality imaging. Current robots available for interventions highlight different approaches to utilizing real-time fusion and procedure planning. As robotics become increasingly integrated into interventional radiology clinical practice, the continued innovation and adoption of fusion-based approaches will enable more seamless use of this technology, offering the potential for improved safety, standardization, and clinical efficacy. This review explores key techniques in image fusion and highlights the integration of fusion and robotics towards the goal of optimized and automated interventional procedures., (Copyright © 2024. Published by Elsevier Inc.)

Rationale and Objective: Stroke-associated pneumonia (SAP) often appears as a complication following intracerebral hemorrhage (ICH), leading to poor prognosis and increased mortality rates. Previous studies have typically developed prediction models based on clinical data alone, without considering that ICH patients often undergo CT scans immediately upon admission. As a result, these models are subjective and lack real-time applicability, with low accuracy that does not meet clinical needs. Therefore, there is an urgent need for a quick and reliable model to timely predict SAP., Methods: In this retrospective study, we developed an image-based model (DeepSAP) using brain CT scans from 244 ICH patients to classify the presence and severity of SAP. First, DeepSAP employs MRI-template-based image registration technology to eliminate structural differences between samples, achieving statistical quantification and spatial standardization of cerebral hemorrhage. Subsequently, the processed images and filtered clinical data were simultaneously input into a deep-learning neural network for training and analysis. The model was tested on a test set to evaluate diagnostic performance, including accuracy, specificity, and sensitivity., Results: Brain CT scans from 244 ICH patients (mean age, 60.24; 66 female) were divided into a training set (n = 170) and a test set (n = 74). The cohort included 143 SAP patients, accounting for 58.6% of the total, with 66 cases classified as moderate or above, representing 27% of the total. Experimental results showed an AUC of 0.93, an accuracy of 0.84, a sensitivity of 0.79, and a precision of 0.95 for classifying the presence of SAP. In comparison, the model relying solely on clinical data showed an AUC of only 0.76, while the radiomics method had an AUC of 0.74. Additionally, DeepSAP achieved an optimal AUC of 0.84 for the SAP grading task., Conclusion: DeepSAP's accuracy in predicting SAP stems from its spatial normalization and statistical quantification of the ICH region. DeepSAP is expected to be an effective tool for predicting and grading SAP in clinic., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 The Association of University Radiologists. Published by Elsevier Inc. All rights reserved.)

Retinal diseases stand as a primary cause of childhood blindness. Analyzing the progression of these diseases requires close attention to lesion morphology and spatial information. Standard image registration methods fail to accurately reconstruct pediatric fundus images containing significant distortion and blurring. To address this challenge, we proposed a robust deep learning-based image registration method (RDLR). The method consisted of two modules: registration module (RM) and panoramic view module (PVM). RM effectively integrated global and local feature information and learned prior information related to the orientation of images. PVM was capable of reconstructing spatial information in panoramic images. Furthermore, as the registration model was trained on over 280,000 pediatric fundus images, we introduced a registration annotation automatic generation process coupled with a quality control module to ensure the reliability of training data. We compared the performance of RDLR to the other methods, including conventional registration pipeline (CRP), voxel morph (WM), generalizable image matcher (GIM), and self-supervised techniques (SS). RDLR achieved significantly higher registration accuracy (average Dice score of 0.948) than the other methods (ranging from 0.491 to 0.802). The resulting panoramic retinal maps reconstructed by RDLR also demonstrated substantially higher fidelity (average Dice score of 0.960) compared to the other methods (ranging from 0.720 to 0.783). Overall, the proposed method addressed key challenges in pediatric retinal imaging, providing an effective solution to enhance disease diagnosis. Our source code is available at https://github.com/wuwusky/RobustDeepLeraningRegistration ., Competing Interests: Declarations. Ethical Approval: Ethical approval for research used was obtained from each center (the ethics committee of Zhongshan Ophthalmic Center, Sun Yat-sen University (2020KYPJ175), and the ethics committee of Guangdong Women and Children Hospital (202201057)). Consent to Participate: Informed consent was obtained from all individual participants included in the study. Written informed consent was obtained from the parents. Consent for Publication: The authors affirm that human research participants provided informed consent for publication of the images in all figures and tables. Competing Interests: The authors declare no competing interests., (© 2024. The Author(s) under exclusive licence to Society for Imaging Informatics in Medicine.)

Retinal image registration is of utmost importance due to its wide applications in medical practice. In this context, we propose ConKeD, a novel deep learning approach to learn descriptors for retinal image registration. In contrast to current registration methods, our approach employs a novel multi-positive multi-negative contrastive learning strategy that enables the utilization of additional information from the available training samples. This makes it possible to learn high-quality descriptors from limited training data. To train and evaluate ConKeD, we combine these descriptors with domain-specific keypoints, particularly blood vessel bifurcations and crossovers, that are detected using a deep neural network. Our experimental results demonstrate the benefits of the novel multi-positive multi-negative strategy, as it outperforms the widely used triplet loss technique (single-positive and single-negative) as well as the single-positive multi-negative alternative. Additionally, the combination of ConKeD with the domain-specific keypoints produces comparable results to the state-of-the-art methods for retinal image registration, while offering important advantages such as avoiding pre-processing, utilizing fewer training samples, and requiring fewer detected keypoints, among others. Therefore, ConKeD shows a promising potential towards facilitating the development and application of deep learning-based methods for retinal image registration., Competing Interests: Declarations Conflict of Interest The authors declare no competing interests., (© 2024. The Author(s).)

Registration is the process of spatially and/or temporally aligning different images. It is a critical tool that can facilitate the automatic tracking of pathological changes detected in radiological images and align images captured by different imaging systems and/or those acquired using different acquisition parameters. The longitudinal analysis of clinical changes has a significant role in helping clinicians evaluate disease progression and determine the most suitable course of treatment for patients. This study provides a comprehensive review of the role registration-based approaches play in automated change tracking in radiological imaging and explores the three types of registration approaches which include rigid, affine, and nonrigid registration, as well as methods of detecting and quantifying changes in registered longitudinal images: the intensity-based approach and the deformation-based approach. After providing an overview and background, we highlight the clinical applications of these methods, specifically focusing on computed tomography (CT) and magnetic resonance imaging (MRI) in tumors and multiple sclerosis (MS), two of the most heavily studied areas in automated change tracking. We conclude with a discussion and recommendation for future directions., (© 2024. The Author(s).)

The implementation of Machine Vision (MV) systems for Tool Condition Monitoring (TCM) plays a critical role in reducing the total cost of operation in manufacturing while expediting tool wear testing in research settings. However, conventional MV-TCM edge detection strategies process each image independently to infer edge positions, rendering them susceptible to inaccuracies when tool edges are compromised by material adhesion or chipping, resulting in imprecise wear measurements. In this study, an MV system is developed alongside an automated, feature-based image registration strategy to spatially align tool wear images, enabling a more consistent and accurate detection of tool edge position. The MV system was shown to be robust to the machining environment, versatile across both turning and milling machining centers and capable of reducing tool wear image capturing time up to 85% in reference to standard approaches. A comparison of feature detector-descriptor algorithms found SIFT, KAZE, and ORB to be the most suitable for MV-TCM registration, with KAZE presenting the highest accuracy and ORB being the most computationally efficient. The automated registration algorithm was shown to be efficient, performing registrations in 1.3 s on average and effective across a wide range of tool geometries and coating variations. The proposed tool reference line detection strategy, based on spatially aligned tool wear images, outperformed standard methods, resulting in average tool wear measurement errors of 2.5% and 4.5% in the turning and milling tests, respectively. Such a system allows machine tool operators to more efficiently capture cutting tool images while ensuring more reliable tool wear measurements.

Objective. Computational anatomical models have many applications in paediatric radiotherapy. Age-specific computational anatomical models were historically developed to represent average and/or healthy individuals, where cancer patients may present with anatomical variations caused by the disease and/or treatment effects. We developed RT-PAL, a library of computational age-specific voxelized anatomical models tailored to represent the paediatric radiotherapy population. Approach. Data from patients undergoing craniospinal irradiation (CSI) were used (n = 74, median age 7.3y, range: 1-17y). The RT-PAL phantoms were generated using groupwise deformable image registration to spatially normalize and average a sub-set of twenty clinical CTs and contours (n = 74, median age 7.7y, range: 3-14 y). To assess their anatomical and age-dependency plausibility, the RT-PAL models were compared against clinical cancer patient data and two healthy population based libraries of phantoms: the International Commission on Radiological Protection (ICRP) pediatric reference computational phantoms (n = 8, median age 7.5y, range: 1-15y) and a range of 4D paediatric extended cardiac torso (XCAT) phantoms (n = 75, median age 9.1y, range: 1-18y). For each dataset, nineteen organs were segmented on all age models to determine their volume. Each set was evaluated through a linear fit of organ volume with age, where comparisons were made relative to the linear fit of the clinical data. Main Results. Overall good anatomical plausibility was found for the RT-PAL phantoms. The age-dependency reported was comparable to both the clinical data and other phantoms, demonstrating their efficacy as a library of age-specific phantoms. Larger discrepancies with the clinical, ICRP and XCAT organ data were attributable to differences in organ filling, segmentation strategy and age distribution of the datasets, limitations of RT-PAL generation methodology, and/or possible anatomical differences between healthy and cancer populations. Significance. The RT-PAL models showed potential in representing the paediatric radiotherapy cohort, who are most likely to benefit from dedicated, age-specific anatomical phantoms., (Creative Commons Attribution license.)

Distraction osteogenesis (DO) is widely utilized for treating limb length discrepancy, nonunion, bone deformities and defects. This study sought to develop a 4D time-lapse morphometry method to quantify bone formation and resorption in mouse femur during DO based on image registration of longitudinal in vivo micro-CT scans. Female C57BL/6 mice (n = 7) underwent osteotomy, followed by 5 days of latency, 10 days of distraction and 35 days of consolidation. The mice were scanned with micro-CT at Days 5, 15, 25, 35, 45, and 50. Histological sectioning and Movat Pentachrome straining were performed at Day 50. After registration of two consecutive micro-CT images of the same bone (day x and day y), the spatially- and temporally-linked sequences of formation, resorption and quiescent bones at the distraction gap were identified and bone formation and resorption rates (BFR dayx-y and BRR dayx-y ) were calculated. The overall percentage error of the registration method was 2.98% ± 0.89% and there was a strong correlation between histologically-measured bone area fraction and micro-CT-determined bone volume fraction at Day 50 (r = 0.89, p < 0.05). The 4D time-lapse morphometry indicated a rapid bone formation during the first 10 days of the consolidation phase (BFR day15-25  = 0.14 ± 0.05 mm 3 /day), followed by callus reshaping via equivalent bone formation and resorption rates. The 4D time-lapse morphometry method developed in this study allows for a continuous quantitative monitoring of the dynamic process of bone formation and resorption following distraction, which may offer a better understanding of the mechanism for mechano-regulated bone regeneration and aid for development of new treatment strategies of DO., (© 2024 Orthopaedic Research Society.)

Nowadays, for detecting breast cancer in its early stages, the focus is on multispectral transmission imaging. Frame accumulation is a promising technique to enhance the grayscale level of the multispectral transmission images. Still, during the image acquisition process, human respiration or camera jitter causes the displacement of the frame's sequence which leads to the loss of accuracy and image quality of the frame accumulated image is reduced. In this article, we have proposed a new method named "repeated pair image registration and accumulation "to resolve the issue. In this method first pair of images from the sequence is first registered and accumulated followed by the next pair to be registered and accumulated. Then these two accumulated frames are registered and accumulated again. This process is repeated until all the frames from the sequence are processed and the final image is obtained. This method is tested on the sequence of breast frames taken at 600 nm, 620 nm, 670 nm, and 760 nm wavelength of light and proved the enhancement of quality, accuracy, and grayscale by various mathematical assessments. Furthermore, the processing time of our proposed method is very low because descent gradient optimization algorithm is used here for image registration purpose. This optimization algorithm has high speed as compared to other methods and is verified by registering a single image of each wavelength by three different methods. It has laid the foundations of early detection of breast cancer using multispectral transmission imaging., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024. Published by Elsevier B.V.)

Modeling and manufacturing of personalized cranial implants are important research areas that may decrease the waiting time for patients suffering from cranial damage. The modeling of personalized implants may be partially automated by the use of deep learning-based methods. However, this task suffers from difficulties with generalizability into data from previously unseen distributions that make it difficult to use the research outcomes in real clinical settings. Due to difficulties with acquiring ground-truth annotations, different techniques to improve the heterogeneity of datasets used for training the deep networks have to be considered and introduced. In this work, we present a large-scale study of several augmentation techniques, varying from classical geometric transformations, image registration, variational autoencoders, and generative adversarial networks, to the most recent advances in latent diffusion models. We show that the use of heavy data augmentation significantly increases both the quantitative and qualitative outcomes, resulting in an average Dice Score above 0.94 for the SkullBreak and above 0.96 for the SkullFix datasets. The results show that latent diffusion models combined with vector quantized variational autoencoder outperform other generative augmentation strategies. Moreover, we show that the synthetically augmented network successfully reconstructs real clinical defects, without the need to acquire costly and time-consuming annotations. The findings of the work will lead to easier, faster, and less expensive modeling of personalized cranial implants. This is beneficial to numerous people suffering from cranial injuries. The work constitutes a considerable contribution to the field of artificial intelligence in the automatic modeling of personalized cranial implants., Competing Interests: Declaration of competing interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Marek Wodzinski reports financial support was provided by National Centre for Research and Development. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 The Authors. Published by Elsevier Ltd.. All rights reserved.)

A deep understanding of neuron structure and function is crucial for elucidating brain mechanisms, diagnosing and treating diseases. Optical microscopy, pivotal in neuroscience, illuminates neuronal shapes, projections, and electrical activities. To explore the projection of specific functional neurons, scientists have been developing optical-based multimodal imaging strategies to simultaneously capture dynamic in vivo signals and static ex vivo structures from the same neuron. However, the original position of neurons is highly susceptible to displacement during ex vivo imaging, presenting a significant challenge for integrating multimodal information at the single-neuron level. This study introduces a graph-model-based approach for cell image matching, facilitating precise and automated pairing of sparsely labeled neurons across different optical microscopic images. It has been shown that utilizing neuron distribution as a matching feature can mitigate modal differences, the high-order graph model can address scale inconsistency, and the nonlinear iteration can resolve discrepancies in neuron density. This strategy was applied to the connectivity study of the mouse visual cortex, performing cell matching between the two-photon calcium image and the HD-fMOST brain-wide anatomical image sets. Experimental results demonstrate 96.67% precision, 85.29% recall rate, and 90.63% F1 Score, comparable to expert technicians. This study builds a bridge between functional and structural imaging, offering crucial technical support for neuron classification and circuitry analysis., Competing Interests: Declaration of competing interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Anan Li reports financial support was provided by National Natural Science Foundation of China. Anan Li reports financial support was provided by Ministry of Science and Technology of the People's Republic of China. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024. Published by Elsevier B.V.)

The lung is characterized by high elasticity and complex structure, which implies that the lung is capable of undergoing complex deformation and the shape variable is substantial. Large deformation estimation poses significant challenges to lung image registration. The traditional U-Net architecture is difficult to cover complex deformation due to its limited receptive field. Moreover, the relationship between voxels weakens as the number of downsampling times increases, that is, the long-range dependence issue. In this paper, we propose a novel multilevel registration framework which enhances the correspondence between voxels to improve the ability of estimating large deformations. Our approach consists of a convolutional neural network (CNN) with a two-stream registration structure and a cross-scale mapping attention (CSMA) mechanism. The former extracts the robust features of image pairs within layers, while the latter establishes frequent connections between layers to maintain the correlation of image pairs. This method fully utilizes the context information of different scales to establish the mapping relationship between low-resolution and high-resolution feature maps. We have achieved remarkable results on DIRLAB (TRE 1.56 ± 1.60) and POPI (NCC 99.72% SSIM 91.42%) dataset, demonstrating that this strategy can effectively address the large deformation issues, mitigate long-range dependence, and ultimately achieve more robust lung CT image registration., Competing Interests: Declaration of competing interest The authors declare no conflicts of interest to the manuscript content., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

We present a multi-stage and multi-resolution deformable image registration framework for image-guidance at a small animal proton irradiation platform. The framework is based on list-mode proton radiographies acquired at different angles, which are used to deform a 3D treatment planning CT relying on normalized mutual information (NMI) or root mean square error (RMSE) in the projection domain. We utilized a mouse X-ray micro-CT expressed in relative stopping power (RSP), and obtained Monte Carlo simulations of proton images in list-mode for three different treatment sites (brain, head and neck, lung). Rigid transformations and controlled artificial deformation were applied to mimic position misalignments, weight loss and breathing changes. Results were evaluated based on the residual RMSE of RSP in the image domain including the comparison of extracted local features, i.e. between the reference micro-CT and the one transformed taking into account the calculated deformation. The residual RMSE of the RSP showed that the accuracy of the registration framework is promising for compensating rigid (>97% accuracy) and non-rigid (∼95% accuracy) transformations with respect to a conventional 3D-3D registration. Results showed that the registration accuracy is degraded when considering the realistic detector performance and NMI as a metric, whereas the RMSE in projection domain is rather insensitive. This work demonstrates the pre-clinical feasibility of the registration framework on different treatment sites and its use for small animal imaging with a realistic detector. Further computational optimization of the framework is required to enable the use of this tool for online estimation of the deformation., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2023 The Author(s). Published by Elsevier GmbH.. All rights reserved.)

Background and Objective: Registration of pulmonary computed tomography (CT) images with radiation-induced lung diseases (RILD) was essential to investigate the voxel-wise relationship between the formation of RILD and the radiation dose received by different tissues. Although various approaches had been developed for the registration of lung CTs, their performances remained clinically unsatisfactory for registration of lung CT images with RILD. The main difficulties arose from the longitudinal change in lung parenchyma, including RILD and volumetric change of lung cancers, after radiation therapy, leading to inaccurate registration and artifacts caused by erroneous matching of the RILD tissues., Methods: To overcome the influence of the parenchymal changes, a divide-and-conquer approach rooted in the coherent point drift (CPD) paradigm was proposed. The proposed method was based on two kernel ideas. One was the idea of component structure wise registration. Specifically, the proposed method relaxed the intrinsic assumption of equal isotropic covariances in CPD by decomposing a lung and its surrounding tissues into component structures and independently registering the component structures pairwise by CPD. The other was the idea of defining a vascular subtree centered at a matched branch point as a component structure. This idea could not only provide a sufficient number of matched feature points within a parenchyma, but avoid being corrupted by the false feature points resided in the RILD tissues due to globally and indiscriminately sampling using mathematical operators. The overall deformation model was built by using the Thin Plate Spline based on all matched points., Results: This study recruited 30 pairs of lung CT images with RILD, 15 of which were used for internal validation (leave-one-out cross-validation) and the other 15 for external validation. The experimental results showed that the proposed algorithm achieved a mean and a mean of maximum 1 % of average surface distances <2 and 8 mm, respectively, and a mean and a maximum target registration error <2 mm and 5 mm on both internal and external validation datasets. The paired two-sample t-tests corroborated that the proposed algorithm outperformed a recent method, the Stavropoulou's method, on the external validation dataset (p < 0.05)., Conclusions: The proposed algorithm effectively reduced the influence of parenchymal changes, resulting in a reasonably accurate and artifact-free registration., Competing Interests: Declaration of competing interest The authors have no relevant financial or non-financial interests to disclose., (Copyright © 2024 Elsevier B.V. All rights reserved.)

Deep learning technologies have dramatically reshaped the field of medical image registration over the past decade. The initial developments, such as regression-based and U-Net-based networks, established the foundation for deep learning in image registration. Subsequent progress has been made in various aspects of deep learning-based registration, including similarity measures, deformation regularizations, network architectures, and uncertainty estimation. These advancements have not only enriched the field of image registration but have also facilitated its application in a wide range of tasks, including atlas construction, multi-atlas segmentation, motion estimation, and 2D-3D registration. In this paper, we present a comprehensive overview of the most recent advancements in deep learning-based image registration. We begin with a concise introduction to the core concepts of deep learning-based image registration. Then, we delve into innovative network architectures, loss functions specific to registration, and methods for estimating registration uncertainty. Additionally, this paper explores appropriate evaluation metrics for assessing the performance of deep learning models in registration tasks. Finally, we highlight the practical applications of these novel techniques in medical imaging and discuss the future prospects of deep learning-based image registration., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier B.V. All rights reserved.)

Purpose: The aim of this study is to develop a framework for quantitative analysis of longitudinal T2-weighted MRIs (T2w) following radiotherapy (RT) for prostate cancer., Materials and Methods: The developed methodology includes: (i) deformable image registration of longitudinal series to pre-RT T2w for automated detection of prostate, peripheral zone (PZ), and gross tumor volume (GTV); and (ii) T2w signal-intensity harmonization based on three reference tissues. The RE gistration and HARM onization ( REHARM ) framework was applied on T2w acquired in a clinical trial consisting of two pre-RT and three post-RT MRI exams. Image registration was assessed by the DICE coefficient between automatic and manual contours, and intensity normalization via inter-patient histogram intersection. Longitudinal consistency was evaluated by the repeatability coefficient and Pearson correlation ( r ) between the two T2w exams before RT., Results: T2w from 107 MRI exams (23 patients) were utilized. Following REHARM , the histogram intersections for prostate, PZ and GTV increased from median = 0.43/0.16/0.13 to 0.66/0.44/0.46. The repeatability in T2w intensity estimation was better for the automatic than the manual contours for all three regions of interest ( r  = 0.9, p  < 0.0001, for GTV). The changes in the tissues' T2w values pre- and post-RT became significant, indicating the measurable quantitative signal related to radiation., Conclusions: The developed methodology allows to automate longitudinal analysis reducing data acquisition-related variation and improving consistency. The quantitative characterization of RT-induced changes in T2w will lead to new understanding of radiation effects enabling prediction modeling of RT response., Competing Interests: The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: MA declares consulting or advisory Role at Varian Medical Systems and research funding from Varian Medical Systems (Inst) and Varian Medical Systems (Inst). ADP declares consulting or advisory role at Merck and Pfizer and research funding from Veracyte (Inst). AP declares a past role as co-chair of the RTOG/NRG GU Translational Research Program., (© 2024 The Author(s).)

Background: Image-guided surgical navigation systems are widely regarded as the benchmark for computer-assisted surgical robotic platforms, yet a persistent challenge remains in addressing intraoperative image drift and mismatch. It can significantly impact the accuracy and precision of surgical procedures. Therefore, further research and development are necessary to mitigate this issue and enhance the overall performance of these advanced surgical platforms., Objective: The primary objective is to improve the precision of image guided puncture navigation systems by developing a computed tomography (CT) and structured light imaging (SLI) based navigation system. Furthermore, we also aim to quantifying and visualize intraoperative image drift and mismatch in real time and provide feedback to surgeons, ensuring that surgical procedures are executed with accuracy and reliability., Methods: A CT-SLI guided orthopedic navigation puncture system was developed. Polymer bandages are employed to pressurize, plasticize, immobilize and toughen the surface of a specimen for surgical operations. Preoperative CT images of the specimen are acquired, a 3D navigation map is reconstructed and a puncture path planned accordingly. During surgery, an SLI module captures and reconstructs the 3D surfaces of both the specimen and a guiding tube for the puncture needle. The SLI reconstructed 3D surface of the specimen is matched to the CT navigation map via two-step point cloud registrations, while the SLI reconstructed 3D surface of the guiding tube is fitted by a cylindrical model, which is in turn aligned with the planned puncture path. The proposed system has been tested and evaluated using 20 formalin-soaked lower limb cadaver specimens preserved at a local hospital., Results: The proposed method achieved image registration RMS errors of 0.576 ± 0.146 mm and 0.407 ± 0.234 mm between preoperative CT and intraoperative SLI surface models and between preoperative and postoperative CT surface models. In addition, preoperative and postoperative specimen surface and skeletal drifts were 0.033 ± 0.272 mm and 0.235 ± 0.197 mm respectively., Conclusion: The results indicate that the proposed method is effective in reducing intraoperative image drift and mismatch. The system also visualizes intraoperative image drift and mismatch, and provides real time visual feedback to surgeons., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (© 2024 He, Xu, Zhang, Wang, Sun, Li, Liu, Tian, Huang and Cai.)

In this paper, a segmentation-based image fusion method is proposed for the fusion of MR and CT images to obtain a high contrast fused image that contains complementary information from both input images. The proposed method uses the fuzzy C-mean method to extract information about the skull from the CT image. This skull information is used to extract soft tissue information from the MR image. Both the skull information and the soft tissue information are then fused using the fusion rule. The efficiency of the proposed method over other state-of-the-art fusion methods is analyzed and compared using qualitative and quantitative analysis methods. Qualitative analysis shows the improvement in the contrast between the bone and the soft tissue using the proposed method over other state-of-the-art methods without introducing any artifacts or distortions. Classical and gradient-based quantitative analysis also show significant improvement in the fused image obtained using the proposed method over the five state-of-the-art methods. The percentage improvement in the standard deviation, average gradient, entropy, spatial frequency, QABF, and LABF of the proposed method over the best value obtained by the five state-of-the-art methods is 27.11%, 12.06%, 23.64%, 11.30%, 5.59%, and 13.70% respectively., (© 2024. The Author(s) under exclusive licence to Society for Imaging Informatics in Medicine.)

Annotation of multiple regions of interest across the whole mouse brain is an indispensable process for quantitative evaluation of a multitude of study endpoints in neuroscience digital pathology. Prior experience and domain expert knowledge are the key aspects for image annotation quality and consistency. At present, image annotation is often achieved manually by certified pathologists or trained technicians, limiting the total throughput of studies performed at neuroscience digital pathology labs. It may also mean that simpler and quicker methods of examining tissue samples are used by non-pathologists, especially in the early stages of research and preclinical studies. To address these limitations and to meet the growing demand for image analysis in a pharmaceutical setting, we developed AnNoBrainer, an open-source software tool that leverages deep learning, image registration, and standard cortical brain templates to automatically annotate individual brain regions on 2D pathology slides. Application of AnNoBrainer to a published set of pathology slides from transgenic mice models of synucleinopathy revealed comparable accuracy, increased reproducibility, and a significant reduction (~ 50%) in time spent on brain annotation, quality control and labelling compared to trained scientists in pathology. Taken together, AnNoBrainer offers a rapid, accurate, and reproducible automated annotation of mouse brain images that largely meets the experts' histopathological assessment standards (> 85% of cases) and enables high-throughput image analysis workflows in digital pathology labs., Competing Interests: Declarations. Competing Interests: All authors that are/were employees of Merck Sharp & Dohme LLC, a subsidiary of Merck & Co., Inc., Rahway, NJ, USA and may hold stocks and/or stock options in Merck & Co., Inc., Rahway, NJ, USA. A subset of manuscript authors are inventors on a patent related to this work (patent application number: 34187-56244). All of this work/code is licensed under the MIT permissive free software license., (© 2024. Merck & Co., Inc., Rahway, NJ, USA and its affiliates, 2024.)

Techniques are developed for generating uncertainty estimates for convolutional neural network (CNN)-based methods for registering the locations of lesions between the craniocaudal (CC) and mediolateral oblique (MLO) mammographic X-ray image views. Multi-view lesion correspondence is an important task that clinicians perform for characterizing lesions during routine mammographic exams. Automated registration tools can aid in this task, yet if the tools also provide confidence estimates, they can be of greater value to clinicians, especially in cases involving dense tissue where lesions may be difficult to see. A set of deep ensemble-based techniques, which leverage a negative log-likelihood (NLL)-based cost function, are implemented for estimating uncertainties. The ensemble architectures involve significant modifications to an existing CNN dual-view lesion registration algorithm. Three architectural designs are evaluated, and different ensemble sizes are compared using various performance metrics. The techniques are tested on synthetic X-ray data, real 2D X-ray data, and slices from real 3D X-ray data. The ensembles generate covariance-based uncertainty ellipses that are correlated with registration accuracy, such that the ellipse sizes can give a clinician an indication of confidence in the mapping between the CC and MLO views. The results also show that the ellipse sizes can aid in improving computer-aided detection (CAD) results by matching CC/MLO lesion detects and reducing false alarms from both views, adding to clinical utility. The uncertainty estimation techniques show promise as a means for aiding clinicians in confidently establishing multi-view lesion correspondence, thereby improving diagnostic capability., (© 2024. The Author(s).)

This paper presents a novel hybrid approach to feature detection designed specifically for enhancing Feature-Based Image Registration (FBIR). Through an extensive evaluation involving state-of-the-art feature detectors such as BRISK, FAST, ORB, Harris, MinEigen, and MSER, the proposed hybrid detector demonstrates superior performance in terms of keypoint detection accuracy and computational efficiency. Three image acquisition methods (i.e., rotation, scene-to-model, and scaling transformations) are considered in the comparison. Applied across a diverse set of remote-sensing images, the proposed hybrid approach has shown marked improvements in match points and match rates, proving its effectiveness in handling varied and complex imaging conditions typical in satellite and aerial imagery. The experimental results have consistently indicated that the hybrid detector outperforms conventional methods, establishing it as a valuable tool for advanced image registration tasks.

Optical and synthetic aperture radar (SAR) images exhibit non-negligible intensity differences due to their unique imaging mechanisms, which makes it difficult for classical SIFT-based algorithms to obtain sufficiently correct correspondences when processing the registration of these two types of images. To tackle this problem, an accurate optical and SAR image registration algorithm based on the SIFT algorithm (OS-PSO) is proposed. First, a modified ratio of exponentially weighted averages (MROEWA) operator is introduced to resolve the sudden dark patches in SAR images, thus generating more consistent gradients between optical and SAR images. Next, we innovatively construct the Harris scale space to replace the traditional difference in the Gaussian (DoG) scale space, identify repeatable key-points by searching for local maxima, and perform localization refinement on the identified key-points to improve their accuracy. Immediately after that, the gradient location orientation histogram (GLOH) method is adopted to construct the feature descriptors. Finally, we propose an enhanced matching method. The transformed relation is obtained in the initial matching stage using the nearest neighbor distance ratio (NNDR) and fast sample consensus (FSC) methods. And the re-matching takes into account the location, scale, and main direction of key-points to increase the number of correctly corresponding points. The proposed OS-PSO algorithm has been implemented on the Gaofen and Sentinel series with excellent results. The superior performance of the designed registration system can also be applied in complex scenarios, including urban, suburban, river, farmland, and lake areas, with more efficiency and accuracy than the state-of-the-art methods based on the WHU-OPT-SAR dataset and the BISTU-OPT-SAR dataset.

Background and Purpose: Image-based data mining (IBDM) requires spatial normalisation to reference anatomy, which is challenging in breast radiotherapy due to variations in the treatment position, breast shape and volume. We aim to optimise spatial normalisation for breast IBDM., Materials and Methods: Data from 996 patients treated with radiotherapy for early-stage breast cancer, recruited in the REQUITE study, were included. Patients were treated supine (n = 811), with either bilateral or ipsilateral arm(s) raised (551/260, respectively) or in prone position (n = 185). Four deformable image registration (DIR) configurations for extrathoracic spatial normalisation were tested. We selected the best-performing DIR configuration and further investigated two pathways: i ) registering prone/supine cohorts independently and ii ) registering all patients to a supine reference. The impact of arm positioning in the supine cohort was quantified. DIR accuracy was estimated using Normalised Cross Correlation (NCC), Dice Similarity Coefficient (DSC), mean Distance to Agreement (MDA), 95 % Hausdorff Distance (95 %HD), and inter-patient landmark registration uncertainty (ILRU)., Results: DIR using B-spline and normalised mutual information (NMI) performed the best across all evaluation metrics. Supine-supine registrations yielded highest accuracy (0.98 ± 0.01, 0.91 ± 0.04, 0.23 ± 0.19 cm, 1.17 ± 1.18 cm, 0.51 ± 0.26 cm for NCC, DSC, MDA, 95 %HD, and ILRU), followed by prone-prone and supine-prone registrations. Arm positioning had no significant impact on registration performance. For the best DIR strategy, uncertainty of 0.44 and 0.81 cm in the breast and shoulder regions was found., Conclusions: B-spline algorithm using NMI and registered supine and prone cohorts independently provides the most optimal spatial normalisation strategy for breast IBDM., Competing Interests: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (© 2024 Published by Elsevier B.V. on behalf of European Society of Radiotherapy & Oncology.)

Purpose: AI-assisted techniques for lesion registration and segmentation have the potential to make CT-based tumor follow-up assessment faster and less reader-dependent. However, empirical evidence on the advantages of AI-assisted volumetric segmentation for lymph node and soft tissue metastases in follow-up CT scans is lacking. The aim of this study was to assess the efficiency, quality, and inter-reader variability of an AI-assisted workflow for volumetric segmentation of lymph node and soft tissue metastases in follow-up CT scans. Three hypotheses were tested: (H1) Assessment time for follow-up lesion segmentation is reduced using an AI-assisted workflow. (H2) The quality of the AI-assisted segmentation is non-inferior to the quality of fully manual segmentation. (H3) The inter-reader variability of the resulting segmentations is reduced with AI assistance., Materials and Methods: The study retrospectively analyzed 126 lymph nodes and 135 soft tissue metastases from 55 patients with stage IV melanoma. Three radiologists from two institutions performed both AI-assisted and manual segmentation, and the results were statistically analyzed and compared to a manual segmentation reference standard., Results: AI-assisted segmentation reduced user interaction time significantly by 33% (222 s vs. 336 s), achieved similar Dice scores (0.80-0.84 vs. 0.81-0.82) and decreased inter-reader variability (median Dice 0.85-1.0 vs. 0.80-0.82; ICC 0.84 vs. 0.80), compared to manual segmentation., Conclusion: The findings of this study support the use of AI-assisted registration and volumetric segmentation for lymph node and soft tissue metastases in follow-up CT scans. The AI-assisted workflow achieved significant time savings, similar segmentation quality, and reduced inter-reader variability compared to manual segmentation., (© 2024. The Author(s).)

Purpose: Resecting the posterior cruciate ligament (PCL) increases posterior laxity and increases the flexion gap more than the extension gap in the native (i.e. healthy) knee. These two effects could lead to significant anterior displacement of the medial femoral condyle in kneeling following total knee arthroplasty even when using a tibial insert with a high degree of medial conformity. Using an insert with ball-in-socket medial conformity and a flat lateral articular surface, the primary purpose was to determine whether the medial femoral condyle remained stable with and without PCL retention during kneeling., Methods: Two groups of patients were studied, one with PCL retention (22 patients) and the other with PCL resection (25 patients), while kneeling at 90º flexion. Following 3D model-to-2D image registration, A-P displacements of both femoral condyles were determined relative to the dwell point of the medial socket., Results: With PCL resection versus PCL retention, the medial femoral condyle was 5.1 ± 3.7 mm versus 0.8 ± 2.1 mm anterior of the dwell point (p < 0.0001). Patient-reported function scores were comparable (p ≥ 0.1610) despite a significantly shorter follow-up of 7.8 ± 0.9 months with PCL retention than 19.6 ± 4.9 months with PCL resection (p < 0.0001). Range of motion was 126 ± 8° versus 122 ± 6° with and without PCL retention, respectively (p = 0.057)., Conclusion: Surgeons that use a highly conforming tibial insert design can stabilize the medial femoral condyle during kneeling by retaining the PCL. In patients with PCL resection, the 9 mm high anterior lip of the insert with ball-in-socket medial conformity was insufficient to prevent significant anterior displacement of the medial femoral condyle when weight-bearing on the anterior tibia., (© 2024. The Author(s) under exclusive licence to SICOT aisbl.)

The assessment of deformable registration uncertainty is an important task for the safety and reliability of registration methods in clinical applications. However, it is typically done by a manual and time-consuming procedure. We propose a novel automatic method to predict registration uncertainty based on multi-category features and supervised learning. Three types of features, including deformation field statistical features, deformation field physiologically realistic features, and image similarity features, are introduced and calculated to train the random forest regressor for local registration uncertain prediction. Deformation field statistical features represent the numerical stability of registration optimization, which are correlated to the uncertainty of deformation fields; deformation field physiologically realistic features represent the biomechanical properties of organ motions, which mathematically reflect the physiological reality of deformation; image similarity features reflect the similarity between the warped image and fixed image. The multi-category features comprehensively reflect the registration uncertainty. The strategy of spatial adaptive random perturbations is also introduced to accurately simulate spatial distribution of registration uncertainty, which makes deformation field statistical features more discriminative to the uncertainty of deformation fields. Experiments were conducted on three publicly available thoracic CT image datasets. Seventeen randomly selected image pairs are used to train the random forest model, and 9 image pairs are used to evaluate the prediction model. The quantitative experiments on lung CT images show that the proposed method outperforms the baseline method for uncertain prediction of classical iterative optimization-based registration and deep learning-based registration with different registration qualities. The proposed method achieves good performance for registration uncertain prediction, which has great potential in improving the accuracy of registration uncertain prediction., (© 2024. International Federation for Medical and Biological Engineering.)

Introduction: Frontotemporal lobar degeneration (FTLD) is associated with FTLD due to tau (FTLD-tau) or TDP (FTLD-TDP) inclusions found at autopsy. Arterial Spin Labeling (ASL) MRI is often acquired in the same session as a structural T1-weighted image (T1w), enabling detection of regional changes in cerebral blood flow (CBF). We hypothesize that ASL-T1w registration with more degrees of freedom using boundary-based registration (BBR) will better align ASL and T1w images and show increased sensitivity to regional hypoperfusion differences compared to manual registration in patient participants. We hypothesize that hypoperfusion will be associated with a clinical measure of disease severity, the FTLD-modified clinical dementia rating scale sum-of-boxes (FTLD-CDR)., Materials and Methods: Patients with sporadic likely FTLD-tau (sFTLD-tau; N  = 21), with sporadic likely FTLD-TDP (sFTLD-TDP; N  = 14), and controls ( N  = 50) were recruited from the Connectomic Imaging in Familial and Sporadic Frontotemporal Degeneration project (FTDHCP). Pearson's Correlation Coefficients (CC) were calculated on cortical vertex-wise CBF between each participant for each of 3 registration methods: (1) manual registration, (2) BBR initialized with manual registration (manual+BBR), (3) and BBR initialized using FLIRT (FLIRT+BBR). Mean CBF was calculated in the same regions of interest (ROIs) for each registration method after image alignment. Paired t -tests of CC values for each registration method were performed to compare alignment. Mean CBF in each ROI was compared between groups using t -tests. Differences were considered significant at p  < 0.05 (Bonferroni-corrected). We performed linear regression to relate FTLD-CDR to mean CBF in patients with sFTLD-tau and sFTLD-TDP, separately ( p  < 0.05, uncorrected)., Results: All registration methods demonstrated significant hypoperfusion in frontal and temporal regions in each patient group relative to controls. All registration methods detected hypoperfusion in the left insular cortex, middle temporal gyrus, and temporal pole in sFTLD-TDP relative to sFTLD-tau. FTLD-CDR had an inverse association with CBF in right temporal and orbitofrontal ROIs in sFTLD-TDP. Manual+BBR performed similarly to FLIRT+BBR., Discussion: ASL is sensitive to distinct regions of hypoperfusion in patient participants relative to controls, and in patients with sFTLD-TDP relative to sFTLD-tau, and decreasing perfusion is associated with increasing disease severity, at least in sFTLD-TDP. BBR can register ASL-T1w images adequately for controls and patients., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision., (Copyright © 2024 Mihailescu, Hlava, Cook, Mandelli, Lee, Boeve, Dickerson, Gorno-Tempini, Rogalski, Grossman, Gee, McMillan and Olm.)

Medical image registration has become pivotal in recent years with the integration of various imaging modalities like X-ray, ultrasound, MRI, and CT scans, enabling comprehensive analysis and diagnosis of biological structures. This paper provides a comprehensive review of registration techniques for medical images, with an in-depth focus on 2D-2D image registration methods. While 3D registration is briefly touched upon, the primary emphasis remains on 2D techniques and their applications. This review covers registration techniques for diverse modalities, including unimodal, multimodal, interpatient, and intra-patient. The paper explores the challenges encountered in medical image registration, including geometric distortion, differences in image properties, outliers, and optimization convergence, and discusses their impact on registration accuracy and reliability. Strategies for addressing these challenges are highlighted, emphasizing the need for continual innovation and refinement of techniques to enhance the accuracy and reliability of medical image registration systems. The paper concludes by emphasizing the importance of accurate medical image registration in improving diagnosis.

Background: Disease or injury may cause a change in the biomechanical properties of the lungs, which can alter lung function. Image registration can be used to measure lung ventilation and quantify volume change, which can be a useful diagnostic aid. However, lung registration is a challenging problem because of the variation in deformation along the lungs, sliding motion of the lungs along the ribs, and change in density., Purpose: Landmark correspondences have been used to make deformable image registration robust to large displacements., Methods: To tackle the challenging task of intra-patient lung computed tomography (CT) registration, we extend the landmark correspondence prediction model deep convolutional neural network-Match by introducing a soft mask loss term to encourage landmark correspondences in specific regions and avoid the use of a mask during inference. To produce realistic deformations to train the landmark correspondence model, we use data-driven synthetic transformations. We study the influence of these learned landmark correspondences on lung CT registration by integrating them into intensity-based registration as a distance-based penalty., Results: Our results on the public thoracic CT dataset COPDgene show that using learned landmark correspondences as a soft constraint can reduce median registration error from approximately 5.46 to 4.08 mm compared to standard intensity-based registration, in the absence of lung masks., Conclusions: We show that using landmark correspondences results in minor improvements in local alignment, while significantly improving global alignment., (© 2024 The Authors. Medical Physics published by Wiley Periodicals LLC on behalf of American Association of Physicists in Medicine.)

Variational Autoencoders (VAEs) are an efficient variational inference technique coupled with the generated network. Due to the uncertainty provided by variational inference, VAEs have been applied in medical image registration. However, a critical problem in VAEs is that the simple prior cannot provide suitable regularization, which leads to the mismatch between the variational posterior and prior. An optimal prior can close the gap between the evidence's real and variational posterior. In this paper, we propose a multi-stage VAE to learn the optimal prior, which is the aggregated posterior. A lightweight VAE is used to generate the aggregated posterior as a whole. It is an effective way to estimate the distribution of the high-dimensional aggregated posterior that commonly exists in medical image registration based on VAEs. A factorized telescoping classifier is trained to estimate the density ratio of a simple given prior and aggregated posterior, aiming to calculate the KL divergence between the variational and aggregated posterior more accurately. We analyze the KL divergence and find that the finer the factorization, the smaller the KL divergence is. However, too fine a partition is not conducive to registration accuracy. Moreover, the diagonal hypothesis of the variational posterior's covariance ignores the relationship between latent variables in image registration. To address this issue, we learn a covariance matrix with low-rank information to enable correlations with each dimension of the variational posterior. The covariance matrix is further used as a measure to reduce the uncertainty of deformation fields. Experimental results on four public medical image datasets demonstrate that our proposed method outperforms other methods in negative log-likelihood (NLL) and achieves better registration accuracy., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

Purpose: Electromagnetic tracking (EMT) has great potential as a quality assurance tool in interstitial brachytherapy. Since its clinical application in most cases comprises a comparison with brachytherapy plan data, EMT registration and plan data are crucial. Registration uncertainties influence EMT outcomes and further decision-making processes. The impact of registration using four algorithms on the resulting data was investigated., Material and Methods: Registration abilities of four sub-algorithms, which share the same principle of an iterative closest point approach were considered. Sub-algorithms differ regarding underlying mathematical norms, and also include a non-geometric implementation focusing on stochastic optimization. This study focused on the registration of point clouds originating from data obtained during treatment with interstitial brachytherapy in breast cancer patients. Data were acquired from a hybrid device prototype, capable of EMT measurements and data extracted from treatment plan. Main criteria for good registration performance were good geometric agreement, including low Euclidean distances between registered data sets as well as short registration time. In addition, previously published patient data with different than employed in the original publication algorithm were re-analyzed to further investigate the impact of registration method on clinically influential results., Results: The stochastic algorithm presented much higher registration times, and earliest implementation of geometric approaches occasionally misaligned patient data. The remaining two algorithms performed almost identically resulted in sufficiently accurate registration performance. The re-analysis of previously published data demonstrated minor inconsistencies between former registration method and different algorithm; however, the overall trends with almost identical mean Euclidean distance of 2.22 mm (2.19 mm in the original work) could be reproduced., Conclusions: The choice of registration algorithm can influence the quality of registration. Therefore, it is advised to regularly examine the registration performance, especially in in-house programs., Competing Interests: The authors declare no conflict of interest., (Copyright © 2024 Termedia.)

Objective. Head and neck cancer patients experience systematic as well as random day to day anatomical changes during fractionated radiotherapy treatment. Modelling the expected systematic anatomical changes could aid in creating treatment plans which are more robust against such changes. Approach. Inter- patient correspondence aligned all patients to a model space. Intra- patient correspondence between each planning CT scan and on treatment cone beam CT scans was obtained using diffeomorphic deformable image registration. The stationary velocity fields were then used to develop B-Spline based patient specific (SM) and population average (AM) models. The models were evaluated geometrically and dosimetrically. A leave-one-out method was used to compare the training and testing accuracy of the models. Main results. Both SMs and AMs were able to capture systematic changes. The average surface distance between the registration propagated contours and the contours generated by the SM was less than 2 mm, showing that the SM are able to capture the anatomical changes which a patient experiences during the course of radiotherapy. The testing accuracy was lower than the training accuracy of the SM, suggesting that the model overfits to the limited data available and therefore, also captures some of the random day to day changes. For most patients the AMs were a better estimate of the anatomical changes than assuming there were no changes, but the AMs could not capture the variability in the anatomical changes seen in all patients. No difference was seen in the training and testing accuracy of the AMs. These observations were highlighted in both the geometric and dosimetric evaluations and comparisons. Significance. In this work, a SM and AM are presented which are able to capture the systematic anatomical changes of some head and neck cancer patients over the course of radiotherapy treatment. The AM is able to capture the overall trend of the population, but there is large patient variability which highlights the need for more complex, capable population models., (Creative Commons Attribution license.)

We introduce a novel AI-driven approach to unsupervised fundus image registration utilizing our Generalized Polynomial Transformation (GPT) model. Through the GPT, we establish a foundational model capable of simulating diverse polynomial transformations, trained on a large synthetic dataset to encompass a broad range of transformation scenarios. Additionally, our hybrid pre-processing strategy aims to streamline the learning process by offering model-focused input. We evaluated our model's effectiveness on the publicly available AREDS dataset by using standard metrics such as image-level and parameter-level analyzes. Linear regression analysis reveals an average Pearson correlation coefficient (R) of 0.9876 across all quadratic transformation parameters. Image-level evaluation, comprising qualitative and quantitative analyzes, showcases significant improvements in Structural Similarity Index (SSIM) and Normalized Cross Correlation (NCC) scores, indicating its robust performance. Notably, precise matching of the optic disc and vessel locations with minimal global distortion are observed. These findings underscore the potential of GPT-based approaches in image registration methodologies, promising advancements in diagnosis, treatment planning, and disease monitoring in ophthalmology and beyond., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision., (Copyright © 2024 Chen, Fan, Meng and Zheng.)

Registering longitudinal infant brain images is challenging, as the infant brain undergoes rapid changes in size, shape and tissue contrast in the first months and years of life. Diffusion tensor images (DTI) have relatively consistent tissue properties over the course of infancy compared to commonly used T1 or T2-weighted images, presenting great potential for infant brain registration. Moreover, groupwise registration has been widely used in infant neuroimaging studies to reduce bias introduced by predefined atlases that may not be well representative of samples under study. To date, however, no methods have been developed for groupwise registration of tensor-based images. Here, we propose a novel registration approach to groupwise align longitudinal infant DTI images to a sample-specific common space. Longitudinal infant DTI images are first clustered into more homogenous subgroups based on image similarity using Louvain clustering. DTI scans are then aligned within each subgroup using standard tensor-based registration. The resulting images from all subgroups are then further aligned onto a sample-specific common space. Results show that our approach significantly improved registration accuracy both globally and locally compared to standard tensor-based registration and standard fractional anisotropy-based registration. Additionally, clustering based on image similarity yielded significantly higher registration accuracy compared to no clustering, but comparable registration accuracy compared to clustering based on chronological age. By registering images groupwise to reduce registration bias and capitalizing on the consistency of features in tensor maps across early infancy, our groupwise registration framework facilitates more accurate alignment of longitudinal infant brain images., Competing Interests: Competing interests The authors report no competing interests.

Synthetic aperture radar (SAR) image registration is an important process in many applications, such as image stitching and remote sensing surveillance. The registration accuracy is commonly affected by the presence of speckle noise in SAR images. When speckle noise is intense, the number of image features acquired by single-feature-based methods is insufficient. An SAR image registration method that combines nonlinear diffusion filtering, Hessian features and edge points is proposed in this paper to reduce speckle noise and obtain more image features. The proposed method uses the infinite symmetric exponential filter (ISEF) for image pre-processing and nonlinear diffusion filtering for scale-space construction. These measures can remove speckle noise from SAR images while preserving image edges. Hessian features and edge points are also employed as image features to optimize the utilization of feature information. Experiments with different noise levels, geometric transformations and image scenes demonstrate that the proposed method effectively improves the accuracy of SAR image registration compared with the SIFT-OCT, SAR-SIFT, Harris-SIFT, NF-Hessian and KAZE-SAR algorithms.

The concept of Image Phase Congruency (IPC) is deeply rooted in the way the human visual system interprets and processes spatial frequency information. It plays an important role in visual perception, influencing our capacity to identify objects, recognize textures, and decipher spatial relationships in our environments. IPC is robust to changes in lighting, contrast, and other variables that might modify the amplitude of light waves yet leave their relative phase unchanged. This characteristic is vital for perceptual tasks as it ensures the consistent detection of features regardless of fluctuations in illumination or other environmental factors. It can also impact cognitive and emotional responses; cohesive phase information across elements fosters a perception of unity or harmony, while inconsistencies can engender a sense of discord or tension. In this survey, we begin by examining the evidence from biological vision studies suggesting that IPC is employed by the human perceptual system. We proceed to outline the typical mathematical representation and different computational approaches to IPC. We then summarize the extensive applications of IPC in computer vision, including denoise, image quality assessment, feature detection and description, image segmentation, image registration, image fusion, and object detection, among other uses, and illustrate its advantages with a number of examples. Finally, we discuss the current challenges associated with the practical applications of IPC and potential avenues for enhancement.

A fast scanner of optical-resolution photoacoustic microscopy is inherently vulnerable to perturbation, leading to severe image distortion and significant misalignment among multiple 2D or 3D images. Restoration and registration of these images is critical for accurately quantifying dynamic information in long-term imaging. However, traditional registration algorithms face a great challenge in computational throughput. Here, we develop an unsupervised deep learning based registration network to achieve real-time image restoration and registration. This method can correct artifacts from B-scan distortion and remove misalignment among adjacent and repetitive images in real time. Compared with conventional intensity based registration algorithms, the throughput of the developed algorithm is improved by 50 times. After training, the new deep learning method performs better than conventional feature based image registration algorithms. The results show that the proposed method can accurately restore and register the images of fast-scanning photoacoustic microscopy in real time, offering a powerful tool to extract dynamic vascular structural and functional information., Competing Interests: The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (© 2024 The Authors. Published by Elsevier GmbH.)

Lung injuries, such as ventilator-induced lung injury and radiation-induced lung injury, can lead to heterogeneous alterations in the biomechanical behavior of the lungs. While imaging methods, e.g., X-ray and static computed tomography (CT), can point to regional alterations in lung structure between healthy and diseased tissue, they fall short of delineating timewise kinematic variations between the former and the latter. Image registration has gained recent interest as a tool to estimate the displacement experienced by the lungs during respiration via regional deformation metrics such as volumetric expansion and distortion. However, successful image registration commonly relies on a temporal series of image stacks with small displacements in the lungs across succeeding image stacks, which remains limited in static imaging. In this study, we have presented a finite element (FE) method to estimate strains from static images acquired at the end-expiration (EE) and end-inspiration (EI) timepoints, i.e., images with a large deformation between the two distant timepoints. Physiologically realistic loads were applied to the geometry obtained at EE to deform this geometry to match the geometry obtained at EI. The results indicated that the simulation could minimize the error between the two geometries. Using four-dimensional (4D) dynamic CT in a rat, the strain at an isolated transverse plane estimated by our method showed sufficient agreement with that estimated through non-rigid image registration that used all the timepoints. Through the proposed method, we can estimate the lung deformation at any timepoint between EE and EI. The proposed method offers a tool to estimate timewise regional deformation in the lungs using only static images acquired at EE and EI.

The vector forces at the human-mattress interface are not only crucial for understanding the distribution of vertical and shear forces exerted on the human body during sleep but also serves as a significant input for biomechanical models of sleeping positions, whose accuracy determines the credibility of predicting musculoskeletal system loads. In this study, we introduce a novel method for calculating the interface vector forces. By recording indentations after supine and lateral positions using a vacuum mattress and 3D scanner, we utilize image registration techniques to align body pressure distribution with the mattress deformation scanning images, thereby calculating the vector force values for each unit area (36.25 mm × 36.25 mm). This method was validated through five participants attendance from two perspectives, revealing that (1) the mean summation of the vertical force components is 98.67% ± 7.21% body weight, exhibiting good consistency, and mean ratio of horizontal component force to body weight is 2.18% ± 1.77%. (2) the predicted muscle activity using the vector forces as input to the sleep position model aligns with the measured muscle activity (%MVC), with correlation coefficient over 0.7. The proposed method contributes to the vector force distribution understanding and the analysis of musculoskeletal loads during sleep, providing valuable insights for mattress design and evaluation., (© 2024. The Author(s).)

Purpose: To investigate geometric and dosimetric inter-observer variability in needle reconstruction for temporary prostate brachytherapy. To assess the potential of registrations between transrectal ultrasound (TRUS) and cone-beam computed tomography (CBCT) to support implant reconstructions., Methods and Materials: The needles implanted in 28 patients were reconstructed on TRUS by three physicists. Corresponding geometric deviations and associated dosimetric variations to prostate and organs at risk (urethra, bladder, rectum) were analyzed. To account for the found inter-observer variability, various approaches (template-based, probe-based, marker-based) for registrations of CBCT to TRUS were investigated regarding the respective needle transfer accuracy in a phantom study. Three patient cases were examined to assess registration accuracy in-vivo., Results: Geometric inter-observer deviations >1 mm and >3 mm were found for 34.9% and 3.5% of all needles, respectively. Prostate dose coverage (changes up to 7.2%) and urethra dose (partly exceeding given dose constraints) were most affected by associated dosimetric changes. Marker-based and probe-based registrations resulted in the phantom study in high mean needle transfer accuracies of 0.73 mm and 0.12 mm, respectively. In the patient cases, the marker-based approach was the superior technique for CBCT-TRUS fusions., Conclusion: Inter-observer variability in needle reconstruction can substantially affect dosimetry for individual patients. Especially marker-based CBCT-TRUS registrations can help to ensure accurate reconstructions for improved treatment planning., (Copyright © 2024 The Authors. Published by Elsevier Inc. All rights reserved.)

The dynamics of DNA in the cell nucleus plays a role in cellular processes and fates but the interplay of DNA mobility with the hierarchical levels of DNA organization is still underexplored. Here, we made use of DNA replication to directly label genomic DNA in an unbiased genome-wide manner. This was followed by live-cell time-lapse microscopy of the labeled DNA combining imaging at different resolutions levels simultaneously and allowing one to trace DNA motion across organization levels within the same cells. Quantification of the labeled DNA segments at different microscopic resolution levels revealed sizes comparable to the ones reported for DNA loops using 3D super-resolution microscopy, topologically associated domains (TAD) using 3D widefield microscopy, and also entire chromosomes. By employing advanced chromatin tracking and image registration, we discovered that DNA exhibited higher mobility at the individual loop level compared to the TAD level and even less at the chromosome level. Additionally, our findings indicate that chromatin movement, regardless of the resolution, slowed down during the S phase of the cell cycle compared to the G1/G2 phases. Furthermore, we found that a fraction of DNA loops and TADs exhibited directed movement with the majority depicting constrained movement. Our data also indicated spatial mobility differences with DNA loops and TADs at the nuclear periphery and the nuclear interior exhibiting lower velocity and radius of gyration than the intermediate locations. On the basis of these insights, we propose that there is a link between DNA mobility and its organizational structure including spatial distribution, which impacts cellular processes., (© 2024. The Author(s).)

Purpose: Magnetic resonance imaging (MRI) is a common technique in image-guided neurosurgery (IGN). Recent research explores the integration of methods like ultrasound and tomography, among others, with hyperspectral (HS) imaging gaining attention due to its non-invasive real-time tissue classification capabilities. The main challenge is the registration process, often requiring manual intervention. This work introduces an automatic, markerless method for aligning HS images with MRI., Methods: This work presents a multimodal system that combines RGB-Depth (RGBD) and HS cameras. The RGBD camera captures the patient's facial geometry, which is used for registration with the preoperative MR through ICP. Once MR-depth registration is complete, the integration of HS data is achieved using a calibrated homography transformation. The incorporation of external tracking with a novel calibration method allows camera mobility from the registration position to the craniotomy area. This methodology streamlines the fusion of RGBD, HS and MR images within the craniotomy area., Results: Using the described system and an anthropomorphic phantom head, the system has been characterised by registering the patient's face in 25 positions and 5 positions resulted in a fiducial registration error of 1.88 ± 0.19 mm and a target registration error of 4.07 ± 1.28 mm, respectively., Conclusions: This work proposes a new methodology to automatically register MR and HS information with a sufficient accuracy. It can support the neurosurgeons to guide the diagnosis using multimodal data over an augmented reality representation. However, in its preliminary prototype stage, this system exhibits significant promise, driven by its cost-effectiveness and user-friendly design., (© 2024. The Author(s).)