Search

Deformable image registration (DIR) is an important component of a patient’s radiation therapy treatment. During the planning stage it combines complementary information from different imaging modalities and time points. During treatment, it aligns the patient to a reproducible position for accurate dose delivery. As the treatment progresses, it can inform clinicians of important changes in anatomy which trigger plan adjustment. And finally, after the treatment is complete, registering images at subsequent time points can help to monitor the patient’s health. The body’s natural non-rigid motion makes DIR a complex challenge. Recently neural networks have shown impressive improvements in image processing and have been leveraged for DIR tasks. This thesis is a compilation of neural network-based approaches addressing lingering issues in medical DIR, namely 1) multi-modality registration, 2) registration with different scan extents, and 3) modeling large motion in registration. For the first task we employed a cycle consistent generative adversarial network to translate images in the MRI domain to the CT domain, such that the moving and target images were in a common domain. DIR could then proceed as a synthetically bridged mono-modality registration. The second task used advances in network-based inpainting to artificially extend images beyond their scan extent. The third task leveraged axial self-attention networks’ ability to learn long range interactions to predict the deformation in the presence of large motion. For all these studies we used images from the head and neck, which exhibit all of these challenges, although these results can be generalized to other parts of the anatomy.The results of our experiments yielded networks that showed significant improvements in multi-modal DIR relative to traditional methods. We also produced network which can successfully predict missing tissue and demonstrated a DIR workflow that is independent of scan length. Finally, we trained a network whose accuracy is a balance between large and small motion prediction, and which opens the door to non-convolution-based DIR. By leveraging the power of artificial intelligence, we demonstrate a new paradigm in deformable image registration. Neural networks learn new patterns and connections in imaging data which go beyond the hand-crafted features of traditional image processing. This thesis shows how each step of registration, from the image pre-processing to the registration itself, can benefit from this exciting and cutting-edge approach.

Mammography is the current standard imaging method for detecting breast cancer by using x-rays to produce 2D images of the breast. However, with mammography alone there is difficulty determining whether a lesion is benign or malignant and reduced sensitivity to detecting lesions in dense breasts. Ultrasound imaging used in conjunction with mammography has shown valuable contributions for lesion characterization by differentiating between solid and cystic lesions. Conventional breast ultrasound has high false positive rates; however, it has shown improved abilities to detect lesions in dense breasts. Breast ultrasound is typically performed freehand to produce anterior-to-posterior 2D images in a different geometry (supine) than mammography (upright). This difference in geometries is likely responsible for the finding that at least 10% of the time lesions found in the ultrasound images do not correspond with lesions found in mammograms. To solve this problem additional imaging techniques must be investigated to aid a radiologist in identifying corresponding lesions in the two modalities to ensure early detection of a potential cancer. This dissertation describes and validates automated deformable mapping methods to register and relate corresponding lesions between multi-modality images acquired using 3D mammography (Digital Breast Tomosynthesis (DBT) and dedicated breast Computed Tomography (bCT)) and 3D ultrasound (Automated Breast Ultrasound (ABUS)). The methodology involves the use of finite element modeling and analysis to simulate the differences in compression and breast orientation to better align lesions acquired from images from these modalities. Preliminary studies were performed using several multimodality compressible breast phantoms to determine breast lesion registrations between: i) cranio-caudal (CC) and mediolateral oblique (MLO) DBT views and ABUS, ii) simulated bCT and DBT (CC and MLO views), and iii) simulated bCT and ABUS. Distances between the centers of masses, dCOM, of corresponding lesions were used to assess the deformable mapping method. These phantom studies showed the potential to apply this technique for real breast lesions with mean dCOM registration values as low as 4.9 ± 2.4 mm for DBT (CC view) mapped to ABUS, 9.3 ± 2.8 mm for DBT (MLO view) mapped to ABUS, 4.8 ± 2.4 mm for bCT mapped to ABUS, 5.0 ± 2.2 mm for bCT mapped to DBT (CC view), and 4.7 ± 2.5 mm for bCT mapped to DBT (MLO view). All of the phantom studies showed that using external fiducial markers helped improve the registration capability of the deformable mapping algorithm. An IRB-approved proof-of-concept study was performed with patient volunteers to validate the deformable registration method on 5 patient datasets with a total of up to 7 lesions for DBT (CC and MLO views) to ABUS registration. Resulting dCOM’s using the deformable method showed statistically significant improvements over rigid registration techniques with a mean dCOM of 11.6 ± 5.3 mm for DBT (CC view) mapped to ABUS and a mean dCOM of 12.3 ± 4.8 mm for DBT (MLO view) mapped to ABUS. The present work demonstrates the potential for using deformable registration techniques to relate corresponding lesions in 3D x-ray and 3D ultrasound images. This methodology should improve a radiologists’ characterization of breast lesions which can reduce patient callbacks, misdiagnoses, additional patient dose and unnecessary biopsies. Additionally, this technique can save a radiologist time in navigating 3D image volumes and the one-to-one lesion correspondence between modalities can aid in the early detection of breast malignancies.

Prostate cancer (PCa) is a prevalent disease which affects 1 in 6 men in the United States and has overtaken lung cancer as the leading cause of cancer related deaths in American men and number two worldwide. Among several diagnostic imaging tests that are available for detection of PCa in the market today, Magnetic Resonance Imaging (MRI) occupies a unique position in the detection of PCa due to its excellent soft tissue contrast and its ability to generate tissue property dependent multi-parametric data. While MRI has become an increasingly valuable tool in the management of men with PCa, its use to identify aggressive disease and characterize extent have yet to be developed. Multi-parametric MRI (MP-MRI) studies have been shown to increase sensitivity and specificity towards PCa detection compared to any single MRI dataset. The ability to develop and evaluate MP-MRI to prospectively detect disease, assess aggressiveness and delineate extent, first requires the retrospective validation against post-surgical pathology sections. Despite the large effort made by many groups in this area of research, the correlation of in vivo MP-MRI with pathology is still a challenge and to date is insufficient to develop highly accurate models of disease. To address this problem this thesis showcases (1) a novel registration approach called LATIS (Local Affine Transformation assisted by Internal Structures) for co-registering post prostatectomy pseudo-whole mount (PWM) pathological sections with in vivo MRI images and (2) MP-MRI based predictive model for disease detection using a composite biomarker score based on a unique database of pathology co-registered MR data sets. Also showcased in this thesis is a study where r1 and r2* relaxivities of a common paramagnetic contrast agent were measured in blood and saline at both 3T and 7T. This is important information to have when attempting to perform DCE-MRI studies as part of a MP-MRI protocol at ultra-high magnetic field strengths.

The long-term goal of this research is to develop medical image registration techniques that can improve cancer detection, diagnosis, and therapy. Both rigid-body and deformable image registration methods were developed for human magnetic resonance (MR) images. As reported in Chapter 2, our automatic, three-dimensional (3D), rigid-body registration method has special features that include multi-resolution, multi-similarity, and restarting mechanisms. These features improved the robustness of MR image registration. The automatic registration method has been evaluated for potential applications in prostate MR imaging. However, tissue deformation and patient positions often raise problems for rigid-body registration. In order to improve registration accuracy, 3D thin-plate spline (TPS)-based deformable registration methods were developed and evaluated for pelvic MR images that were acquired at different patient positions/conditions. As described in Chapter 3, our TPS registration methods have several potential applications for image-guided therapy of the prostate. With the recent advancement of small animal imaging technology, MR imaging can provide high-resolution images of a tumor with subtle anatomic details, molecular imaging such as positron emission tomography (PET) can provide functional and metabolic information regarding the tumor. Registration of PET and MR images can combine the strength from the two imaging modalities and thus provides complementary information at the cellular and molecular levels. Chapter 4 proposes and evaluates a finite-element model (FEM)-based registration method for tumor PET and MR images, which can have unique applications in cancer early detection and therapy assessment. Chapter 5 further demonstrates clinical applications of multimodality image registration for prostate cancer patients. Methods were developed for MRI/SPECT registration and CT/SPECT fusion. Finally, Chapter 6 describes the future work of image registration techniques and the conclusion of the research work.