Search

To help diagnose, treat, manage, prevent and predict diseases, medical image analysis plays an increasingly crucial role in modern health care. In particular, using machine learning (ML) and deep learning (DL) techniques to process medical imaging data such as MRI, CT and X-Rays scans has been a research hot topic. Accurate and generalisable medical image segmentation using ML and DL is one of the most challenging medical image analysis tasks. The challenges are mainly caused by two key reasons: a) the variations of data statistics across different clinical centres or hospitals, and b) the lack of extensive annotations of medical data. To tackle the above challenges, one of the best ways is to learn disentangled representations. Learning disentangled representations aims to separate out, or disentangle, the underlying explanatory generative factors into disjoint subsets. Importantly, disentangled representations can be efficiently learnt from raw training data with limited annotations. Although, it is evident that learning disentangled representations is well suited for the challenges, there are several open problems in this area. First, there is no work to systematically study how much disentanglement is achieved with different learning and design biases and how different biases affect the task performance for medical data. Second, the benefit of leveraging disentanglement to design models that generalise well on new data has not been well studied especially in medical domain. Finally, the independence prior for disentanglement is a too strong assumption that does not approximate well the true generative factors. According to these problems, this thesis focuses on understanding the role of disentanglement in medical image analysis, measuring how different biases affect disentanglement and the task performance, and then finally using disentangled representations to improve generalisation performance and exploring better representations beyond disentanglement. In the medical domain, content-style disentanglement is one of the most effective frameworks to learn disentangled presentations. It disentangles and encodes image "content" into a spatial tensor, and image appearance or "style" into a vector that contains information on imaging characteristics. Based on an extensive review of disentanglement, I conclude that it is unclear how different design and learning biases affect the performance of content-style disentanglement methods. Hence, two metrics are proposed to measure the degree of content-style disentanglement by evaluating the informativeness and correlation of representations. By modifying the design and learning biases in three popular content-style disentanglement models, the degree of disentanglement and task performance of different model variants have been evaluated. A key conclusion is that there exists a sweet spot between task performance and the degree of disentanglement; achieving this sweet spot is the key to design disentanglement models. Generalising deep models to new data from new centres (termed here domains) remains a challenge. This is largely attributed to shifts in data statistics (domain shifts) between source and unseen domains. With the findings of aforementioned disentanglement metrics study, I design two content-style disentanglement approaches for generalisation. First, I propose two data augmentation methods that improve generalisation. The Resolution Augmentation method generates more diverse data by rescaling images to different resolutions. Subsequently, the Factor-based Augmentation method generates more diverse data by projecting the original samples onto disentangled latent spaces, and combining the learned content and style factors from different domains. To learn more generalisable representations, I integrate gradient-based meta-learning in disentanglement. Gradient-based meta-learning splits the training data into meta-train and meta-test sets to simulate and handle the domain shifts during training, which has shown superior generalisation performance. Considering limited annotations of data, I propose a novel semi-supervised meta-learning framework with disentanglement. I explicitly model the representations related to domain shifts. Disentangling the representations and combining them to reconstruct the input image, allows unlabeled data to be used to better approximate the true domain shifts within a meta-learning setting. Humans can quickly learn to accurately recognise anatomy of interest from medical images with limited guidance. Such recognition ability can easily generalise to new images from different clinical centres and new tasks in other contexts. This rapid and generalisable learning ability is mostly due to the compositional structure of image patterns in the human brain, which is less incorporated in the medical domain. In this thesis, I explore how compositionality can be applied to learning more interpretable and generalisable representations. Overall, I propose that the ground-truth generative factors that generate the medical images satisfy the compositional equivariance property. Hence, a good representation that approximates well the ground-truth factor has to be compositionally equivariant. By modelling the compositional representations with the learnable von-Mises-Fisher kernels, I explore how different design and learning biases can be used to enforce the representations to be more compositionally equivariant under different learning settings. Overall, this thesis creates new avenues for further research in the area of generalisable representation learning in medical image analysis, which we believe are key to more generalised machine learning and deep learning solutions in healthcare. In particular, the proposed metrics can be used to guide future work on designing better content-style frameworks. The disentanglement-based meta-learning approach sheds light on leveraging meta-learning for better model generalisation in a low-data regime. Finally, compositional representation learning we believe will play an increasingly important role in designing more generalisable and interpretable models in the future.

Since the rise of deep learning, new medical image segmentation methods have rapidly been proposed with promising results, with each one reporting marginal improvements on the previous state-of-the-art (SOTA) method. However, on visual inspection, errors are often revealed, such as topological mistakes (e.g. holes or folds), that are not detected using traditional evaluation metrics, such as Dice. Moreover, correct topology is often essential in ensuring segmentations are anatomically and pathologically plausible and, ultimately, suitable for downstream image processing tasks. Therefore, there is a need for methods to focus on ensuring that the predicted segmentations are topologically correct, rather than just optimising the pixel-wise accuracy. In this thesis, I propose a method that utilises prior knowledge of anatomy to segment structures, whilst preserving the known topology. The presented model, Topology Encouraging Deformation Segmentation Network (TEDS-Net), performs segmentation by deforming a prior shape with the same topological features as the anatomy of interest using learnt topology-preserving deformation fields. However, here I show that such fields only guarantee topology preservation in the continuous domain and that their properties begin to break down when applied in discrete spaces. To overcome this effect, I introduced additional modifications in TEDS-Net to more strictly enforce topology preservation, a step often overlooked in previous work. Across this thesis, TEDS-Net is applied to a range of natural and medical image segmentation tasks. I show how it can be used for multiple topology types, multiple structures and in both two- and three-dimensions. Further, I show how TEDS-Net can be used to segment whole volumes using minimally annotated training data. Across these experiments, TEDS-Net outperforms all SOTA baselines on topology, whilst maintaining competitive pixel-wise accuracy. Finally, TEDS-Net is integrated into a whole medical imaging pipeline, to illustrate the importance of topologically correct segmentations for downstream tasks. TEDS-Net is used to segment the developing cortical plate from in-utero fetal brain ultrasound scans in 3D, to enable the characterisation of its complex growth and development during gestation. To the best of my knowledge, this task has only been previously attempted from magnetic resonance imaging (MRI), despite ultrasound being the modality of choice in prenatal care. This is likely due to large acoustic shadows obstructing key brain regions in ultrasound. Due to TEDS-Net anatomical constraints, it can anatomically guide the cortical plate segmentation in regions of shadows, producing a complete segmentation that enables accurate downstream analysis.

MRI plays an important role in the continuing search for a sensitive osteoarthritis (OA) imaging biomarker able to detect early, pre-morphological alterations in cartilage composition. Determining the compositional recovery pattern of cartilage following acute joint loading could potentially present a more sensitive biomarker for defining cartilage health [1]. However, only a limited amount of studies have assessed both the immediate effect of joint loading on cartilage, as well as its post-loading recovery. In addition, when assessing the compositional responses of cartilage to joint loading, previous studies usually did not incorporate the measurement error of the used quantitative MRI technique into their analysis. Therefore, an uncertainty persists whether or not compositional MRI techniques are sensitive enough to measure changes in water and macromolecular content of cartilage, or if previous studies were merely measuring noise. Consequently, an objective of this thesis is to increase our understanding of and reliability in quantitative T2 and T1ρ relaxation time mapping to detect compositional responses of cartilage following a joint loading activity. Furthermore, to obtain the quantitative morphological and compositional measures of cartilage, detailed region-specific delineation of cartilage is required. This delineation (or segmentation) of cartilage is laborious and time-consuming as it is usually performed manually by an expert observer. Many new advances in image analysis, particularly those in convolutional neural networks (CNNs) and deep learning, have enabled a time-efficient semi- or fully-automated alternative to this process [2, 3]. This thesis explores the utility of deep CNNs generated segmentations for accurate surface-based analysis of cartilage morphology and composition from knee MRIs as well as of cortical bone thickness from knee CTs. Chapter 1 will provide an introduction into the structure and biomechanics of articular cartilage and the role of MRI in imaging the degenerative joint disorder, osteoarthritis as well as the effects of different joint loading activities on cartilage morphology and composition. Chapter 2 explains the principle of MRI and the pulse sequences used in the following chapter for the morphometric and compositional assessment of articular cartilage. Chapter 3 describes the use of 3D Cartilage Surface Mapping (3D-CaSM) [3] to assess variations in cartilage T1ρ and T2 relaxation times of young, healthy participants following a mild, unilateral stepping activity. By evaluating and incorporating the intrasessional repeatability of the T1ρ and T2 mapping techniques, I aim to highlight those cartilage areas experiencing exercise-induced compositional changes greater than measurement error. A significant amount of time is needed to manually segment the regions-of-interest required to perform the 3D-CaSM used in Chapter 3. Therefore, in Chapter 4, I assessed the use of deep convolutional neural networks for automating the segmentation process for multiple knee joint tissues simultaneous and increase the time-efficiency for evaluating knee MR datasets. I evaluated the use of a conditional Generative Adversarial Network (cGAN) as a potentially improved method for automated segmentation compared to the widely used convolutional neural network, U-Net. In Chapter 5 I combined the 3D-CaSM and automated segmentation methods presented in Chapters 3 and 4, respectively to assess the use of fully automatic segmentations of femoral and tibial bone-cartilage structures for accurate surface-based analysis of cartilage morphology and composition on knee MR images. This was performed on publicly available data from the Osteoarthritis Initiative, a multicentre observational study with expert manual segmentations provided by the Zuse Institute in Berlin. Chapter 6 describes an automated pipeline for subchondral cortical bone thickness mapping from knee CT data. I developed a method of using automated segmentations of articular cartilage and bone from knee MRI data to determine the periarticular bone surface which is covered by cartilage. This surface was then used to perform cortical bone thickness measurements on corresponding CT data. I validated this pipeline using data from the EU-funded, multi-centre observational study called Applied Private-Public partneRship enabling OsteoArthritis Clinical Headway (APPROACH). Chapter 7 summarises the main conclusions and contributions of the works presented in this thesis as well as providing directions for future work.

Accurate fetal biometric measurement and diagnosis rely on the quality of acquisition of 2D ultrasound (US) diagnostic planes. Optimal imaging output mostly depends on the ability of the sonographer to identify and localise a set of key anatomical landmarks which normally exhibit certain geometrical relationship patterns. Localisation of structures in US can be challenging because of the variability of image appearance and pose with respect to the transducer. In this doctoral thesis, a multiple proposal approach is introduced to facilitate an automatic localisation of multiple anatomical structures. Inspired by the image acquisition process, the knowledge of the sonographer is modelled implicitly in a multiple-stage hypothesis learning framework based on a database of 2D fetal US images acquired from multiple arbitrary orientations. Information learned from each stage allows the model to incorporate geometric constraints in a structured manner which further helps to reduce the number of possible valid hypotheses due to the variety of viewpoints and articulations. In return, the multiple proposal framework ensures good recall and precise localisation of key anatomical structures on unconstrained images of arbitrary viewing angles. There are a couple of key research contributions brought forward by this doctoral thesis. Firstly, a comprehensive dataset with annotated ground truth for respective anatomical structures in each image is explicitly described. This dataset includes images from multiple machines and sonographers' expertise, thereby guaranteeing a collection of challenging images of various shapes, textures and layout configurations. An efficient coarse classifier based on integral channel features is then designed to provide automatic abdomen localisation. This first-stage proposal investigates a simple yet elegant implementation of a soft-cascade rejection threshold architecture with score recalibration and shows its impending practicality for US image object detection at potentially real-time speed. This improves object recall at a much-reduced time complexity. Next, an automated segmentation-driven detection technique based on probabilistic superpixels-based Bag-of-Words (BOW) is presented. It is shown that this model, which is largely inspired by perceptual grouping of low level and mid-level image descriptors, is a good alternative to a conventional sliding-window mechanism in localising multiple key anatomical structures, such as the stomach and the umbilical vein (UV). This is achieved by reducing searching time and complexity compared to multi-scale sliding mechanism, and by jointly learning and modelling the spatial relationship of multi-class objects at training and testing time. Using this approach, the model transforms a pixel-wise probabilistic output into meaningful objects based on the knowledge of neighbouring superpixel regions, thus refining the location of these key structures. This method improves the average precision of the object detection (specifically for structures of deformable nature like the UV) by localising the exact object region as compared to the bounding boxes generated from the sliding-window approach. Overall, this doctoral thesis has sought to investigate computer vision and machine learning techniques augmented by the knowledge of sonography for fetal US image analysis interpretation.

Global assessment of the myocardial tissue wall motion from cine MRI (Magnetic Resonance Imaging) is widely used in clinical practice, however, the estimation is done, in the majority of cases, visually rather than quantitatively. This doctoral thesis proposes steps forward in the local assessment of the left ventricular (LV) function in order to provide a better estimation of the damage post acute myocardial infarction (MI). Accurate post acute MI tissue damage estimation from cine MR tracking might help clinicians to perform, better, more personalised, revascu larisation treatments. This thesis makes three technical contributions. The first technical contribution focuses on LV scar segmentation from Late Gadolium Enhancement (LGE) using supervoxels. LGE represents the gold standard for assessing the viability of the heart muscle. However, considerable user input is needed in order to correctly segment the scar. A global intensity threshold is used, and a further, remote, area of "healthy myocardium" also needs to be segmented. Furthermore, an additional number of corrections are required to remove false positive areas of enhancement. We found that by using a strong regulariser such as a modified version of the Single Linear Iterative Clustering (SLIC) supervoxels, we can eliminate the false positive enhancement areas. Furthermore, there is no need for a remote "healthy myocardium" area to be contoured. The second technical contribution is to provide a 4D map of supervoxels (voxels clustered together based on local similarity) computed using similarity in strain curves. Our study found that while LGE performs extremely well in highlighting the area where scarring occurs, at the same time it underestimates the area of affected myocardium when compared with strain analysis. Here we investigated various registration methods, based on optical flow deformable registration, in order to evaluate the accuracy of computing the displacement curves, and subsequently validated the registration method against manually placed landmarks in the LV. The third technical contribution is a modification to the maskSLIC algorithm to generate six cluster centers in each short axis, basal and mid-ventricular slice, equally distributed at 60◦ from each other in order to mimic the AHA 16 segments model used in clinical practice. Considering clinical perspectives centered approach, these technical advances will contribute to the field of regional assessment of the cardiac function, therefore potentially improving patient outcome.

This thesis addresses the task of developing automatic algorithms for analysing the two-dimensional ultrasound video footage obtained from fetal heart screening scans. These scans are typically performed in the second trimester of pregnancy to check for congenital heart anomalies and require significant training and anatomical knowledge to perform. The aim is to develop a tool that runs at high frame rates with no user initialisation and infers the visibility, position, orientation, view classification, and cardiac phase of the heart, and additionally the locations of cardiac structures of interest (such as valves and vessels) in a manner that is robust to the various sources of variation that occur in real-world ultrasound scanning. This is the first work to attempt such a detailed automated analysis of these videos. The problem is posed as a Bayesian filtering problem, which provides a principled framework for aggregating uncertain measurements across a number of frames whilst exploiting the constraints imposed by anatomical feasibility. The resulting inference problem is solved approximately with a particle filter, whose state space is partitioned to reduce the problems associated with filtering in high-dimensional spaces. Rotation-invariant features are captured from the videos in an efficient way in order to tackle the problem of unknown orientation. These are used within random forest learning models, including a novel formulation to predict circular-valued variables. The algorithm is validated on an annotated clinical dataset, and the results are compared to estimates of inter- and intra-observer variation, which are significant in both cases due to the inherent ambiguity in the imagery. The results suggest that the algorithm's output approaches these benchmarks in several respects, and fall slightly behind in others. The work presented here is an important first step towards developing automated clinical tools for the detection of congenital heart disease.

Techniques described to date for the reproducible and noninvasive quantification of regional cardiac function have been largely relegated to research settings due to time-consuming and cumbersome image acquisition and analysis. In this thesis, feature tracking algorithms are developed for 2-D+Time cardiac magnetic resonance (CMR) and 3-D+Time cardiac computed tomography (CCT) image sequences that are easily acquired clinically, while emphasising reproducibility and automation in their design. First, a commercially-implemented CMR feature tracking algorithm for the analysis of steady state free precession (SSFP) cine series is evaluated in patients with hypertrophic cardiomyopathy (HCM) and arrhythmogenic right ventricular cardiomyopathy (ARVC), which primarily affect the left ventricle (LV) and right ventricle (RV), respectively, and functional impairment compared with control populations is found in both cases. The limitations of this implementation are then used to guide development of an automated algorithm for the same purpose, making use of fully convolutional neural networks (CNN) for segmentation and spline registration across all frames simultaneously for tracking. This study is performed in the subjects with HCM, and functional impairment is again identified in disease subjects. Finally, as myocardial contraction is inherently a 3-D phenomenon, a technique is developed for quantification of regional function from 3-D+Time functional CCT studies using simultaneous registration of automatically generated Loop subdivision surface models for tracking. This study is performed in canine mongrels, and compared with the current state of the art technique for CCT functional analysis. This work demonstrates the feasibility of automated, reproducible cardiac functional analysis from CMR and CCT image sequences. While work remains to be done in extending the principles demonstrated and modular components described to fully automated whole-heart analysis, it is hoped that this thesis will accelerate the clinical adoption of regional functional analysis.

This thesis proposes different models for a variety of applications, such as semantic segmentation, in-the-wild face recognition, microscopy cell counting and detection, standardized re-orientation of 3D ultrasound fetal brain and Magnetic Resonance (MR) cardiac video segmentation. Our approach is to employ the large-scale machine learning models, in particular deep neural networks. Expert knowledge is either mathematically modelled as a differentiable hidden layer in the Artificial Neural Networks, or we tried to break the complex tasks into several small and easy-to-solve tasks. Multi-scale contextual information plays an important role in pixel-wise predic- tion, e.g. semantic segmentation. To capture the spatial contextual information, we present a new block for learning receptive field adaptively by within-layer recurrence. While interleaving with the convolutional layers, receptive fields are effectively enlarged, reaching across the entire feature map or image. The new block can be initialized as identity and inserted into any pre-trained networks, therefore taking benefit from the "pre-train and fine-tuning" paradigm. Current face recognition systems are mostly driven by the success of image classification, where the models are trained to by identity classification. We propose a multi-column deep comparator networks for face recognition. The architecture takes two sets (each contains an arbitrary number of faces) of images or frames as inputs, facial part-based (e.g. eyes, noses) representations of each set are pooled out, dynamically calibrated based on the quality of input images, and further compared with local "experts" in a pairwise way. Unlike the computer vision applications, collecting data and annotation is usually more expensive in biomedical image analysis. Therefore, the models that can be trained with fewer data and weaker annotations are of great importance. We approach the microscopy cell counting and detection based on density estimation, where only central dot annotations are needed. The proposed fully convolutional regression networks are first trained on a synthetic dataset of cell nuclei, later fine-tuned and shown to generalize to real data. In 3D fetal ultrasound neurosonography, establishing a coordinate system over the fetal brain serves as a precursor for subsequent tasks, e.g. localization of anatomical landmarks, extraction of standard clinical planes for biometric assessment of fetal growth, etc. To align brain volumes into a common reference coordinate system, we decompose the complex transformation into several simple ones, which can be easily tackled with Convolutional Neural Networks. The model is therefore designed to leverage the closely related tasks by sharing low-level features, and the task-specific predictions are then combined to reproduce the transformation matrix as the desired output. Finally, we address the problem of MR cardiac video analysis, in which we are interested in assisting clinical diagnosis based on the fine-grained segmentation. To facilitate segmentation, we present one end-to-end trainable model that achieves multi-view structure detection, alignment (standardized re-orientation), and fine- grained segmentation simultaneously. This is motivated by the fact that the CNNs in essence is not rotation equivariance or invariance, therefore, adding the pre-alignment into the end-to-end trainable pipeline can effectively decrease the complexity of segmentation for later stages of the model.

This thesis explores the integration of Meshnet models with distributed learning techniques to enhance MRI brain scan analysis, with a focus on optimizing brain tissue segmentation while maintaining secure distributed systems. Through refining Meshnet's architecture and training strategies, the goal is to enhance accuracy in identifying brain segmentations. Distributed learning strategies, particularly centralized aggregation, are investigated to enable collaborative model training while ensuring data privacy. Additionally, Coinstac is integrated for secure gradient aggregation from diverse nodes, facilitating collaborative analysis without compromising confidentiality. Implementation of a serverless architecture using public clouds extends global accessibility while upholding robust security measures. The primary aim is to empower professionals with advanced tools for collaborative research.

Changes in retinal structure are related to different eye diseases. Various retinal imaging techniques, such as fundus imaging and optical coherence tomography (OCT) imaging modalities, have been developed for non-intrusive ophthalmology diagnoses according to the vasculature changes. However, it is time consuming or even impossible for ophthalmologists to manually label all the retinal structures from fundus images and OCT images. Therefore, computer aided diagnosis system for retinal imaging plays an important role in the assessment of ophthalmologic diseases and cardiovascular disorders. The aim of this PhD thesis is to develop segmentation methods to extract clinically useful information from these retinal images, which are acquired from different imaging modalities. In other words, we built the segmentation methods to extract important structures from both 2D fundus images and 3D OCT images. In the first part of my PhD project, two novel level set based methods were proposed for detecting the blood vessels and optic discs from fundus images. The first one integrates Chan-Vese's energy minimizing active contour method with the edge constraint term and Gaussian Mixture Model based term for blood vessels segmentation, while the second method combines the edge constraint term, the distance regularisation term and the shape-prior term for locating the optic disc. Both methods include the pre-processing stage, used for removing noise and enhancing the contrast between the object and the background. Three automated layer segmentation methods were built for segmenting intra-retinal layers from 3D OCT macular and optic nerve head images in the second part of my PhD project. The first two methods combine different methods according to the data characteristics. First, eight boundaries of the intra-retinal layers were detected from the 3D OCT macular images and the thickness maps of the seven layers were produced. Second, four boundaries of the intra-retinal layers were located from 3D optic nerve head images and the thickness maps of the Retinal Nerve Fiber Layer (RNFL) were plotted. Finally, the choroidal layer segmentation method based on the Level Set framework was designed, which embedded with the distance regularisation term, edge constraint term and Markov Random Field modelled region term. The thickness map of the choroidal layer was calculated and shown.

Texture can be broadly stated as spatial variation of image intensities. Texture analysis and classification is a well researched area for its importance to many computer vision applications. Consequently, much research has focussed on deriving powerful and efficient texture descriptors. Local binary patterns (LBP) and its variants are simple yet powerful texture descriptors. LBP features describe the texture neighbourhood of a pixel using simple comparison operators, and are often calculated based on varying neighbourhood radii to provide multi-resolution texture descriptions. A comprehensive evaluation of different LBP variants on a common benchmark dataset is missing in the literature. This thesis presents the performance for different LBP variants on texture classification and retrieval tasks. The results show that multi-scale local binary pattern variance (LBPV) gives the best performance over eight benchmarked datasets. Furthermore, improvements to the Dominant LBP (D-LBP) by ranking dominant patterns over complete training set and Compound LBP (CM-LBP) by considering 16 bits binary codes are suggested which are shown to outperform their original counterparts. The main contribution of the thesis is the introduction of multi-dimensional LBP features, which preserve the relationships between different scales by building a multi-dimensional histogram. The results on benchmarked classification and retrieval datasets clearly show that the multi-dimensional LBP (MD-LBP) improves the results compared to conventional multi-scale LBP. The same principle is applied to LBPV (MD-LBPV), again leading to improved performance. The proposed variants result in relatively large feature lengths which is addressed using three different feature length reduction techniques. Principle component analysis (PCA) is shown to give the best performance when the feature length is reduced to match that of conventional multi-scale LBP. The proposed multi-dimensional LBP variants are applied for medical image analysis application. The first application is nailfold capillary (NC) image classification. Performance of MD-LBPV on NC images is highest, whereas for second application, HEp-2 cell classification, performance of MD-LBP is highest. It is observed that the proposed texture descriptors gives improved texture classification accuracy.

The aim of this research was to address the computational complexity in designing multimodality Computer-Aided Diagnosis (CAD) systems for characterising breast lesions, by harnessing the general purpose computational potential of consumer-level Graphics Processing Units (GPUs) through parallel programming methods. The complexity in designing such systems lies on the increased dimensionality of the problem, due to the multiple imaging modalities involved, on the inherent complexity of optimal design methods for securing high precision, and on assessing the performance of the design prior to deployment in a clinical environment, employing unbiased system evaluation methods. For the purposes of this research, a Pattern Recognition (PR)-system was designed to provide highest possible precision by programming in parallel the multiprocessors of the NVIDIA’s GPU-cards, GeForce 8800GT or 580GTX, and using the CUDA programming framework and C++. The PR-system was built around the Probabilistic Neural Network classifier and its performance was evaluated by a re-substitution method, for estimating the system’s highest accuracy, and by the external cross validation method, for assessing the PR-system’s unbiased accuracy to new, “unseen” by the system, data. Data comprised images of patients with histologically verified (benign or malignant) breast lesions, who underwent both ultrasound (US) and digital mammography (DM). Lesions were outlined on the images by an experienced radiologist, and textural features were calculated. Regarding breast lesion classification, the accuracies for discriminating malignant from benign lesions were, 85.5% using US-features alone, 82.3% employing DM-features alone, and 93.5% combining US and DM features. Mean accuracy to new “unseen” data for the combined US and DM features was 81%. Those classification accuracies were about 10% higher than accuracies achieved on a single CPU, using sequential programming methods, and 150-fold faster. In addition, benign lesions were found smoother, more homogeneous, and containing larger structures. Additionally, the PR-system design was adapted for tackling other medical problems, as a proof of its generalisation. These included classification of rare brain tumours, (achieving 78.6% for overall accuracy (OA) and 73.8% for estimated generalisation accuracy (GA), and accelerating system design 267 times), discrimination of patients with micro-ischemic and multiple sclerosis lesions (90.2% OA and 80% GA with 32-fold design acceleration), classification of normal and pathological knee cartilages (93.2% OA and 89% GA with 257-fold design acceleration), and separation of low from high grade laryngeal cancer cases (93.2% OA and 89% GA, with 130-fold design acceleration). The proposed PR-system improves breast-lesion discrimination accuracy, it may be redesigned on site when new verified data are incorporated in its depository, and it may serve as a second opinion tool in a clinical environment.

Medical imaging has seen a rapid development in its clinical use in assessment of treatment outcome, disease monitoring and diagnosis over the last few decades. Yet, the vast amount of available image data limits the practical use of this potentially very valuable source of information for radiologists and physicians. Therefore, the design of computer-aided medical image analysis is of great importance to imaging in clinical practice. This thesis deals with the problem of deformable image registration in the context of lung imaging, and addresses three of the major challenges involved in this challenging application, namely: designing an image similarity for multi-modal scans or scans of locally changing contrast, modelling of complex lung motion, which includes sliding motion, and approximately globally optimal mathematical optimisation to deal with large motion of small anatomical features. The two most important contributions made in this thesis are: the formulation of a multi-dimensional structural image representation, which is independent of modality, robust to intensity distortions and very discriminative for different image features, and a discrete optimisation framework, based on an image-adaptive graph structure, which enables a very efficient optimisation of large dense displacement spaces and deals well with sliding motion. The derived methods are applied to two different clinical applications in pulmonary image analysis: motion correction for breathing-cycle computed tomography (CT) volumes, and deformable multi-modal fusion of CT and magnetic resonance imaging chest scans. The experimental validation demonstrates improved registration accuracy, a high quality of the estimated deformations, and much lower computational complexity, all compared to several state-of-the-art deformable registration techniques.

Chronic obstructive pulmonary disease (COPD) describes a range of lung conditions including emphysema, chronic bronchitis and small airways disease. While COPD is a major cause of death and debilitating illness, current clinical assessment methods are inadequate: they are a poor predictor of patient outcome and insensitive to mild disease. A new imaging technology, hyperpolarised xenon MRI, offers the hope of improved diagnostic techniques, based on regional measurements using functional imaging. There is a need for quantitative analysis techniques to assist in the interpretation of these images. The aim of this work is to develop these techniques as part of a clinical trial into hyperpolarised xenon MRI. In this thesis we develop a fully automated pipeline for deriving regional measurements of lung function, making use of the multiple imaging modalities available from the trial. The core of our pipeline is a novel method for automatically segmenting the pulmonary lobes from CT data. This method combines a Hessian-based filter for detecting pulmonary fissures with anatomical cues from segmented lungs, airways and pulmonary vessels. The pipeline also includes methods for segmenting the lungs from CT and MRI data, and the airways from CT data. We apply this lobar map to the xenon MRI data using a multi-modal image registration technique based on automatically segmented lung boundaries, using proton MRI as an intermediate stage. We demonstrate our pipeline by deriving lobar measurements of ventilated volumes and diffusion from hyperpolarised xenon MRI data. In future work, we will use the trial data to further validate the pipeline and investigate the potential of xenon MRI in the clinical assessment of COPD. We also demonstrate how our work can be extended to build personalised computational models of the lung, which can be used to gain insights into the mechanisms of lung disease.

Vitamin D is very important for endochondral ossification and it is commonly insufficient during pregnancy (Javaid et al., 2006). Insufficiency of vitamin D during pregnancy predicts bone mass and hence predicts adult osteoporosis (Javaid et al., 2006). The relationship between maternal vitamin D and manually measured fetal biometry has been studied (Mahon et al., 2009). However, manual fetal biometry especially volumetric measurements are subjective, time-consuming and possibly irreproducible. Computerised measurements can overcome or at least reduce such problems. This thesis concerns the development and evaluation of novel methods to do this. This thesis makes three contributions. Firstly, we have developed a novel technique based on the Random Forests (RF) classifier to segment and measure several fetal femoral characteristics from 3D ultrasound volumes automatically. We propose a feature selection step in the training stage to eliminate irrelevant features and utilise the "good" ones. We also develop a weighted voting mechanism to weight tree probabilistic decisions in the RF classifier. We show that the new RF classifier is more accurate than the classic method (Yaqub et al., 2010b, Yaqub et al., 2011b). We achieved 83% segmentation precision using the proposed technique compared to manually segmented volumes. The proposed segmentation technique was also validated on segmenting adult brain structures in MR images and it showed excellent accuracy. The second contribution is a wavelet-based image fusion technique to enhance the quality of the fetal femur and to compensate for missing information in one volume due to signal attenuation and acoustic shadowing. We show that using image fusion to increase the image quality of ultrasound images of bony structures leads to a more accurate and reproducible assessment and measurement qualitatively and quantitatively (Yaqub et al., 2010a, Yaqub et al., 2011a). The third contribution concerns the analysis of data from a cohort study of 450 fetal femoral ultrasound volumes (18-21 week gestation). The femur length, cross-sectional areas, volume, splaying indices and angles were automatically measured using the RF method. The relationship between these measurements and the fetal gestational age and maternal vitamin D was investigated. Segmentation of a fetal femur is fast (2.3s/volume), thanks to the parallel implementation. The femur volume, length, splaying index were found to significantly correlate with fetal gestational age. Furthermore, significant correlations between the automatic measurements and 10 nmol increment in maternal 25OHD during second trimester were found.

Tissue microarrays (TMAs) are a high-throughput technique that facilitates the survey of very large numbers of tumours, important both in clinical and research applications. However, the assessment of stained TMA sections is laborious and still needs to be carried manually, constituting a bottleneck in the pathologist?s work-flow. This process is also prone to perceptual errors and observer variability.Thus, there is strong motivation for the development of automated quantitative analysis of TMA image data. The analysis of breast TMA sections subjected to nuclear immunostaining begins with the classification of each spot as to the maintype of tissue that it contains, namely tumour, normal, stroma, or fat. Tumour and normal spots are then assigned a so-called quickscore composed of a pair or integer values, the first reflecting the proportion of epithelial nuclei that are stained, and the second reflecting the strength of staining of those nuclei. In this work, an approach was developed to analyse breast TMA spots subjectedto progesterone receptor immunohistochemistry. Spots were classified into their four main types through a method that combined a bag of features approachand classifiers based on either multi-layer perceptrons or latent Dirichlet allocation models. A classification accuracy of 74.6 % was achieved. Tumour and normal spots were scored via an approach that involved the computation of global features formalising the quickscore values used by pathologists, and the use of Gaussian processes for ordinal regression to predict actual quickscores based on global features. Mean absolute errors of 0.888 and 0.779 were achieved in the prediction of the first and second quickscore values, respectively. By setting thresholds on prediction confidence, it was possible to classify and score fractions of spots with substantially higher accuracies and lower mean absolute errors. Amethod for the segmentation of TMA spots into regions of different types was also investigated, to explore the generative nature of latent Dirichlet allocation models.

Image registration methods underpin many analysis techniques in neuroimaging. They are essential in group studies when images of different individuals or different modalities need to be brought into a common reference frame. This thesis explores the potential of brain connectivity- driven alignment and develops surface registration techniques for magnetic resonance imaging (MRI), which is a noninvasive neuroimaging tool for probing function and structure of the human brain. The first part of this work develops a novel surface registration framework, based on free mesh deformations, which aligns cortical and subcortical surfaces by matching structural connectivity patterns derived using probabilistic tractography (diffusion-weighted MRI). Structural, i.e. white matter, connectivity is a good predictor of functional specialisation and structural connectivity-driven registration can therefore be expected to enhance the alignment of functionally homologous areas across subjects. The second part validates developed methods for cortical surfaces. Resting State Networks are used in an innovative way to delineate several functionally distinct regions, which were then used to quantify connectivity-driven registration performance by measuring the inter- subject overlap before and after registration. Consequently, the proposed method is assessed using an independent imaging modality and the results are compared to results from state-of-the-art cortical geometry-driven surface registration methods. A connectivity-driven registration pipeline is also developed for, and applied to, the surfaces of subcortical structures such as the thalamus. It is carefully validated on a set of artificial test examples and compared to another novel surface registration paradigm based on spherical wavelets. The proposed registration pipeline is then used to explore the differences in the alignment of two groups of subjects, healthy controls and Alzheimer's disease patients, to a common template. Finally, we propose how functional connectivity can be used instead of structural connectivity for driving registrations, as well as how the surface-based framework can be extended to a volumetric one. Apart from providing the benefits such as the improved functional alignment, we hope that the research conducted in this thesis will also represent the basis for the development of templates of structural and functional brain connectivity.

In this thesis, we develop a motion correction method to overcome the degradation of image quality introduced by respiratory motion in positron emission tomography (PET), so that diagnostic performance for lung cancer can be improved. Lung cancer is currently the most common cause of cancer death both in the UK and in the world. PET/CT, which is a combination of PET and CT, providing clinicians with both functional and anatomical information, is routinely used as a non-invasive imaging technique to diagnose and stage lung cancer. However, since a PET scan normally takes 15-30 minutes, respiration is inevitable in data acquisition. As a result, thoracic PET images are substantially degraded by respiratory motion, not only by being blurred, but also by being inaccurately attenuation corrected due to the mismatch between PET and CT. If these challenges are not addressed, the diagnosis of lung cancer may be misled. The main contribution of this thesis is to propose a novel process for respiratory motion correction, in which non-attenuation corrected PET images (PET-NAC) are registered to a reference position for motion correction and then multiplied by a voxel-wise attenuation correction factor (ACF) image for attenuation correction. The ACF image is derived from a CT image which matches the reference position, so that no attenuation correction artefacts would occur. In experiments, the motion corrected PET images show significant improvements over the uncorrected images, which represent the acquisitions typical of current clinical practice. The enhanced image quality means that our method has the potential to improve diagnostic performance for lung cancer. We also develop an automatic lesion detection method based on motion corrected images. A small lung lesion is only 2 or 3 voxels in diameter and of marginal contrast. It could easily be missed by human observers. Our method aims to provide radiologists with a map of potential lesions for decision so that diagnostic efficiency can be improved. It utilises both PET and CT images. The CT image provides a lung mask, to which lesion detection is confined, whereas the PET image provides distribution of glucose metabolism, according to which lung lesions are detected. Experimental results show that respiratory motion correction significantly increases the success of lesion detection, especially for small lesions, and most of the lung lesions can be detected by our method. The method can serve as a useful computer-aided image analysing tool to help radiologists read images and find malignant lung lesions. Finally, we explore the possibility of incorporating temporal information into respiratory motion correction. Conventionally, respiratory gated PET images are individually registered to the reference position. Temporal continuity across the respiratory period is not considered. We propose a spatio-temporal registration algorithm, which models temporally smooth deformation in order to improve the registration performance. However, we discover that the improvement introduced by temporal information is relatively small at the cost of a much longer computation time. Spatial registration with regularisation yields similar results but is superior in speed. Therefore, it is preferable for respiratory motion correction.

Uncertainty in the sizing of breast cancer masses is a major issue in breast screening programs, as there is a tendency to severely underestimate the sizing of malignant masses, especially with ultrasound imaging as part of the standard triple assessment. Due to this issue about 20% of all surgically treated women have to undergo a second resection, therefore the aim of this thesis is to address this issue by developing novel image analysis methods. Ultrasound elasticity imaging has been proven to have a better ability to differentiate soft tissues compared to standard B-mode. Thus a novel segmentation algorithm is presented, employing elasticity imaging to improve the sizing of malignant breast masses in ultrasound. The main contributions of this work are the introduction of a novel filtering technique to significantly improve the quality of the B-mode image, the development of a segmentation algorithm and their application to an ongoing clinical trial. Due to the limitations of the employed ultrasound device, the development of a method to improve the contrast and signal to noise ratio of B-mode images was required. Thus, an autoregressive model based filter on the radio-frequency signal is presented which is able to reduce the misclassification error on a phantom by up to 90% compared to the employed device, achieving similar results to a state-of-the art ultrasound system. By combining the output of this filter with elasticity data into a region based segmentation framework, a computationally highly efficient segmentation algorithm using Graph-cuts is presented. This method is shown to successfully and reliably segment objects on which previous highly cited methods have failed. Employing this method on 18 cases from a clinical trial, it is shown that the mean absolute error is reduced by 2 mm, and the bias of the B-Mode sizing to underestimate the size was overcome. Furthermore, the ability to detect widespread DCIS is demonstrated.

Medical computer vision is a novel research discipline based on the application of computer vision methods to data sets acquired via medical imaging techniques. This work focuses on magnetic resonance imaging (MRI) data sets, particularly in studies of schizophrenia and multiple sclerosis. Research on these diseases is challenged by the lack of appropriate morphometric tools to accurately quantify lesion growth, assess the effectiveness of a drug treatment, or investigate anatomical information believed to be evidence of schizophrenia. Thus, most hypotheses involving these conditions remain unproven. This thesis contributes towards the development of such morphometric techniques. A framework combining several tools is established, allowing for compensation of bias fields, boundary detection by modelling partial volume effects (PVE), and a combined statistical and geometrical segmentation method. Most importantly, it also allows for the computation of confidence bounds in the location of the object being segmented by bounding PVE voxels. Bounds obtained in such fashion encompass a significant percentage of the volume of the object (typically 20-60%). A statistical model of the intensities contained in PVE voxels is used to provide insight into the contents of PVE voxels and further narrow confidence bounds. This not only permits a reduction by an order of magnitude in the width of the confidence intervals, but also establishes a statistical mechanism to obtain probability distributions on shape descriptors (e.g. volume), instead of just a raw magnitude or a set of confidence bounds. A challenging clinical study is performed using these tools: to investigate differences in asymmetry of the temporal horns in schizophrenia. This study is of high clinical relevance. The results show that our tools are sufficiently accurate for studies of this kind, thus providing clinicians, for the first time, with the means to corroborate unproven hypotheses or reliably assess patient evolution.

The field of medical artificial intelligence (AI) has seen significant advancements with the availability of digitalized medical data. Machine learning (ML) and deep learning (DL) models have been developed to leverage these datasets, aiding in clinical decision-making and the delivery of evidence-based care. Medical imaging has particularly benefited from ML and DL algorithms, with successful applications in image classification, segmentation, and detection. Similarly, electronic health records (EHR) data analysis has facilitated risk prediction, disease phenotyping, and treatment outcome assessment. However, the field still faces practical challenges, such as the heterogeneity and missingness of data in EHR, and the scarcity of gold-standard labels in medical imaging. This thesis aims to address these challenges and contribute to the field of medical AI by developing innovative techniques and methodologies. It focuses on building generalizable and explainable AI models with limited labeled data and leveraging privileged information for clinical decision support. To achieve these objectives, strategies such as bias mitigation, data augmentation, regularization, multi-source data integration, and ensembles are proposed or employed. Furthermore, the thesis investigates the utilization of privileged information, which refers to data or information accessible only during training and not during inference. In the medical field, privileged information is prevalent due to multiple data sources and the varying availability of modalities and variations in medical care protocols. By leveraging privileged information, novel algorithms under the Learning Using Privileged Information (LUPI) paradigm and the Learning Using Partially Available Privileged Information (LUPAPI) paradigm are proposed to enhance model performance and address issues of data missingness in multimodal settings. These algorithms allow models to make predictions without relying on specific data during inference, while still benefiting from its inclusion. The thesis consists of several chapters that tackle specific tasks and challenges. Chapter 2 presents an automated pipeline for segmenting coronary arteries in X-ray coronary angiography images. Chapter 3 focuses on the diagnosis of acute respiratory distress syndrome (ARDS) using EHR data, while Chapter 4 extends this work by applying the LUPI paradigm and LUPAPI paradigm. Chapter 5 addresses the challenge of label uncertainty in ARDS detection using chest X-ray images. Finally, Chapter 6 concludes the thesis by summarizing the key findings and discussing future directions. In conclusion, this thesis contributes to the advancement of medical AI by developing techniques for robust and explainable decision-support models with limited labeled data. It also explores the utilization of privileged information to enhance model performance. The proposed methodologies have the potential to improve patient care and outcomes, paving the way for further research and development in the field of medical AI.

Abstract: Medical images play an essential role in detecting and diagnosing numerous diseases. With different medical imaging modalities, a large volume of images is generated every day across healthcare organizations worldwide, providing visualization of lesion appearance (e.g., dermoscopic images) anatomical information (e.g., MRI images), or cellular structures (e.g., laser scattering images). After obtaining the medical images, manual analysis and diagnosis is done by clinicians based on their prior knowledge and experiences. However, the diagnosis process can take a significant amount of time and the decision is subjective and biased toward different clinicians. The main goal of this dissertation is to develop machine learning methods for automatic medical image analysis and diagnosis. In this dissertation, two types of medical images, including dermoscopic images and laser scattering images, are used. The dermoscopic images are used to detect the malignant lesions from the benign lesions. Experimental and simulated laser scattering images are used for label-free cell identification, and cell property characterization, respectively. This thesis presents two methods based on machine learning for automatic skin lesion analysis. Skin lesion segmentation with auxiliary task is proposed for the accurate segmentation of the pigment regions, which does not require extra labeling information compared with the multi-task learning methods. An automatic skin lesion classification method based on mid-level feature learning is proposed for melanoma detection. State-of-the-art results have been obtained and performances are discussed by extensive verification. This thesis further presents a machine learning technique for the analysis of laser scattering images. Typically, scattering patterns of the staurosporine-treated and non-treated SH-SY5Y neuroblastoma cells are obtained and classified, aiming at providing a better understanding of Parkinson's disease. In addition, multi-wavelength multi-direction laser scattering patterns of single cells have also been simulated to discuss the roles of two factors, cell surface roughness, and mitochondria number, in contributing to the scattering patterns. A systematic and thorough study has been done by extensive experiments. Theoretical analysis about the influence of the multi-wavelength multi-direction scattering patterns has been included. Satisfactory performance has been achieved for both the experimental and simulated data.

This dissertation concerns the design, analysis and High-Performance Computing (HPC) implementation of fast algorithms for large deformation diffeomorphic registration and its application in quantifying abnormal anatomical deformations in Magnetic Resonance Image (MRI) scans of brain tumor patients. Image registration finds point correspondences between two images by solving an optimization problem. It is a fundamental and computationally expensive operation that finds applications in computer vision and medical image analysis. Diffeomorphic registration is a non-convex and nonlinear inverse problem and, as a result, presents significant numerical and computational challenges. Designing and implementing efficient and accurate numerical schemes on modern computer architectures is the key to accelerating and sometimes even enabling the development of image analysis workflows. In this dissertation, we contribute to several aspects of diffeomorphic registration: (i) a novel preconditioner that improves performance and scalability, (ii) algorithms and their scalable implementation on heterogeneous compute architectures, and (ii) applications in neuroradiology. Our work on diffeomorphic image registration is based on CLAIRE – a formulation, algorithmic framework, and software developed at the University of Texas at Austin. As the first highlight of our contributions, we introduced a novel two-level Hessian preconditioner that results in an improvement of 2.5× in CLAIRE’s performance. As a second highlight, our optimized HPC implementation yields orders of magnitude speedup as CLAIRE now supports GPU architectures and distributed memory parallelism via GPU-aware message passing interface (MPI). CLAIRE can register clinical-grade brain MRI scans of size 256³ in under 5 seconds on a single NVIDIA V100 GPU. For research-grade high-resolution volumetric images, e.g., mouse brain CLARITY images of size 2816 × 3016 × 1162, CLAIRE takes under 30 minutes using 256 NVIDIA V100 GPUs on the Texas Advanced Computing Center’s (TACC) Longhorn supercomputer. To the best of our knowledge, CLAIRE is the most scalable image registration algorithm and software. CLAIRE has been open-sourced under the GNU v3 license and is available on Github at https://github.com/andreasmang/claire. Our target clinical application concerns the utilization of image registration to characterize the mass effect in MRI scans of patients with glioblastoma, a fatal brain cancer. Mass effect is the mechanical deformation in surrounding healthy tissue caused by the growing tumor. The location and degree of mass effect could aid in differential diagnosis and treatment planning. Towards this end, we introduce an algorithm that integrates CLAIRE, statistical analysis for abnormality detection, and machine learning to quantify and localize mass effect. Given a patient’s brain tumor scan, we generate a clinical summary with (i) an estimate of the degree of mass effect along with a severity label – mild, moderate, or severe with up to 62% accuracy, (ii) a heatmap of mass effect for the brain scan and, (iii) a list of specific anatomical regions, e.g. frontal lobe, which is statistically likely to possess significant mass effect.

In different areas, particularly medical image analysis, there is a vital need to access and analyse dynamic three dimensional (3D) images of the anatomical structures of the human body. This can enable specialists to track events as well as clinically conduct and evaluate surgical and radio therapeutical procedures. For example, measuring the 3D kinematics of knee joints in a dynamic manner is essential for understanding their normal functions and diagnosing any pathology, such as ligament injury and osteoarthritis. For evaluations of subsequent treatments, such as surgery and rehabilitation, and designs of joint replacements, having knowledge of the movements of knee joints is necessary. Image registration is increasingly being applied to medical image analysis. Whereas in mono-modal registration, the images to be registered are acquired by the same sensor, in multi-modal image registration, they can be taken from different devices or imaging protocols which makes this registration process much more challenging. The invasive or non-invasive nature of the registration method used, the computational time it requires as well as its accuracy and robustness against a large range of initial displacements are the most important features used for its evaluation. As currently available approaches have limited capabilities to register images with large initial displacements and are either not sufficiently accurate or very computationally expensive, the objective of this research is to propose new registration methods, that provide dynamic 3D images, to address these issues. In the first part of this study, I conducted research on registering an individuals’ natural knee bones that can provide 3D information of knee joint kinematics which can be very helpful for improving the accuracy of diagnosis and enabling targeted treatments. A fast, accurate and robust hybrid rigid body registration method based on two different multi-modal similarity measures, the edge position difference (EPD) and sum-of-conditional variance (SCV), is proposed. It uses a gradient descent optimisation technique to register multi-modal images and determine the best transformation parameters. It helps to achieve a trade-off among different challenges, including time complexity, accuracy and robustness against a large range of initial displacements. To evaluate it, several experiments were performed on two different databases: one collected from the knee bones of four patients and the other from three knee cadavers installed on a mechanical positioning system, with the results showing that this method is accurate, fast and robust against large initial displacement. Then, I conducted research on registering implanted human knee joints and proposed a non-invasive, robust 3D-to-2D registration method which can be used for 3D evaluations of the status of knee implants after joint replacement surgeries. In this method, 3D models of the implants for an individual with the relevant post-operative fluoroscopy frames are able to be used in the registration process. As a result, it is possible to perform 3D analysis at any time after a surgery by simply taking single-plane radiographs. This approach uses the EPD multi-modal similarity measure together with a steepest descent optimisation method. It applies coarse-to-fine registration steps to determine the transformation parameters that lead to the best alignment between the model used and X-ray images to be registered. The experimental results showed that not only does the proposed registration method have a high success rate but that it is also much faster than the most relevant competitive approach. Although the experiments were designed for a 3D analysis of total knee arthroplasty (TKA) components, this proposed method can be applied to other joints such as the ankle or hip. In the final part of my research, I developed a multi-frame 2D fluoroscopy to 3D model registration method for measuring the kinematics of post-operative knee joints. It uses a coarse-to-fine approach and applies the normalised EPD (NEPD) and SCV similarity measures together with a gradient descent optimisation method and an interpolation estimation one. In order to measure the kinematics of post- operative knee joints, after a TKA surgery, a 3D knee implant model can be registered with a number of single-plane fluoroscopy frames of the patient’s knee. Generally, when this number is quite high, the computational cost for registering the frames and a 3D model is expensive. Therefore, in order to speed up the registration process, a cubic spline interpolation prediction method is applied to initialise and estimate the 3D positions of the 3D model in each fluoroscopy frame instead of applying a registration algorithm on all the frames, one after the other. The estimated 3D positions are then tuned using a registration improvement step. The experimental results demonstrated that the proposed registration method is much faster than the best existing one and achieves almost the same accuracy. It also provides smooth registration results which can lead to more natural 3D modelling of joint movements.

Medical image is an important information source to understand the patient’s condition. As a result, interpreting the medical images is a critical part of the clinical procedure. However, physicians’ visual evaluation of medical images in clinics has a few challenges such as the limited human resources, the unavailability of appropriate experts, the increasing number ofmedical images, and so on.Deep learning is a subarea of machine learning that uses deep neural networks to learn the patterns behind the given data set. Deep learning showed excellent performances in image analysis problems including medical image analysis. In this dissertation, I propose and evaluate new methods for evaluating the images of three different vision problems inophthalmology.The first problem is the early detection of retinopathy of prematurity (ROP). ROP is a leading cause of childhood blindness globally, and early detection is a key to preventing ROP to progress to severe conditions. We developed two convolutional neural networks with different depths. The deeper model showed an excellent performance including bettermetrics than an experienced human expert.The second problem is about transfer learning in retinal vascular diseases. We propose a transfer learning method that uses the detection of a well-studied retinal vascular disease as a source problem and uses the knowledge to the detection of an under-studied retinal vascular disease. Our proposed method showed better performance with more robustness tothe stochasticity in the training process and the reduction of sample size.The final problem is to predict the treatment response to a drug from the baseline characteristics. Both symbolic features like clinical measurements and medical images are considered for the modeling. To merge the two types of input, we proposed two approaches. The results showed the potential of the proposed method to the problem.Deep learning is successfully applied to three medical image analysis problems in ophthalmology in this dissertation. These results offer key evidence for further development of deep learning-based medical image analysis systems in the future.

Medical image segmentation is a procedure to analyse an image’s content to find an organ, cancer, tumour, or possible abnormalities. Since hospitals and medical centres generate billions of images daily worldwide, manual analysis of the images is frustrating. Therefore there is a need to improve automatic techniques to examine the content of images. Deep Convolutional Neural Networks (DCNNs) are one of the most reliable and successful approaches to analyse images’ content. However, the main problem is a lack of rules to design a network, and trial and error is the usual approach to find a network structure along with its training parameters. Regarding the diversity of medical images, existing with various types of noises and artefacts, the limited number of available labelled medical images, and limited available computational facilities, designing a CNN for medical image analysis is even more complicated. Because of the importance of medical image segmentation, during the last decade, various CNNs are designed manually; however, most of these networks work well for the segmentation of a specific dataset or application. One of the solutions to address this problem is developing networks automatically. Neuroevolution, which is the combination of an evolutionary algorithm and Neural Networks (NNs), can automatically design a network. Evolutionary algorithms are relatively easy to understand and implement; however, they need considerable computation to evolve a network. Since Nerouevolution is computationally demanding, there is very limited previous work regarding applying Neuroevolution for medical image segmentation. Existing works just set up a part of the parameters to develop a network and have been applied to a limited number of datasets. The most significant drawbacks of existing works are lack of robustness and generalizability; also, most of them are computationally expensive. In this thesis, several Neroevolutionary-based frameworks are developed for 2D and 3D medical image segmentation. Firstly, a new block-based encoding model is developed to generate variable length 2D Deep Convolutional Neural Networks (DCNNs). The proposed encoding model could find appropriate values for several hyperparameters to create and train a DCNN. Also, a Genetic Algorithm (GA) is employed to evolve the generated networks. Besides, a comprehensive analysis is done to find an appropriate population size and generations, and consequently, an improved model is developed. In addition, to improve the results’ quality, an ensemble of found networks is utilised for final segmentation. Then to find a 3D evolutionary network, two approaches are examined. According to the proposed 2D model, a 3D model is developed to generate a population of 3D networks and evolve the 3D networks to find an appropriate 3D network for 3D medical image segmentation. Since evolving 3D networks is computationally expensive, a second approach is also introduced. In the second approach, the possibility of using a 2D evolutionary model to create a 3D network is examined and named Converted 3D network. Because of the diversity of medical images and the complexity of medical image analysis, sometimes more complicated CNN is needed. To address this issue, also another evolutionary model is developed in this thesis to generate more accurate and complex DCNNs using the combination of Dense and Residual blocks. In the proposed DenseRes model, a new encoding model is introduced, which is able to create a variable-length network with variable filter sizes within a block. In the DenseRes model, all required parameters to generate and train a network are included in the search. Most of the time, the Region Of Interest (ROI) is a small part of a medical image with almost the similar colour and texture of the surrounding organs. Therefore, more precise network architectures, like attention networks, are needed to process the images. To do so, two different approaches are introduced in this thesis to develop evolutionary attention networks. First, a 2D evolutionary attention model is proposed that is able to find an appropriate attention gate to transfer the block’s input to its output. Since some useful information will be lost during the downsampling in DCNNs, another 2D and 3D evolutionary attention framework is introduced to address this issue. In this model, besides creating a network structure along with its training parameters, an evolutionary algorithm is employed to find an appropriate model to recover and transfer feature maps from downsampling to the upsampling part of a network. The effectiveness of the proposed models is examined using various publicly available datasets. Results are compared with multiple manual and automatically designed models. The significant findings of this thesis can summarise as: (1) the proposed models obtain much better segmentation accuracy compared to state-of-the-art models, also, the proposed models are computationally cheap, even for developing 3D evolutionary networks; (2) converting a 2D evolutionary model to a 3D model is a reliable, fast, and accurate approach to create 3D networks; (3) including more constructive parameters in the search space can lead to more precise networks; (4) the initial population plays a significant role in the final results and decreasing training time; moreover, using variable filter sizes within a block can obtain better results compared to using a fixed one; (5) recovering a downsampling’s feature maps and transferring them to the corresponding upsampling part can considerably improve segmentation accuracy; (6) the proposed models are robust and general such that they can be applied for the segmentation of various medical images (CT and MRI) for different organs and tumour segmentation; (7) all the proposed encoding models are compatible with conventional crossover and mutation techniques, without any extra effort to create a new crossover technique or using a method to check the correctness of layers’ sequences.

Developing algorithms to better interpret images has been a fundamental problem in the field of medical imaging analysis. Recent advances in machine learning, especially deep convolutional neural networks (DCNNs), have demonstrated great improvement to the speed and accuracy of many medical image analysis tasks, such as image registration, anatomical structures/tissue segmentation, and computer-aided diagnosis. Despite previous progress, these problems still remain challenging due to the limited amount of labeled data, large anatomical variance among patients, etc.In this dissertation, we propose various approaches to address the aforementioned challenges in order to achieve better accuracy, higher efficiency, and use fewer labeled data. First, to address the difficulty of accurately detecting pulmonary nodules in its early stage, we propose a novel CAD framework that consists entirely of 3D DCNNs for detecting pulmonary nodules and reducing false positives in chest CT images. Second, to avoid training several deep learning models to solve nodule detection, false-positive reduction, and segmentation separately which may be suboptimal and resource-intensive, we propose NoduleNet to solve the three tasks jointly in a multi-task fashion. To avoid friction between different tasks and encourage feature diversification, we incorporate two major design tricks: 1) decoupled feature maps for nodule detection and false positive reduction, and 2) a segmentation refinement subnet for increasing the precision of nodule segmentation. Third, to address the limitation in scope and/or scale of previous works on organs-at-risk (OAR) delineation - with only a few OARs delineated and a limited number of samples tested, we propose a new deep learning model that can delineate a comprehensive set of 28 OARs in the head and neck area, trained with 215 CT samples collected and carefully annotated by experienced radiation oncologists with over ten years of experience. The accuracy of our model was compared to both previous state-of-the-art methods and a radiotherapy practitioner. Moreover, we deployed our deep learning model in actual RT planning of new patient cases, and evaluated the clinical utility of the model. Fourth, to reduce the information loss from cropping/downsampling 3D images due to limited GPU memory, we propose a new framework for combining 3D and 2D models, in which the segmentation is realized through high-resolution 2D convolutions, but guided by spatial contextual information extracted from a low-resolution 3D model. A self-attention mechanism is implemented to control which 3D features should be used to guide 2D segmentation. Last but not least, since DCNNs often require a large amount of data with manual annotation for training and are difficult to generalize to unseen classes, we propose a new few-shot segmentation framework RP-Net to address this issue. RP-Net has two important modules: 1) a context relation encoder (CRE) that uses correlation to capture local relation features between foreground and background regions, and 2) a recurrent mask refinement module that repeatedly uses the CRE and a prototypical network to recapture the change of context relationship and refine the segmentation mask iteratively.

In recent years, fully convolutional networks (FCNs) have become the state-of-the-art solutions for various image segmentation tasks. Duo to the network setups, however, the standard FCNs tend to have issues of 1) overlooking small components; and 2) lacking intermediate supervision.To tackle the first issue in FCNs, we develop a two-stage network solution and apply it for white-matter lesion segmentation task. In the first stage, we design three networks with different input sizes and train them with patches from brain MR scans. In the second stage, we process large and small lesion separately, and use ensemble-nets to combine the segmentation results. A number of network setups, including activation functions, training strategy and ensemble paradigms have been explored to improve the segmentation accuracy measured by Dice Similarity Coefficient.The address the second issue, we adopt the notions of network residual and Laplacian pyramids to design a Laplacian module as the building block for new FCNs. The networks are applied to segment follicles and follicular cells. In order to separate individual follicle instances, we take advantage of the topological relationship between follicles and colloid, and use colloid masks as the guidance to identify individual labels. We also utilize Sobel edge maps as a guiding loss to ensure the smoothness of the segmentation results.

Image analysis based on machine learning has gained prominence with the advent of deep learning, particularly in medical imaging. To be effective in addressing challenging image analysis tasks, however, conventional deep neural networks require large corpora of annotated training data, which are unfortunately scarce in the medical domain, thus often rendering fully-supervised learning strategies ineffective.This thesis devises for use in a variety of medical image analysis applications a series of novel deep learning methods, ranging from fully-supervised, single-task learning to scarcely-supervised, multi-task learning that makes efficient use of annotated training data. Specifically, its main contributions include (1) fully-supervised, single-task learning for the segmentation of pulmonary lobes from chest CT scans and the analysis of scoliosis from spine X-ray images; (2) supervised, single-task, domain-generalized pulmonary segmentation in chest X-ray images and retinal vasculature segmentation in fundoscopic images; (3) largely-unsupervised, multiple-task learning via deep generative modeling for the joint synthesis and classification of medical image data; and (4) partly-supervised, multiple-task learning for the combined segmentation and classification of chest and spine X-ray images.

The COVID-19 pandemic caused by the new coronavirus SARS-CoV-2 has changed the world as we know it. An early diagnosis is crucial in order to prevent new outbreaks and control its rapid spread. Medical imaging techniques, such as X-ray or chest computed tomography, are commonly used for this purpose due to their reliability for COVID-19 diagnosis. Computer-aided diagnosis systems could play an essential role in aiding radiologists in the screening process. In this work, a novel Deep Learning-based system, called COVID-XNet, is presented for COVID-19 diagnosis in chest X-ray images. The proposed system performs a set of preprocessing algorithms to the input images for variability reduction and contrast enhancement, which are then fed to a custom Convolutional Neural Network in order to extract relevant features and perform the classification between COVID-19 and normal cases. The system is trained and validated using a 5-fold cross-validation scheme, achieving an average accuracy of 94.43% and an AUC of 0.988. The output of the system can be visualized using Class Activation Maps, highlighting the main findings for COVID-19 in X-ray images. These promising results indicate that COVID-XNet could be used as a tool to aid radiologists and contribute to the fight against COVID-19.

Medical imaging is an essential tool in many areas of medical applications, used for both diagnosis and treatment. However, reading medical images and making diagnosis or treatment recommendations require specially trained medical specialists. The current practice of reading medical images is labor-intensive, time-consuming, costly, and error-prone. It would be more desirable to have a computer-aided system that can automatically make diagnosis and treatment recommendations. Recent advances in deep learning enable us to rethink the ways of clinician diagnosis based on medical images. Early detection has proven to be critical to give patients the best chance of recovery and survival. Advanced computer-aided diagnosis systems are expected to have high sensitivities and small low positive rates. How to provide accurate diagnosis results and explore different types of clinical data is an important topic in the current computer-aided diagnosis research. In this thesis, we will introduce 1) mammograms for detecting breast cancers, the most frequently diagnosed solid cancer for U.S. women, 2) lung Computed Tomography (CT) images for detecting lung cancers, the most frequently diagnosed malignant cancer, and 3) head and neck CT images for automated delineation of organs at risk in radiotherapy. First, we will show how to employ the adversarial concept to generate the hard examples improving mammogram mass segmentation. Second, we will demonstrate how to use the weakly labelled data for the mammogram breast cancer diagnosis by efficiently design deep learning for multi-instance learning. Third, the thesis will walk through DeepLung system which combines deep 3D ConvNets and Gradient Boosting Machine (GBM) for automated lung nodule detection and classification. Fourth, we will show how to use weakly labelled data to improve existing lung nodule detection system by integrating deep learning with probabilistic graphic model. Lastly, we will demonstrate the AnatomyNet which is thousands’ faster and more accurate than previous methods on automated anatomy segmentation.

Diagnostic medical imaging is an important non-invasive tool in medicine. It provides doctors (i.e., radiologists) with rich diagnostic information in clinical practice. Computer-aided diagnosis (CAD) schemes aim to provide a tool to assist the doctors for reading and interpreting medical images. Traditional CAD schemes are based on hand-crafted features and shallow supervised learning algorithms. They are greatly limited by the difficulties of accurate region segmentation and effective feature extraction. In this dissertation, our motivation is to apply deep learning techniques to address these challenges. We comprehensively investigated the feasibilities of applying deep learning technique to develop medical image segmentation and computer-aided diagnosis schemes for different imaging modalities and different tasks. First, we applied a two-step convolutional neural network architecture for selection of abdomen part and segmentation of subtypes of adipose tissue from abdominal CT images. We demonstrated high agreement between the segmentation generated by human and by our proposed deep learning models. Second, we explored to combine transfer learning technique with traditional hand-crafted features to improve the accuracy of breast mass classification from digital mammograms. Our results show that the ensemble of hand-crafted features and transferred features yields improvement of prediction performances. Third, we proposed a 3D fully convolutional network architecture with a novel coarse-to-fine residual module for prostate segmentation from MRI. State-of-art segmentation accuracy was obtained by using this model. We also investigated the feasibilities of applying fully convolutional network for prostate cancer detection based on multi-parametric MRI and obtained promising detection accuracy. Last, we proposed a novel cascaded neural network architecture with post-processing steps for nuclear segmentation from histology images. Superiority of the model was demonstrated by experiments. In summary, these study results demonstrated that deep learning is a very promising technology to help significantly improve efficacy of developing computer-aided diagnosis schemes of medical images and achieve higher performance.

Changes in the morphology of a skin lesion is indicative of melanoma, a deadly type of skin cancer. This dissertation proposes a temporal analysis method to monitor the vascularity, pigmentation, size and other critical morphological attributes of the lesion. Digital images of a skin lesion acquired during follow-up imaging sessions are input to the proposed system. The images are pre-processed to normalize variations introduced over time. The vascularity is modelled as the skin images’ red channel information and its changes by the Kullback-Leibler (KL) divergence of the probability density function approximation of histograms. The pigmentation is quantified as textural energy, changes in the energy and pigment coverage in the lesion. An optical flow field and divergence measure indicates the magnitude and direction of global changes in the lesion. Sub-surface change is predicted based on the surface skin lesion image with a novel approach. Changes in key morphological features such as lesions’ shape, color, texture, size, and border regularity are computed. Future trends of the skin lesions features are estimated by an auto-regressive predictor. Finally, the features extracted using deep convolutional neural networks and the hand-crafted lesion features are compared with classification metrics. An accuracy of 80.5%, specificity of 98.14%, sensitivity of 76.9% with a deep learning neural network is achieved. Experimental results show the potential of the proposed method to monitor a skin lesion in real-time during routine skin exams.

Interaction between a deformable elastic body and an internal or external fluid flow alters the flow pattern. This dissertation describes the effects of elasticity on flow in physiological scenarios. The first part of the thesis describes the influence of soft tissue compliance on flow in the upper airways of pediatric Down syndrome (DS) patients and adolescent Polycystic-Ovarian syndrome patients with obstructive sleep apnea (OSA). Computational fluid dynamics (CFD) of airflow is performed in pre and post-operative geometries of the DS pediatric airway to evaluate effectiveness of a surgery and address the importance of including the subject-specific tissue compliance. A tube law approach and a novel image analysis method are then presented to evaluate the circumferential variation in airway compliance for DS patients. An iterative finite element method is then described to non-invasively estimate patient-specific mechanical properties of the upper airway in these patients. The estimated mechanical properties for a single patient are applied to simulate airway obstruction during inspiratory airflow, before and after surgery. Sensitivity to different flow variables is analyzed and an operating map is created to establish the relationship between tissue elasticity and volumetric airflow. The necessity for performing fluid-structure interaction (FSI) in PCOS subjects with OSA is illustrated through a series of strain maps of upper airway tissue. An inverse methodology based on FSI simulations is described to characterize the soft-palate stiffness in these subjects. Differences in pre and post-operative airflow patterns and tissue motion in a PCOS patient are described using computational modeling and compared with the same for a healthy individual.The second part of the study describes computational FSI modeling of aortic blood flow in Turner syndrome (TS). A continuous measurement tool is developed to automatically compute the longitudinal variation in maximum aortic diameter. Patient-specific, three-dimensional geometric analysis of the aorta is then performed to evaluate localized changes in aorta morphology. A quasi-steady FSI modeling approach is described to estimate hemodynamic and biomechanical functional indices of cardiovascular disease in TS. Patterns of swirling flow, wall shear stress, aorta wall pressure, vessel wall displacement and mechanical stresses are compared for healthy and diseased aortas. The proposed computational technique is then applied to determine longitudinal changes in functional variables for multiple patients. The influence of including the aortic sinus is then addressed to obtain improved predictions of the spatial locations that are most vulnerable to progressive dilatation and vessel wall tear. FSI simulations of aortic blood flow with subject-specific flow boundary conditions and vessel wall stiffness derived from imaging is performed for individualized risk stratification in TS. A FSI methodology employing improved outflow conditions is then presented for TS patients and subjects diagnosed with aortic dissection. A longitudinal comparison of functional measures is presented for 2 TS subjects including the aortic root geometry, using a decoupled FSI methodology. Temporal variations in flow-induced shear and mechanical stress are then assessed using transient blood flow simulations. Finally, the influence of vessel wall thickness and blood rheology on functional indices is estimated in different aortic anomalies observed in TS.

Image segmentation and registration play active roles in machine vision and medicalimage analysis of historical data. Individually, the two has seen important researchcontributions, and the joint treatment of the two problems has become an active areaof research.In this thesis we will explore the joint problem of segmenting and registering atemplate image given a reference image. We formulate the joint problem throughan energy functional that integrates two well studied approaches in segmentationand registration: Geodesic Active Contours and nonlinear elastic registration. Inthe registration regime, the domain is modeled as a St. Venant-Kirchhoff material.We minimize the potential energy of this elastic system using variational methods,and derive an evolution equation which we solve using implicit-explicit integrationmethods. The numerical discretization of the problem allows us to take advantage ofthe Fast Fourier Transform.In the segmentation regime, we will adopt an active contours based energy with aweighted total variation penalty on the segmenting front. This particular choice allowsfor fast solution using the dual formulation of the total variation. The weight of thetotal variation penalty is an edge stopping function which depends on the deformingtemplate. This allows the segmenting front to accurately track spontaneous changesin the shape of objects embedded in the template image as it deforms.

Studies in clinical medicine often demand the quantitative analysis of medical images. These tasks need careful and time-consuming tracing and labeling on fine image structures, cost expensive medical expert labor and often suffer from low reproducibility. We present a collection of methods that quantify important parameters from cardiac computed tomography (CT) and magnetic resonance imaging (MRI) in a fully-automated mode. We first present atlas-based segmentation, active contours models and graph-based segmentation which are components in our novel framework. We then present our algorithms for automated quantification of epicardial fat volume (EFV) from non-contrast cardiac CT, automated pericardial fat volume (PFV) quantification from water\/fat-resolved whole-heart non-contrast coronary magnetic resonance angiography (MRA) and automated coronary calcium scoring (CCS) from non-contrast cardiac CT images. Algorithm quantification results are validated on test scans with ground truth annotated by expert radiologists. Our approaches may potentially be applied in a clinical setting, allowing for accurate quantification of EFV, PFV and CCS without tedious manual tracing.

The effectiveness of colonoscopy depends on the quality of the inspection of the colon. There was no automated measurement method to evaluate the quality of the inspection. This thesis addresses this issue by investigating an automated post-procedure quality measurement technique and proposing a novel approach automatically deciding a percentage of stool areas in images of digitized colonoscopy video files. It involves the classification of image pixels based on their color features using a new method of planes on RGB (red, green and blue) color space. The limitation of post-procedure quality measurement is that quality measurements are available long after the procedure was done and the patient was released. A better approach is to inform any sub-optimal inspection immediately so that the endoscopist can improve the quality in real-time during the procedure. This thesis also proposes an extension to post-procedure method to detect stool, bite-block, and blood regions in real-time using color features in HSV color space. These three objects play a major role in quality measurements in colonoscopy. The proposed method partitions very large positive examples of each of these objects into a number of groups. These groups are formed by taking intersection of positive examples with a hyper plane. This hyper plane is named as 'positive plane'. 'Convex hulls' are used to model positive planes. Comparisons with traditional classifiers such as K-nearest neighbor (K-NN) and support vector machines (SVM) proves the soundness of the proposed method in terms of accuracy and speed that are critical in the targeted real-time quality measurement system.

This dissertation explores innovative applications of diverse image registration techniques in the context of video coding and medical image analysis. Motion-compensated prediction is the key to video coding. Sophisticated motion-compensation technologies need to be integrated into future codecs to ensure its continual improvement. Previous works have explored alternatives to classical motion models that can only estimate translational movements in videos. But the cumulative rate-distortion performance has not been significant enough to see such approaches adopted in mainstream standards. In this thesis, an advanced motion compensation technique is presented that effectively predicts non-translational motion. A novel elastic motion model is derived for this purpose based on elastic image registration with 2-D cosine basis functions. To achieve superior performance, the proposed scheme takes advantage of larger macro-blocks with multi-level partitioning. Next, a novel block-partitioning scheme is introduced that attempts to slice the blocks based on motion-field discontinuity or geometry of edges rather than traditional predefined squares or rectangles. Both geometry-adaptive partitioning and higher-order elastic motion models are advanced coding techniques and can be considered as good candidates for future coders. However, it is vital that they are additive in performance, non-interfering and maintain justifiable complexity. Therefore, the elastic motion compensation platform has been extended by incorporating geometry-adaptive partitioning to attain even better compression. Finally, a new form of image registration, also known as 3D/2D registration is studied for analyzing relative movements in human joints from fluoroscopy images. This technology has many promising applications e.g. image-guided surgery, interventions and radiotherapy etc. However, precise measurement of out-of-plane movements remains a challenge for this to become a reality. In this study, a fast and accurate 3D/2D registration technique is put forward for kinematic analysis. The new algorithm takes advantage of a new multi-modal similarity measure called ‘sum of conditional variances’, a coarse-to-fine Laplacian of Gaussian filtering approach for robust gradient descent optimization and a novel technique for the analytic calculation of the required gradients for out-of-plane rotations. The accuracy and speed of the proposed method has been verified using clinical ‘gold-standard’ Roentgen Stereo Analysis (RSA).

Humans can effortlessly see and interpret the world around them. Yet, the development of computer vision systems capable of doing the same is an extremely challenging task. In developing vision systems, computer vision experts should explicitly express the knowledge required to guide the systems. The complex nature of vision systems however means that vision experts are only quasiexperts. They lack a strong understanding of the specific vision task or the ability to articulate their knowledge. Although some approaches automatically derive this knowledge using pattern recognition and machine learning methods, lack of sufficient labelled data makes these infeasible in some domains. The evolving nature of expertise, incrementally available data and shifting nature of the underlying vision tasks make it difficult for experts to express all knowledge a priori. This is especially true of domains such as medical imaging, where data tends to trickle in over time and expert's discover the precise knowledge only by applying different vision algorithms over time. Therefore, vision experts develop systems incrementally, by trial-and-error, and each ad-hoc revision to the system, although well intended, may in fact degrade the system performance. This thesis proposes ways to mitigate the risks that ad-hoc incremental revisions pose to vision systems. It presents ProcessRDR and ProcessNet frameworks, which are an adaptation of the incremental validated change strategy employed by Ripple Down Rules for the vision domain. These frameworks assist the expert in systematically revising parts of a system, while maintaining the integrity of the whole. The thesis also establishes the role of quasi-expertise in vision domains, and studies its influences on the quality of the resulting vision systems. The studies suggest ways for experts to mitigate the influences of quasi-expertise and support the need for incremental development of vision systems. Incremental development of computer vision systems give experts the capacity to adapt vision systems, as better expertise and more data becomes available. Although the frameworks do not eliminate quasiexpertise or lack of labelled data, they do support the expert in incrementally developing vision systems despite it. The systematic incremental revisions mean that the vision systems can continue to improve over time.

The development of faster and higher resolution MR imaging devices has made accessible mass quantities of image data. Much information can be extracted by analyzing these high spatial resolution images. For instance, tissue volumes, which can be measured through MR images, are used as an indicator in many clinical applications and research studies. Research studies involving tissue volume analysis often require the processing of a vast amount of data hence manual segmentation of the images by experts is very time-consuming. Data segmented by human experts are also likely to show inter- and intra-observer inconsistency. For these reasons, automated segmentation of MR images is of great importance and interest. MR images present many challenges for automated segmentation. In addition to noise, MR images are also corrupted by problems specific to MR imaging such as intensity inhomogeneity. Furthermore, poor contrast at tissue boundaries due to multiple tissues having similar MR intensities also present problems. On the other hand, strong contours may exist where boundaries are not desired, because multiple tissues with very different MR intensities may be present within the same anatomical structure. Prior information, such as spatial atlases and shape priors can be very powerful in these cases. In this thesis, we developed highly accurate and robust graph cuts-based method that automatically segments MR images. The images we are most interested in are those that cannot be correctly segmented using intensity information alone. We developed models for robustly incorporating prior information such as spatial atlas and geometric or statistical shape priors into the efficient graph cuts segmentation framework. Specifically, we developed methods to incorporate spatial atlas, statistical and geometric shape priors with graph cuts for MR image segmentation. We tested our methods on MR brain, abdomen and cardiac images with intensity inhomogeneity, poor contrast at desired boundaries and/or strong contrast at undesired boundaries and obtained encouraging results. Finally, we proposed a way of dealing with objects with curvy boundaries.