Search

Rationale: The management of indeterminate pulmonary nodules (IPNs) remains challenging, resulting in invasive procedures and delays in diagnosis and treatment. Strategies to decrease the rate of unnecessary invasive procedures and optimize surveillance regimens are needed. Objectives: To develop and validate a deep learning method to improve the management of IPNs. Methods: A Lung Cancer Prediction Convolutional Neural Network model was trained using computed tomography images of IPNs from the National Lung Screening Trial, internally validated, and externally tested on cohorts from two academic institutions. Measurements and Main Results: The areas under the receiver operating characteristic curve in the external validation cohorts were 83.5% (95% confidence interval [CI], 75.4–90.7%) and 91.9% (95% CI, 88.7–94.7%), compared with 78.1% (95% CI, 68.7–86.4%) and 81.9 (95% CI, 76.1–87.1%), respectively, for a commonly used clinical risk model for incidental nodules. Using 5% and 65% malignancy thresholds defining low- and high-risk categories, the overall net reclassifications in the validation cohorts for cancers and benign nodules compared with the Mayo model were 0.34 (Vanderbilt) and 0.30 (Oxford) as a rule-in test, and 0.33 (Vanderbilt) and 0.58 (Oxford) as a rule-out test. Compared with traditional risk prediction models, the Lung Cancer Prediction Convolutional Neural Network was associated with improved accuracy in predicting the likelihood of disease at each threshold of management and in our external validation cohorts. Conclusions: This study demonstrates that this deep learning algorithm can correctly reclassify IPNs into low- or high-risk categories in more than a third of cancers and benign nodules when compared with conventional risk models, potentially reducing the number of unnecessary invasive procedures and delays in diagnosis.

BackgroundEstimation of the risk of malignancy in pulmonary nodules detected by CT is central in clinical management. The use of artificial intelligence (AI) offers an opportunity to improve risk prediction. Here we compare the performance of an AI algorithm, the lung cancer prediction convolutional neural network (LCP-CNN), with that of the Brock University model, recommended in UK guidelines.MethodsA dataset of incidentally detected pulmonary nodules measuring 5–15 mm was collected retrospectively from three UK hospitals for use in a validation study. Ground truth diagnosis for each nodule was based on histology (required for any cancer), resolution, stability or (for pulmonary lymph nodes only) expert opinion. There were 1397 nodules in 1187 patients, of which 234 nodules in 229 (19.3%) patients were cancer. Model discrimination and performance statistics at predefined score thresholds were compared between the Brock model and the LCP-CNN.ResultsThe area under the curve for LCP-CNN was 89.6% (95% CI 87.6 to 91.5), compared with 86.8% (95% CI 84.3 to 89.1) for the Brock model (p≤0.005). Using the LCP-CNN, we found that 24.5% of nodules scored below the lowest cancer nodule score, compared with 10.9% using the Brock score. Using the predefined thresholds, we found that the LCP-CNN gave one false negative (0.4% of cancers), whereas the Brock model gave six (2.5%), while specificity statistics were similar between the two models.ConclusionThe LCP-CNN score has better discrimination and allows a larger proportion of benign nodules to be identified without missing cancers than the Brock model. This has the potential to substantially reduce the proportion of surveillance CT scans required and thus save significant resources.

Artificial intelligence (AI) has been present in some guise within the field of radiology for over 50 years. The first studies investigating computer-aided diagnosis in thoracic radiology date back to the 1960s, and in the subsequent years, the main application of these techniques has been the detection and classification of pulmonary nodules. In addition, there have been other less intensely researched applications, such as the diagnosis of interstitial lung disease, chronic obstructive pulmonary disease, and the detection of pulmonary emboli. Despite extensive literature on the use of convolutional neural networks in thoracic imaging over the last few decades, we are yet to see these systems in use in clinical practice. The article reviews current state-of-the-art applications of AI and in detection, classification, and follow-up of pulmonary nodules and how deep-learning techniques might influence these going forward. Finally, we postulate the impact of these advancements on the role of radiologists and the importance of radiologists in the development and evaluation of these techniques.

This paper explores the use of self-supervised deep learning in medical imaging in cases where two scan modalities are available for the same subject. Specifically, we use a large publicly-available dataset of over 20,000 subjects from the UK Biobank with both whole body Dixon technique magnetic resonance (MR) scans and also dual-energy x-ray absorptiometry (DXA) scans. We make three contributions: (i) We introduce a multi-modal image-matching contrastive framework, that is able to learn to match different-modality scans of the same subject with high accuracy. (ii) Without any adaption, we show that the correspondences learnt during this contrastive training step can be used to perform automatic cross-modal scan registration in a completely unsupervised manner. (iii) Finally, we use these registrations to transfer segmentation maps from the DXA scans to the MR scans where they are used to train a network to segment anatomical regions without requiring ground-truth MR examples. To aid further research, our code will be made publicly available., Accepted as a full paper to MICCAI 2021. Code will be made publicly available before September 27th 2021

INTRODUCTION: Deep Learning has been proposed as promising tool to classify malignant nodules. Our aim was to retrospectively validate our Lung Cancer Prediction Convolutional Neural Network (LCP-CNN), which was trained on US screening data, on an independent dataset of indeterminate nodules in an European multicentre trial, to rule out benign nodules maintaining a high lung cancer sensitivity.METHODS: The LCP-CNN has been trained to generate a malignancy score for each nodule using CT data from the U.S. National Lung Screening Trial (NLST), and validated on CT scans containing 2106 nodules (205 lung cancers) detected in patients from from the Early Lung Cancer Diagnosis Using Artificial Intelligence and Big Data (LUCINDA) study, recruited from three tertiary referral centers in the UK, Germany and Netherlands. We pre-defined a benign nodule rule-out test, to identify benign nodules whilst maintaining a high sensitivity, by calculating thresholds on the malignancy score that achieve at least 99 % sensitivity on the NLST data. Overall performance per validation site was evaluated using Area-Under-the-ROC-Curve analysis (AUC).RESULTS: The overall AUC across the European centers was 94.5 % (95 %CI 92.6-96.1). With a high sensitivity of 99.0 %, malignancy could be ruled out in 22.1 % of the nodules, enabling 18.5 % of the patients to avoid follow-up scans. The two false-negative results both represented small typical carcinoids.CONCLUSION: The LCP-CNN, trained on participants with lung nodules from the US NLST dataset, showed excellent performance on identification of benign lung nodules in a multi-center external dataset, ruling out malignancy with high accuracy in about one fifth of the patients with 5-15 mm nodules.

Machine learning based lung cancer prediction models have been proposed to assist clinicians in managing incidental or screen detected indeterminate pulmonary nodules. Such systems may be able to reduce variability in nodule classification, improve decision making and ultimately reduce the number of benign nodules that are needlessly followed or worked-up. In this article, we provide an overview of the main lung cancer prediction approaches proposed to date and highlight some of their relative strengths and weaknesses. We discuss some of the challenges in the development and validation of such techniques and outline the path to clinical adoption.

Background:Magnetic resonance imaging (MRI) offers a non-invasive and objective method of early diagnosis and classification, monitoring disease burden and treatment response for patients (pts) with axial spondyloarthritis (axSpA) including ankylosing spondylitis (AS).1Numerous scoring schemes such as the AS Spine MRI Activity (ASspiMRIa) score are available for the quantitative assessment of MRI, but are subject to intra- and inter-rater variability, labor intensive and costly. Nevertheless, quantification of MRI changes has become an important tool to demonstrate treatment success of biologic drugs in axSpA.Objectives:To evaluate the performance of machine learning (ML) based software for automated Berlin grading of spinal MRI bone marrow oedema in pts with AS and compare with expert scoring.Methods:Fully automated ML software (Figure) was developed to detect and label 23 vertebrae, define vertebral units (VU) as per the Berlin modification of the ASspiMRIa score, and score each VU as either 0 (score of 0) or 1 (score of 1, 2 or 3). The ML algorithm was based on the previously developed SpineNet software.2Analysis included 108 pts from the secukinumab MEASURE 1 study3, in which imaging was done using T1 and STIR sagittal MRI at baseline and Weeks 16, 52, 104, 156 and 208. Two expert readers, blinded to treatment and visit, evaluated all images by ASspiMRIa score. The scores from Reader 2 (R2) were binned into two groups: 0 vs 1, 2, or 3. As a result of multiple pt time points and expert reading sessions, the complete dataset comprised of 10,988 VU. Ten-way cross-validation at per-VU was used to train and validate the ML software. The dataset was split into 10 randomly selected subsets, ensuring that each pt appears in only one subset, after which 8 subsets were used for training the ML software, 1 was used to check for correct training and 1 was used for validation. The process was repeated ten times such that all 10 subsets were used for validation. Accuracy weighted for the frequency of each category, sensitivity and specificity were calculated using scores from R2 as reference. Intra-reader accuracy was also calculated.Results:Accuracy of the software in relation to expert reader scores was 67% with a sensitivity of 0.63 and specificity of 0.70. The intra-reader accuracy was 71% and 77% for R1 and R2, respectively. Individual VU scoring of the Software vs. R2 are presented in the Table as a confusion matrix.Conclusion:Automated scoring of MR images in AS pts provided moderate agreement to that of expert reader-based assessments. ML software has potential to provide an automated guided-reading approach to scoring MR images, which may enable further clinical insights.References:[1]Lukas C, et al. J Rheumatol. 2007;34:862-70.[2]Jamaludin A, et al. Eur Spine J. 2017;26:1374-83.[3]Baeten D, et al. N Engl J Med. 2015;373,2534-48.Figure.Processing pipeline of automated Berlin scoring softwareTable.Confusion matrix between the software and R2SoftwareScore = 0SoftwareScore = 1, 2 or 3Total VU scoredR2 Score = 07199 (70%)3068 (30%)10,267R2 Score = 1, 2 or 3251 (35%)475 (65%)7267,4503,54310,993Percentages calculated as a fraction over the total in each row. Overall accuracy is the average of the highlighted percentages.Disclosure of Interests:Amir Jamaludin: None declared, Rhydian Windsor: None declared, Sarim Ather: None declared, Timor Kadir: None declared, Andrew Zisserman: None declared, Juergen Braun Grant/research support from: Abbvie (Abbott), Amgen, BMS, Boehringer, Celgene, Celltrion, Centocor, Chugai, Eli Lilly and Company, Medac, MSD (Schering Plough), Mundipharma, Novartis, Pfizer (Wyeth), Roche, Sanofi- Aventis, and UCB Pharma, Consultant of: Abbvie (Abbott), Amgen, BMS, Boehringer, Celgene, Celltrion, Centocor, Chugai, EBEWE Pharma, Eli Lilly and Company, Medac, MSD (Schering-Plough), Mundipharma, Novartis, Pfizer (Wyeth), Roche, Sanofi-Aventis, and UCB Pharma, Speakers bureau: Abbvie (Abbott), Amgen, BMS, Boehringer, Celgene, Celltrion, Centocor, Chugai, EBEWE Pharma, Eli Lilly and Company, Medac, MSD (Schering-Plough), Mundipharma, Novartis, Pfizer (Wyeth), Roche, Sanofi-Aventis, and UCB Pharma, Lianne S. Gensler Grant/research support from: Pfizer, Novartis, UCB, Consultant of: AbbVie, Eli Lilly, GSK, Novartis, UCB, Pedro Machado Consultant of: Abbvie, Celgene, Janssen, Lilly, MSD, BMS, Novartis, Pfizer, Roche and UCB, Speakers bureau: AbbVie, Centocor, Eli Lilly, Janssen, MSD, Novartis, Pfizer and UCB Pharma, Mikkel Ǿstergaard Grant/research support from: AbbVie, Bristol-Myers Squibb, Celgene, Merck, and Novartis, Consultant of: AbbVie, Bristol-Myers Squibb, Boehringer Ingelheim, Celgene, Eli Lilly, Hospira, Janssen, Merck, Novartis, Novo Nordisk, Orion, Pfizer, Regeneron, Roche, Sandoz, Sanofi, and UCB, Speakers bureau: AbbVie, Bristol-Myers Squibb, Boehringer Ingelheim, Celgene, Eli Lilly, Hospira, Janssen, Merck, Novartis, Novo Nordisk, Orion, Pfizer, Regeneron, Roche, Sandoz, Sanofi, and UCB, Denis Poddubnyy Grant/research support from: AbbVie, MSD, Novartis, and Pfizer, Consultant of: AbbVie, Bristol-Myers Squibb, Eli Lilly, MSD, Novartis, Pfizer, Roche, UCB, Speakers bureau: AbbVie, Bristol-Myers Squibb, Eli Lilly, MSD, Novartis, Pfizer, Roche, UCB, Thibaud Coroller Shareholder of: Novartis, Employee of: Novartis, Brian Porter Shareholder of: Novartis, Employee of: Novartis, Shephard Mpofu Shareholder of: Novartis, Employee of: Novartis, Aimee Readie Shareholder of: Novartis, Employee of: Novartis

We describe a method to automatically predict scoliosis in Dual-energy X-ray Absorptiometry (DXA) scans. We also show that in- termediate representations, which in our case are segments of body parts, help improve performance. Hence, we propose a two step process for pre- diction: (i) we learn to segment body parts via a segmentation Convolutional Neural Network (CNN), which we show outperforms the noisy labels it was trained on, and (ii) we predict with a classification CNN that uses as input both the raw DXA scan and also the intermediate representation, i.e. the segmented body parts. We demonstrate that this two step process can predict scoliosis with high accuracy, and can also localize the spinal curves (i.e. geometry) without additional supervision. Furthermore, we also propose a soft score of scoliosis based on the classification CNN which correlates to the severity of scoliosis.

This paper describes a novel automatic system for Modic changes classification of vertebral endplates. Modic changes are classes of vertebral degenerations visible as intensity variations in magnetic resonance images (MRI). The system operates on T1 and T2 MRI. We introduce three main novelties: 1. a vertebrae alignment scheme via precise bounding boxes obtained through corner localisation, 2. vertebral endplate classification in three dimensions, and 3. Modic changes classification. The system was trained and validated using a large dataset of 785 patients, containing MRIs sourced from a wide range of acquisition protocols. The proposed system achieved 87.8 % classification accuracy on our dataset.

The objective of this work is to automatically produce radiological gradings of spinal lumbar MRIs and also localize the predicted pathologies. We show that this can be achieved via a Convolutional Neural Network (CNN) framework that takes intervertebral disc volumes as inputs and is trained only on disc-specific class labels. Our contributions are: (i) a CNN architecture that predicts multiple gradings at once, and we propose variants of the architecture including using 3D convolutions; (ii) showing that this architecture can be trained using a multi-task loss function without requiring segmentation level annotation; and (iii) a localization method that clearly shows pathological regions in the disc volumes. We compare three visualization methods for the localization. The network is applied to a large corpus of MRI T2 sagittal spinal MRIs (using a standard clinical scan protocol) acquired from multiple machines, and is used to automatically compute disk and vertebra gradings for each MRI. These are: Pfirrmann grading, disc narrowing, upper/lower endplate defects, upper/lower marrow changes, spondylolisthesis, and central canal stenosis. We report near human performances across the eight gradings, and also visualize the evidence for these gradings localized on the original scans.

The goal of this work is to accurately and reliably localize anatomical landmarks in 3D Computed Tomography (CT) scans of the upper bodies of cancer patients even in the presence of pathologies and imaging artifacts that may markedly change the appearances of anatomical structures. We propose a method based on dense matching of parts-based graphical models. For landmark localization, we replace population averaged models by personalized models that are adapted to each test image at runtime. We do so by jointly leveraging weighted combinations of labeled training exemplars. We report results for localizing standard anatomical landmarks in clinical 3D CT volumes, using a database of 83 lung cancer patients. We compare our method against both (baseline) population averaged graphical models and against atlas-based deformable registration and show the method is in each case able to localize landmarks with significantly improved reliability and accuracy.

A significant proportion of patients scanned in a clinical setting have follow-up scans. We show in this work that such longitudinal scans alone can be used as a form of 'free' self-supervision for training a deep network. We demonstrate this self-supervised learning for the case of T2-weighted sagittal lumbar Magnetic Resonance Images (MRIs). A Siamese convolutional neural network (CNN) is trained using two losses: (i) a contrastive loss on whether the scan is of the same person (i.e. longitudinal) or not, together with (ii) a classification loss on predicting the level of vertebral bodies. The performance of this pre-trained network is then assessed on a grading classification task. We experiment on a dataset of 1016 subjects, 423 possessing follow-up scans, with the end goal of learning the disc degeneration radiological gradings attached to the intervertebral discs. We show that the performance of the pre-trained CNN on the supervised classification task is (i) superior to that of a network trained from scratch; and (ii) requires far fewer annotated training samples to reach an equivalent performance to that of the network trained from scratch., Comment: 3rd Workshop on Deep Learning in Medical Image Analysis

We describe a method to automatically predict radiological scores in spinal Magnetic Resonance Images (MRIs). Furthermore, we also identify and localize the pathologies that are the reasons for these scores. We term these pathological regions the ``evidence hotspots'. Our contributions are two fold: (i) a Convolutional Neural Network (CNN) architecture and training scheme to predict multiple radiological scores on multiple slice sagittal MRIs. The scheme uses multi-task CNN training with augmentation, and handles the class imbalance common in medical classification tasks. (ii) the prediction of a heat-map of evidence hotspots for each score. For both of these, all that is required for training is the class label of the disc or vertebrae, no stronger supervision (such as slice labels) is needed. We report state-of-the-art and near-human performances across multiple radiological scorings including: Pfirrmann grading, disc narrowing, endplate defects, and marrow changes.

In this paper, we present a number of enhancements to the Kadir/Brady salient region detector which result in a significant improvement in performance. The modifications we make include: stabilising the difference between consecutive scales when calculating the inter-scale saliency, a new sampling strategy using overlap of pixels, partial volume estimation and parzen windowing. Repeatability is used as the criterion for evaluating the performance of the algorithm. We observe the repeatability for distinctive regions selected from an image and from the same image after applying a particular transformation. The transformations we use include planar rotation, pixel translation, spatial scaling, and intensity shifts and scaling. Experimental results show that the average repeatability rate is improved from 46% to approximately 78% when all the enhancements are applied. We also compare our algorithm with other region detectors on a set of sequences of real images, and our detector outperforms most of the state of the art detectors. © 2006 Elsevier Inc. All rights reserved.

Objective The segmentation of ultrasound (US) images is useful for several applications in computer aided interventions including the registration of pre-operative CT or MRI to intra-operative US. Shadowing, intensity inhomogeneity and speckle are the common effects on US images. They render the segmentation algorithms developed for other modalities inappropriate due to poor robustness. We present a novel method for classification of hepatic structures including vasculature and liver parenchyma on US images.

This paper describes a fully automatic system for obtaining the standard Pfirrmann degeneration grading of individual intervertebral spinal discs in T2 MRI scans. It involves detecting and labeling all the vertebrae in the scan and then learning a regression from the disc region to the grading. In developing the regression function we investigate a spectrum of support regions which involve differing degrees of segmentation of the scan: our intention is to ascertain to what extent segmentation is necessary or detrimental in obtaining robust and accurate measurements. The methods are assessed on a heterogeneous clinical dataset containing 1,710 Pfirrmann-graded discs, from 285 symptomatic back pain patients. We are able to predict the grade to \(\pm 1\) precision at 85.8 % accuracy. Our novel method proposes new image features that outperform previous features and utilizes techniques to improve robustness to MR imaging variations.

The goal of this work is to reliably and accurately localize anatomical landmarks in 3-D computed tomography scans, particularly for the deformable registration of whole-body scans, which show huge variation in posture, and the spatial distribution of anatomical features. Parts-based graphical models (GM) have shown attractive properties for this task because they capture naturally anatomical relationships between landmarks. Unfortunately, standard GMs are learned from manually annotated training images and the quantity of landmarks is limited by the high cost of expert annotation. We propose a novel method that automatically learns new corresponding landmarks from a database of 3-D whole-body CT scans, using a limited initial set of expert-labeled ground-truth landmarks. The newly learned landmarks, called B-landmarks, are used to build enriched GMs. We compare our method of deformable registration based on such GM landmarks to a conventional deformable registration method and to a “baseline” state-of-the-art GM. The results show our method finds new relevant anatomical correspondences and improves by up to 35% the matching accuracy of highly variable skeletal and soft-tissue landmarks of clinical interest.

Many computer vision problems can be considered to consist of two main tasks: the extraction of image content descriptions and their subsequent matching. The appropriate choice of type and level of description is of course task dependent, yet it is generally accepted that the low-level or so called early vision layers in the Human Visual System are context independent. This paper concentrates on the use of low-level approaches for solving computer vision problems and discusses three inter-related aspects of this: saliencys scale selection and content description. In contrast to many previous approaches which separate these tasks, we argue that these three aspects are intrinsically related. Based on this observation, a multiscale algorithm for the selection of salient regions of an image is introduced and its application to matching type problems such as tracking, object recognition and image retrieval is demonstrated.

We describe a method to automatically detect and label the vertebrae in human lumbar spine MRI scans. Our contribution is to show that marrying two strong algorithms (the DPM object detector of Felzenszwalb et al. [1], and inference using dynamic programming on chains) together with appropriate modelling, results in a simple, computationally cheap procedure, that achieves state-of-the-art performance. The training of the algorithm is principled, and heuristics are not required. The method is evaluated quantitatively on a dataset of 371 MRI scans, and it is shown that the method copes with pathologies such as scoliosis, joined vertebrae, deformed vertebrae and disks, and imaging artifacts. We also demonstrate that the same method is applicable (without retraining) to CT scans.

Multi-atlas segmentation has been widely used to segment various anatomical structures. The success of this technique partly relies on the selection of atlases that are best mapped to a new target image after registration. Recently, manifold learning has been proposed as a method for atlas selection. Each manifold learning technique seeks to optimize a unique objective function. Therefore, different techniques produce different embeddings even when applied to the same data set. Previous studies used a single technique in their method and gave no reason for the choice of the manifold learning technique employed nor the theoretical grounds for the choice of the manifold parameters. In this study, we compare side-by-side the results given by 3 manifold learning techniques (Isomap, Laplacian Eigenmaps and Locally Linear Embedding) on the same data set. We assess the ability of those 3 different techniques to select the best atlases to combine in the framework of multi-atlas segmentation. First, a leave-one-out experiment is used to optimize our method on a set of 110 manually segmented atlases of hippocampi and find the manifold learning technique and associated manifold parameters that give the best segmentation accuracy. Then, the optimal parameters are used to automatically segment 30 subjects from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). For our dataset, the selection of atlases with Locally Linear Embedding gives the best results. Our findings show that selection of atlases with manifold learning leads to segmentation accuracy close to or significantly higher than the state-of-the-art method and that accuracy can be increased by fine tuning the manifold learning process.

The goal of this work is to accurately localize anatomical structures in 3D Computed Tomography (CT) scans. Clinical protocols vary greatly in the scanned Field-Of-View (FOV), hence a typical scan may contain only a subset of a larger group of anatomical landmarks learned from the whole body. We extend parts-based graphical models for landmark localization in such partial FOV scans, by augmenting the state space with extra states with learnt penalties, which account for placements of each invisible landmark outside of the FOV. The resulting algorithm maintains the attractive computational properties of the original approach, whilst achieving accurate localization of the visible landmarks. We report results for localizing 11 standard anatomical landmarks (skeletal and soft tissue), using 3,000 truncated 3D CT images from a database of lung cancer patients. We compare our method against a baseline graphical model and report landmark localization error reduced on average by 54%, for a wide range of previously unknown FOV sizes.

In this paper, we investigate a number of Bayesian techniques for predicting 1-year- survival and making treatment selection recommendations for lung cancer. We have carried out two sets of experiments on the English Lung Cancer Dataset. For 1-year-survival prediction, the Naïve Bayes (NB) algorithm achieved an area under the curve value of 81%, outperforming the Bayesian Networks learned by the M(3) and K2 structure learning algorithms. For treatment recommendation, the Bayesian Network, whose structure was learned by the MC(3) algorithm, has marginally outperformed NB, based on producing concordant results with the recorded treatments in the dataset. We observed that in cases where the classifier recommendations were discordant with the recorded treatments, the 1-year-survival rate decreased by 15%. We also observed that discordance between the classifier and the dataset was more dominant in cases where the recorded treatment was non-curative or was not frequently encountered in the dataset.

Survival prediction and treatment selection in lung cancer care are characterised by high levels of uncertainty. Bayesian Networks (BNs), which naturally reason with uncertain domain knowledge, can be applied to aid lung cancer experts by providing personalised survival estimates and treatment selection recommendations. Based on the English Lung Cancer Database (LUCADA), we evaluate the feasibility of BNs for these two tasks, while comparing the performances of various causal discovery approaches to uncover the most feasible network structure from expert knowledge and data. We show first that the BN structure elicited from clinicians achieves a disappointing area under the ROC curve of 0.75 (± 0.03), whereas a structure learned by the CAMML hybrid causal discovery algorithm, which adheres with the temporal restrictions, achieves 0.81 (± 0.03). Second, our causal intervention results reveal that BN treatment recommendations, based on prescribing the treatment plan that maximises survival, can only predict the recorded treatment plan 29% of the time. However, this percentage rises to 76% when partial matches are included.

Alzheimer's disease (AD) severely affects the hippocampus: it loses mass and shrinks as the disease advances. Thus delineation of the hippocampus is an important task in the clinical study of AD. Because of its simplicity and good performance, multi-atlas based segmentation has become a popular approach for medical image segmentation. We propose to use manifold learning for atlas selection in the framework of multi-atlas based segmentation. The framework only benefits when selecting atlases similar to the target image. Since manifold learning assigns each image a coordinate in low-dimensional space by respecting the neighborhood relationship, it is well suited for atlas selection. The key contribution is that we use manifold learning based on a metric derived from non-rigid transformation as the resulting embedding better captures deformations or shape differences between images than similarity measures based on voxel intensity. The proposed method is evaluated in a leave-one-out experiment on a set of 110 hippocampus images; we report mean Dice score of 0.9114 (0.0227). The method was validated against a state-of-the-art method for hippocampus segmentation.

We propose a method for accurately localizing anatomical landmarks in 3D medical volumes based on dense matching of parts-based graphical models. Our novel approach replaces population mean models by jointly leveraging weighted combinations of labeled exemplars (both spatial and appearance) to obtain personalized models for the localization of arbitrary landmarks in upper body images. We compare the method to a baseline population-mean graphical model and atlas-based deformable registration optimized for CT-CT registration, by measuring the localization accuracy of 22 anatomical landmarks in clinical 3D CT volumes, using a database of 83 lung cancer patients. The average mean localization error across all landmarks is 2.35 voxels. Our proposed method outperforms deformable registration by 73%, 93% for the most improved landmark. Compared to the baseline population-mean graphical model, the average improvement of localization accuracy is 32%; 67% for the most improved landmark.

Radiofrequency ablation of liver cancer is a minimally invasive alternative to open surgery. Typically, the preoperative planning is done on an MR (or CT) scan, while the intervention relies on ultrasound (US) guidance. Registration of intra-operative US and preoperative MR (or CT) would assist navigation and increase the confidence of RFA needle positioning. In this paper we present a novel method for registration of US and MR images of the liver. Hepatic vessels are extracted from 2D US by an algorithm that models US dynamics. It generates 2D probability maps representing hepatic vessels which are then combined into probability volumes. A multi-resolution registration framework performs registration of the pre-processed MR with two 3D vessel probability images. The accuracy, robustness and speed of the method were assessed by registering eight US/MR datasets. High robustness (86%) and reasonable accuracy (1.98 degrees, 4.10 mm), acceptable for the RFA clinical application, suggest that the method has a good potential for intra-operative use.

We propose a method for reliably and accurately identifying anatomical landmarks in 3D CT volumes based on dense matching of parts-based graphical models. Such a system can be used to establish reliable correspondences in medical images which can be useful on their own or as part of more complex processing e.g. atlas building. We propose and investigate novel methods for efficiently optimizing parameters of appearance models for landmark localization in 3D images. We also investigate the trade-off between the number of model parameters and registration accuracy. We present results for the localization of 22 landmarks in clinical 3D CT volumes of cancer patients and optimization of part-specific patch scales. Over-fitting is likely due to an intrinsically high variability of the data and a limited labeled training and test set, here 83 scans, so we employ a rigorous bootstrap analysis to validate the results. The average mean and maximum registration error over all landmarks is reduced by 31% and 25% for the optimized model, compared to an empirically determined baseline. Additionally, we show a significantly improved performance over standard methods as the number of free parameters increases from an isotropic patch scale shared by all parts, to specific anisotropic patch scales learnt for each part in the model.

Organ segmentation is a challenging problem on which recent progress has been made by incorporation of local image statistics that model the heterogeneity of structures outside of an organ of interest. However, most of these methods rely on landmark based segmentation, which has certain drawbacks. We propose to perform organ segmentation with a novel level set algorithm that incorporates local statistics via a highly efficient point tracking mechanism. Specifically, we compile statistics on these tracked points to allow for a local intensity profile outside of the contour and to allow for a local surface area penalty, which allows us to capture fine detail where it is expected. The local intensity and curvature models are learned through landmarks automatically embedded on the surface of the training shapes. We use Parzen windows to model the internal organ intensities as one distribution since this is sufficient for most organs. In addition, since the method is based on level sets, we are able to naturally take advantage of recent work on global shape regularization. We show state-of-the-art results on the challenging problems of liver and kidney segmentation.

We are investigating techniques to develop organ specific models of normal functional Positron Emission Tomography (PET) uptake. A key issue in the development of models of functional uptake is the appropriate normalization of the PET uptake across multiple studies. It is well known that the tracer uptake in PET is subject is a range of variations including protocols and scanners, reconstruction algorithm as well as biological effects. In this study, we investigate and report results of normalization strategies for a group of organs. In particular, we evaluate the performance of reference tissue normalization strategies applied to our problem and propose a novel normalization strategy that estimates the normal uptake for a particular organ from that of a number of reference tissue regions that may be assumed to be disease-free. Results from the quantitative analysis indicate that a normalisation method using weighted contribution from a number of reference regions perform better than the best normalisation using a single reference region (liver), and show equivalent performance of both normalisation strategies for the abnormal voxel classification experiment.

Recently, the Non-Parametric (NP) Windows has been proposed to estimate the statistics of real 1D and 2D signals. NP Windows is accurate, because it is equivalent to sampling images at a high (infinite) resolution for an assumed interpolation model. This paper extends the proposed approach to consider joint distributions of image-pairs. Secondly, Green's Theorem is used to simplify the previous NP Windows algorithm. Finally, a resolution aware NP Windows algorithm is proposed, to improve robustness to relative scaling between an image-pair. Comparative testing of 2D image registration was performed using translation-only and affine transformations. Although more expensive than other methods, NP Windows frequently demonstrated superior performance for bias (distance between ground truth and global maximum) and frequency of convergence. Unlike other methods, the number of samples and histogram bin-size has little effect on NP Windows, and the prior selection of a kernel is not required.

Purpose: While is it feasible to utilize quantitative techniques such as target registration error for commissioning Deformable Image Registration (DIR), clinical assessment of DIR quality is typically visual and qualitative. We propose a quantitative approach to registration quality assessment, based on measures of deformation quality, numerical robustness, and surrogate measures of alignment accuracy. Methods: Original planning CTs were registered to re-planning CTs for 27 Head and Neck patients using a selection of registration parameters intended to produce a range of DIR results. Registrations were graded by an expert on a scale of 0–5 (from “very poor” to “perfect “) for four broad regions (Skull, Nose- Oral Cavity-Jaw, Neck, Thorax) by visually assessing alignment accuracy and deformation field quality. Grades were assigned for each region and for the overall case. For each registration, ten quantitative measures of registration quality were calculated for each voxel, without user input. These measures consisted of measures of deformation quality, numerical robustness and surrogates for alignment accuracy. Acceptable ranges for each measure were set for each feature based on physical tissue properties and from other examples of good registrations. An average voxel pass-rate was calculated for each region and for the registration overall. Results: Good correlation (r2 = 0.86) was found between the overall clinical grading and the quantitative registration assessment. Additionally, good correlations were found with the measures of deformation quality (r2 = 0.86–0.91) and robustness (r2 = 0.88). There were no examples of gross anatomical registration failures; consequently, the clinical grades were dominated by the assessment of the deformation field. Conclusion: Quantitative characterization of clinical registration may assist in clinical assessment of patient-specific registration. While anatomical alignment must still be assessed visually, this approach was able to highlight areas of concerns within the registration, and provide a quantitative assessment without user mark-up. M Gooding, LC Pickup and T Kadir are employees of Mirada Medical Ltd

A non-parametric (NP) sampling method is introduced for obtaining the joint distribution of a pair of images. This method based on NP windowing and is equivalent to sampling the images at infinite resolution. Unlike existing methods, arbitrary selection of kernels is not required and the spatial structure of images is used. NP windowing is applied to a registration application where the mutual information (MI) between a reference image and a warped template is maximised with respect to the warp parameters. In comparisons against the current state of the art MI registration methods NP windowing yielded excellent results with lower bias and improved convergence rates.

International audience; The paper gives a snapshot of the state of the art in affine covariant region detectors, and compares their performance on a set of test images under varying imaging conditions. Six types of detectors are included: detectors based on affine normalization around Harris (Mikolajczyk and Schmid, 2002; Schaffalitzky and Zisserman, 2002) and Hessian points (Mikolajczyk and Schmid, 2002), a detector of ‘maximally stable extremal regions', proposed by Matas et al. (2002); an edge-based region detector (Tuytelaars and Van Gool, 1999) and a detector based on intensity extrema (Tuytelaars and Van Gool, 2000), and a detector of ‘salient regions', proposed by Kadir, Zisserman and Brady (2004). The performance is measured against changes in viewpoint, scale, illumination, defocus and image compression. The objective of this paper is also to establish a reference test set of images and performance software, so that future detectors can be evaluated in the same framework.

Breast parenchymal density has been found to be a strong indicator for breast cancer risk, however, to date, measures of breast density are qualitative and require the judgement of the radiologist. Objective, quantitative measures of breast density are crucial tools for assessing the association between the risk of breast cancer and mammographic density as well as for quantification of density changes to the breast. Various schemes have been proposed for classifying breast density patterns, though again each requires the judgement of the clinician to assign a particular region of tissue to its class and so it is time-consuming and prone to inter- and intra-radiologist disagreement. Motivated by recent results in texture classification, we present a new approach to breast parenchymal pattern classification. The proposed scheme uses texture models to capture the mammographic appearance within the breast area: parenchymal density patterns are modelled as a statistical distribution of clustered, rotationally invariant filter responses in a low dimensional space. This robust representation can accommodate large variations in intra-class mammogram appearance and can be trained in a straight-forward manner. Key to the approach is that parenchymal patterns can occupy disconnected regions in feature space. Objective descriptors of breast density based on the digital mammogram, are developed and validated.

In this paper we describe a novel technique for detecting salient regions in an image. The detector is a generalization to affine invariance of the method introduced by Kadir and Brady [10]. The detector deems a region salient if it exhibits unpredictability in both its attributes and its spatial scale.

This paper presents a novel approach to sign language recognition that provides extremely high classification rates on minimal training data. Key to this approach is a 2 stage classification procedure where an initial classification stage extracts a high level description of hand shape and motion. This high level description is based upon sign linguistics and describes actions at a conceptual level easily understood by humans. Moreover, such a description broadly generalises temporal activities naturally overcoming variability of people and environments. A second stage of classification is then used to model the temporal transitions of individual signs using a classifier bank of Markov chains combined with Independent Component Analysis. We demonstrate classification rates as high as 97.67% for a lexicon of 43 words using only single instance training outperforming previous approaches where thousands of training examples are required.

This paper presents an overview of the Scale Saliency algorithm introduced in (Kadir and Brady, 2001). Scale Saliency is a novel method for measuring the saliency of image regions and selecting optimal scales for their analysis. The model underlying the algorithm deems image regions salient if they are simultaneously unpredictable in some feature-space and over scale. The algorithm possesses a number of attractive properties: invariance to planar rotation, scaling, intensity shifts and translation; robustness to noise, changes in viewpoint, and intensity scalings. Moreover, the approach offers a more general model of feature saliency compared with conventional ones, such as those based on kernel convolution, for example wavelet analysis, since such techniques define saliency and scale only with respect to a particular set of basis morphologies. Finally, we present a generalised version of the original algorithm which is invariant to affine transformations.

Purpose: Deformable Image Registration (DIR) is gaining wider clinical acceptance in radiation oncology. The aim of this work is to characterise a DIR algorithm on publically available 4DCT lung images, such that comparison can be performed against other algorithms. We propose an evaluation method of registration accuracy that takes into account the initial misregistration of the datasets. Methods: The “DIR Validation dataset” (http://www.creatis.insa-lyon.fr/rio/dir_validation_data) provides benchmark data for evaluating 3D CT registration algorithms. It consists of six 4DCT lung datasets (1x1x2mm resolution) with 100 landmarks identified on the end-exhalation and end-inhalation phases. Images were registered to end-inhalation using proprietary form of optical flow in commercial software (Mirada RTx, Mirada Medical, UK). Target registration error was measured before and after DIR, referred to as Initial Registration Error (IRE) and Final Registration Error (FRE). Results: The mean FRE over all landmarks was 1.37±1.81mm. FRE increased with IRE. Mean FRE of 0.86, 0.86, 1.53, 3.38, 4.45, 7.58mm was observed for IRE in the ranges 0–5, 5–10, 10–15, 15–20, 20–25, >25 mm. Higher FRE was observed at the inferior lung, where IRE was greater. Out-of-plane motion contributed more to IRE, and therefore to FRE. Maximum FRE of 20.6mm was observed for IRE of 32.1mm, located at the posterior of the middle lobe for dataset 2. Sub-voxel registration accuracy was achieved for up to 10mm IRE, and increased linearly at 0.3mm FRE/mm IRE thereafter. Conclusion: Publicly available clinical datasets enable algorithms to be compared objectively between publications. However, only reporting average TRE after registration can be misleading as the ability of an algorithm to correct for displacements varies with the IRE or position within the patient. Consequently, algorithms should be characterized using the entire range of initial displacements. For the algorithm assessed, clinically acceptable error within one voxel was achieved for IRE of up to 15mm. TK, AL, and MG are employees of Mirada Medical