Search

Breast conserving surgery followed by radiotherapy is the standard of care for early-stage breast cancer patients. Deformable image registration (DIR) can in principle be of great value for accurate localization of the original tumor site to optimize breast irradiation after surgery. However, current state-of-the-art DIR methods are not very successful when tissue is present in one image but not in the other (i.e., in case of content mismatch). To tackle this challenge, we combined a multi-objective DIR approach with simulated tissue removal. Parameters defining the area to be removed as well as key DIR parameters (that are often tuned manually for each DIR case) are determined by a multi-objective optimization process. In multi-objective optimization, not one, but a set of solutions is found, that represent high-quality trade-offs between objectives of interest. We used three state-of-the-art multi-objective evolutionary algorithms as meta-optimizers to search for the optimal parameters, and tested our approach on four test cases of computed tomography (CT) images of breast cancer patients before and after surgery. Results show that using meta-optimization with simulated tissue removal improves the performance of DIR. This way, sets of high-quality solutions could be obtained with a mean target registration error of 2.4 mm over four test cases and an estimated excised volume that is within 20% from the measured volume of the surgical resection specimen.

Discomfort detection for infants is essential in the healthcare domain, since infants lack the ability to verbalize their pain and discomfort. In this paper, we propose a robust and generic discomfort detection for infants by exploiting a novel and efficient initialization method for facial landmark localization, using an unsupervised rollangle estimation. The roll-angle estimation is achieved by fitting a 1st-order B-spline model to facial features obtained from the scaled-normalized Laplacian of the Gaussian operator. The proposed method can be adopted both for daylight and infrared-light images and supports real-time implementation. Experimental results have shown that the proposed method improves the performance of discomfort detection by 6.0% and 4.2% for the AUC and AP using daylight images, together with 6.9% and 3.8% for infrared-light images, respectively.

In images of the corneal endothelium (CE) acquired by specular microscopy, endothelial cells are commonly only visible in a part of the image due to varying contrast, mainly caused by challenging imaging conditions as a result of a strongly curved endothelium. In order to estimate the morphometric parameters of the corneal endothelium, the analyses need to be restricted to trustworthy regions - the region of interest (ROI) - where individual cells are discernible. We developed an automatic method to find the ROI by Dense U-nets, a densely connected network of convolutional layers. We tested the method on a heterogeneous dataset of 140 images, which contains a large number of blurred, noisy, and/or out of focus images, where the selection of the ROI for automatic biomarker extraction is vital. By using edge images as input, which can be estimated after retraining the same network, Dense U-net detected the trustworthy areas with an accuracy of 98.94% and an area under the ROC curve (AUC) of 0.998, without being affected by the class imbalance (9:1 in our dataset). After applying the estimated ROI to the edge images, the mean absolute percentage error (MAPE) in the estimated endothelial parameters was 0.80% for ECD, 3.60% for CV, and 2.55% for HEX.

Segmentation and identification of the vertebrae in CT images are important steps for automatic analysis of the spine. This paper presents an automatic method based on iterative convolutional neural networks. These utilize the inherent order of the vertebral column to simplify the detection problem, so that the network can be trained with as little as ten manual reference segmentations. Vertebrae are segmented and identified one- by-one in sequential order, using an iterative procedure. Vertebrae are first roughly localized and identified in low-resolution images that enable the analysis of context information, and afterwards reanalyzed in the original high-resolution images to obtain a fine segmentation. The method was trained and evaluated with 15 spine CT scans from the MICCAI CSI 2014 workshop challenge. These scans cover the whole thoracic and lumbar part of the spine of healthy young adults. In contrast to a non-iterative convolutional neural network, which made labeling mistakes, the proposed iterative method correctly identified all vertebrae. Our method achieved a mean Dice coefficient of 0.948 and a mean surface distance of 0.29 mm and thus outperforms the best method that participated in the original challenge.

Quantification of the spin-lattice relaxation time, T1, of tissues is important for characterization of tissues in clinical magnetic resonance imaging (MRI). In T1 mapping, T1 values are estimated from a set of T1-weighted MRI images. Due to the limited spatial resolution of the T1-weighted images, one voxel might consist of two tissues, causing partial volume effects (PVE). In conventional mono-exponential T1 estimation, these PVE result in systematic errors in the T1 map. To account for PVE, single-voxel bi-exponential estimators have been suggested. Unfortunately, in general, they suffer from low accuracy and precision. In this work, we propose a joint multi-voxel bi-exponential T1 estimator (JMBE) and compare its performance to a single-voxel bi-exponential T1 estimator (SBE). Results show that, in contrast to the SBE, and for clinically achievable single-voxel SNRs, the JMBE is accurate and efficient if four or more neighboring voxels are used in the joint estimation framework. This illustrates that, for clinically realistic SNRs, accurate results for quantitative biexponential T1 estimation are only achievable if information of neighboring voxels is incorporated.

CT attenuation correction (CTAC) images acquired with PET/CT visualize coronary artery calcium (CAC) and enable CAC quantification. CAC scores acquired with CTAC have been suggested as a marker of cardiovascular disease (CVD). In this work, an algorithm previously developed for automatic CAC scoring in dedicated cardiac CT was applied to automatic CAC detection in CTAC. The study included 134 consecutive patients undergoing 82-Rb PET/CT. Low-dose rest CTAC scans were acquired (100 kV, 11 mAs, 1.4mm×1.4mm×3mm voxel size). An experienced observer defined the reference standard with the clinically used intensity level threshold for calcium identification (130 HU). Five scans were removed from analysis due to artifacts. The algorithm extracted potential CAC by intensity-based thresholding and 3D connected component labeling. Each candidate was described by location, size, shape and intensity features. An ensemble of extremely randomized decision trees was used to identify CAC. The data set was randomly divided into training and test sets. Automatically identified CAC was quantified using volume and Agatston scores. In 33 test scans, the system detected on average 469mm3/730mm3 (64%) of CAC with 36mm3 false positive volume per scan. The intraclass correlation coefficient for volume scores was 0.84. Each patient was assigned to one of four CVD risk categories based on the Agatston score (0-10, 11-100, 101-400

Optical coherence tomography (OCT) is used to produce high-resolution three-dimensional images of the retina, which permit the investigation of retinal irregularities. In dry age-related macular degeneration (AMD), a chronic eye disease that causes central vision loss, disruptions such as drusen and changes in retinal layer thicknesses occur which could be used as biomarkers for disease monitoring and diagnosis. Due to the topology disrupting pathology, existing segmentation methods often fail. Here, we present a solution for the segmentation of retinal layers in dry AMD subjects by extending our previously presented loosely coupled level sets framework which operates on attenuation coefficients. In eyes affected by AMD, Bruch’s membrane becomes visible only below the drusen and our segmentation framework is adapted to delineate such a partially discernible interface. Furthermore, the initialization stage, which tentatively segments five interfaces, is modified to accommodate the appearance of drusen. This stage is based on Dijkstra's algorithm and combines prior knowledge on the shape of the interface, gradient and attenuation coefficient in the newly proposed cost function. This prior knowledge is incorporated by varying the weights for horizontal, diagonal and vertical edges. Finally, quantitative evaluation of the accuracy shows a good agreement between manual and automated segmentation.

Real-time three-dimensional echocardiography (RT3DE) offers an efficient way to obtain complete 3D images of the heart over an entire cardiac cycle in just a few seconds. The complex 3D wall motion and temporal information contained in these 4D data sequences has the potential to enhance and supplement other imaging modalities for clinical diagnoses based on cardiac motion analysis. In our previous work, a 4D optical flow based method was developed to estimate dynamic cardiac metrics, including strains and displacements, from 4D ultrasound. In this study, in order to evaluate the ability of our method in detecting ischemic regions, coronary artery occlusion experiments at various locations were performed on five dogs. 4D ultrasound data acquired during these experiments were analyzed with our proposed method. Ischemic regions predicted by the outcome of strain measurements were compared to predictions from cardiac physiology with strong agreement. This is the first direct validation study of our image analysis method for biomechanical prediction and in vivo experimental outcome.

With relatively high frame rates and the ability to acquire volume data sets with a stationary transducer, 3D ultrasound systems, based on matrix phased array transducers, provide valuable three-dimensional information, from which quantitative measures of cardiac function can be extracted. Such analyses require segmentation and visual tracking of the left ventricular endocardial border. Due to the large size of the volumetric data sets, manual tracing of the endocardial border is tedious and impractical for clinical applications. Therefore the development of automatic methods for tracking three-dimensional endocardial motion is essential. In this study, we evaluate a four-dimensional optical flow motion tracking algorithm to determine its capability to follow the endocardial border in three dimensional ultrasound data through time. The four-dimensional optical flow method was implemented using three-dimensional correlation. We tested the algorithm on an experimental open-chest dog data set and a clinical data set acquired with a Philips' iE33 three-dimensional ultrasound machine. Initialized with left ventricular endocardial data points obtained from manual tracing at end-diastole, the algorithm automatically tracked these points frame by frame through the whole cardiac cycle. Finite element surfaces were fitted through the data points obtained by both optical flow tracking and manual tracing by an experienced observer for quantitative comparison of the results. Parameterization of the finite element surfaces was performed and maps displaying relative differences between the manual and semi-automatic methods were compared. The results showed good consistency with less than 10% difference between manual tracing and optical flow estimation on 73% of the entire surface. In addition, the optical flow motion tracking algorithm greatly reduced processing time (about 94% reduction compared to human involvement per cardiac cycle) for analyzing cardiac function in three-dimensional ultrasound data sets. A displacement field was computed from the optical flow output, and a framework for computation of dynamic cardiac information is introduced. The method was applied to a clinical data set from a heart transplant patient and dynamic measurements agreed with known physiology as well as experimental results.

We propose a Hybrid Segmentation Engine that consists of component modules, for automated segmentation of radiological patient and the Visible Human data. We integrate boundary-based and region-based segmentation methods to exploit the strength of each method hopefully to cover the weakness of the other method.

This paper presents new results for segmentation of 3D ultrasound using a robust and smooth segmentation method based on a homogeneity-driven three dimensional level set algorithm. The segmentation was applied to echocardiographic data from healthy volunteers acquired with two 3D ultrasound machines based on matrix-phased array technology. A comparison of ventricular volumes and geometry is performed based on manual and automatic methods of segmentation. Results showed good agreement of the segmentation methods and the ultrasound machines for ventricular geometry and quantitative assessment of cardiac function.

This project is part of the Northeast Structural Genomics Consortium (NESG). The goal of this consortium is to develop efficient and integrated technologies for high-throughput (HTP) protein production and 3D structure determination. This project focuses on the design of an image analysis system to classify protein crystal structures in a production oriented environment. The method presented performs classification of microscopic images as protein precipitates or crystals versus clear solutions. Preliminary experiments are presented, which provided high classification accuracy on large datasets of protein crystallization experiments using expert classification for ground truth.

This project is part of the Northeast Structural Genomics Consortium (NESG). The goal of this consortium is to develop efficient and integrated technologies for high-throughput (HTP) protein production and 3D structure determination. This project focuses on the design of an image analysis system to classify protein crystal structures in a production oriented environment. The method performs classification of microscopic images as clear droplets versus non-clear droplets (precipitates and crystals). Using expert classification for ground truth, current results show high classification accuracy with a large image datasets.

Speckle noise corrupts ultrasonic data by introducing sharp changes in an echocardiographic image intensity profile, while attenuation alters the intensity of equally significant cardiac structures. These properties introduce inhomogeneity in the spatial domain and suggests that measures based on phase information rather than intensity are more appropriate for denoising and cardiac border detection. The present analysis method relies on the expansion of temporal ultrasonic volume data on complex exponential wavelet-like basis functions called Brushlets. These basis functions decompose a signal into distinct patterns of oriented textures. Projected coefficients are associated with distinct 'brush strokes' of a particular size and orientation. 4D overcomplete brushlet analysis is applied to temporal echocardiographic values. We show that adding the time dimension in the analysis dramatically improves the quality and robustness of the method without adding complexity in the design of a segmentation tool. We have investigated mathematical and empirical methods for identifying the most 'efficient' brush stroke sizes and orientations for decomposition and reconstruction on both phantom and clinical data. In order to determine the 'best tiling' or equivalently, the 'best brushlet basis', we use an entropy-based information cost metric function. Quantitative validation and clinical applications of this new spatio-temporal analysis tool are reported for balloon phantoms and clinical data sets.

Multimodal groupwise registration has been of growing interest to the image processing community due to developments in scanner technologies (e.g. multiparametric MRI, DCE-CT or PET-MR) that increased both the number of modalities and number of images under consideration. In this work a novel methodology is presented for multimodal groupwise registration that is based on Laplacian eigenmaps, a nonlinear dimensionality reduction technique. Compared to recently proposed dissimilarity metrics based on principal component analysis, the proposed metric should enable a better capture of the intensity relationships between different images in the group. The metric is constructed to be the second smallest eigenvalue from the eigenvector problem defined in Laplacian eigenmaps. The method was validated in three distinct experiments: a non-linear synthetic registration experiment, the registration of quantitative MRI data of the carotid artery, and the registration of multimodal data of the brain (RIRE). The results show increased accuracy and robustness compared to other state-of-the-art groupwise registration methodologies.

Dynamic contrast-enhanced computed tomography (DCE-CT) is an emerging radiological technique, which consists in acquiring a rapid sequence of CT images, shortly after the injection of an intravenous contrast agent. The passage of the contrast agent in a tissue results in a varying CT intensity over time, recorded in time-attenuation curves (TACs), which can be related to the contrast supplied to that tissue via the supplying artery to estimate the local perfusion and permeability characteristics. The time delay between the arrival of the contrast bolus in the feeding artery and the tissue of interest, called the bolus arrival time (BAT), needs to be determined accurately to enable reliable perfusion analysis. Its automated identification is however highly sensitive to noise. We propose an accurate and efficient method for estimating the BAT from DCE-CT images. The method relies on a piecewise linear TAC model with four segments and suitable parameter constraints for limiting the range of possible values. The model is fitted to the acquired TACs in a multiresolution fashion using an iterative optimization approach. The performance of the method was evaluated on simulated and real perfusion data of lung and rectum tumours. In both cases, the method was found to be stable, leading to average accuracies in the order of the temporal resolution of the dynamic sequence. For reasonable levels of noise, the results were found to be comparable to those obtained using a previously proposed method, employing a full search algorithm, but requiring an order of magnitude more computation time.

Purpose: To develop, test, evaluate and apply a novel tool for the white matter fiber-based analysis of T1w/T2w ratio maps quantifying myelin content. Background: The cerebral white matter in the human brain develops from a mostly non-myelinated state to a nearly fully mature white matter myelination within the first few years of life. High resolution T1w/T2w ratio maps are believed to be effective in quantitatively estimating myelin content on a voxel-wise basis. We propose the use of a fiber-tract-based analysis of such T1w/T2w ratio data, as it allows us to separate fiber bundles that a common regional analysis imprecisely groups together, and to associate effects to specific tracts rather than large, broad regions. Methods: We developed an intuitive, open source tool to facilitate such fiber-based studies of T1w/T2w ratio maps. Via its Graphical User Interface (GUI) the tool is accessible to non-technical users. The framework uses calibrated T1w/T2w ratio maps and a prior fiber atlas as an input to generate profiles of T1w/T2w values. The resulting fiber profiles are used in a statistical analysis that performs along-tract functional statistical analysis. We applied this approach to a preliminary study of early brain development in neonates. Results: We developed an open-source tool for the fiber based analysis of T1w/T2w ratio maps and tested it in a study of brain development.

Whole-body diffusion-weighted (WB-DW) MRI in combination with anatomical MRI has shown a great poten- tial in bone and soft tissue tumour detection, evaluation of lymph nodes and treatment response assessment. Because of the vast body coverage, whole-body MRI is acquired in separate stations, which are subsequently combined into a whole-body image. However, inter-station and inter-modality image misalignments can occur due to image distortions and patient motion during acquisition, which may lead to inaccurate representations of patient anatomy and hinder visual assessment. Automated and accurate whole-body image formation and alignment of the multi-modal MRI images is therefore crucial. We investigated several registration approaches for the formation or stitching of the whole-body image stations, followed by a deformable alignment of the multi- modal whole-body images. We compared a pairwise approach, where diffusion-weighted (DW) image stations were sequentially aligned to a reference station (pelvis), to a groupwise approach, where all stations were simultaneously mapped to a common reference space while minimizing the overall transformation. For each, a choice of input images and corresponding metrics was investigated. Performance was evaluated by assessing the quality of the obtained whole-body images, and by verifying the accuracy of the alignment with whole-body anatomical sequences. The groupwise registration approach provided the best compromise between the formation of WB- DW images and multi-modal alignment. The fully automated method was found to be robust, making its use in the clinic feasible.

In radiotherapy treatment planning that is only based on magnetic resonance imaging (MRI), the electron density information usually obtained from computed tomography (CT) must be derived from the MRI by synthesizing a so-called pseudo CT (pCT). This is a non-trivial task since MRI intensities are neither uniquely nor quantitatively related to electron density. Typical approaches involve either a classification or regression model requiring specialized MRI sequences to solve intensity ambiguities, or an atlas-based model necessitating multiple registrations between atlases and subject scans. In this work, we explore a machine learning approach for creating a pCT of the pelvic region from conventional MRI sequences without using atlases. We use a random forest provided with information about local texture, edges and spatial features derived from the MRI. This helps to solve intensity ambiguities. Furthermore, we use the concept of auto-context by sequentially training a number of classification forests to create and improve context features, which are finally used to train a regression forest for pCT prediction. We evaluate the pCT quality in terms of the voxel-wise error and the radiologic accuracy as measured by water-equivalent path lengths. We compare the performance of our method against two baseline pCT strategies, which either set all MRI voxels in the subject equal to the CT value of water, or in addition transfer the bone volume from the real CT. We show an improved performance compared to both baseline pCTs suggesting that our method may be useful for MRI-only radiotherapy.