Search

Motion correction of dynamic contrast enhanced magnetic resonance images (DCE-MRI) is a challenging task, due to changes in image appearance. In this study a groupwise registration, using a principle component analysis (PCA) based metric,1 is evaluated for clinical DCE MRI of the liver. The groupwise registration transforms the images to a common space, rather than to a reference volume as conventional pairwise methods do, and computes the similarity metric on all volumes simultaneously. This groupwise registration method is compared to a pairwise approach using a mutual information metric. Clinical DCE MRI of the abdomen of eight patients were included. Per patient one lesion in the liver was manually segmented in all temporal images (N=16). The registered images were compared for accuracy, spatial and temporal smoothness after transformation, and lesion volume change. Compared to a pairwise method or no registration, groupwise registration provided better alignment. In our recently started clinical study groupwise registered clinical DCE MRI of the abdomen of nine patients were scored by three radiologists. Groupwise registration increased the assessed quality of alignment. The gain in reading time for the radiologist was estimated to vary from no difference to almost a minute. A slight increase in reader confidence was also observed. Registration had no added value for images with little motion. In conclusion, the groupwise registration of DCE MR images results in better alignment than achieved by pairwise registration, which is beneficial for clinical assessment.

In images of the corneal endothelium (CE) acquired by specular microscopy, endothelial cells are commonly only visible in a part of the image due to varying contrast, mainly caused by challenging imaging conditions as a result of a strongly curved endothelium. In order to estimate the morphometric parameters of the corneal endothelium, the analyses need to be restricted to trustworthy regions - the region of interest (ROI) - where individual cells are discernible. We developed an automatic method to find the ROI by Dense U-nets, a densely connected network of convolutional layers. We tested the method on a heterogeneous dataset of 140 images, which contains a large number of blurred, noisy, and/or out of focus images, where the selection of the ROI for automatic biomarker extraction is vital. By using edge images as input, which can be estimated after retraining the same network, Dense U-net detected the trustworthy areas with an accuracy of 98.94% and an area under the ROC curve (AUC) of 0.998, without being affected by the class imbalance (9:1 in our dataset). After applying the estimated ROI to the edge images, the mean absolute percentage error (MAPE) in the estimated endothelial parameters was 0.80% for ECD, 3.60% for CV, and 2.55% for HEX.

Deformable image registration is often a slow process when using conventional methods. To speed up deformable registration, there is growing interest in using convolutional neural networks. They are comparatively fast and can be trained to estimate full-resolution deformation fields directly from pairs of images. Because deep learning-based registration methods often require rigid or affine pre-registration of the images, they do not perform true end-to-end image registration. To address this, we propose a progressive training method for end-to-end image registration with convolutional networks. The network is first trained to find large deformations at a low resolution using a smaller part of the full architecture. The network is then gradually expanded during training by adding higher resolution layers that allow the network to learn more fine-grained deformations from higher resolution data. By starting at a lower resolution, the network is able to learn larger deformations more quickly at the start of training, making pre-registration redundant. We apply this method to pulmonary CT data, and use it to register inhalation to exhalation images. We train the network using the CREATIS pulmonary CT data set, and apply the trained network to register the DIRLAB pulmonary CT data set. By computing the target registration error at corresponding landmarks we show that the error for end-to-end registration is significantly reduced by using progressive training, while retaining sub-second registration times.

Most MRI liver segmentation methods use a structural 3D scan as input, such as a T1 or T2 weighted scan. Segmentation performance may be improved by utilizing both structural and functional information, as contained in dynamic contrast enhanced (DCE) MR series. Dynamic information can be incorporated in a segmentation method based on convolutional neural networks in a number of ways. In this study, the optimal input configuration of DCE MR images for convolutional neural networks (CNNs) is studied. The performance of three different input configurations for CNNs is studied for a liver segmentation task. The three configurations are I) one phase image of the DCE-MR series as input image; II) the separate phases of the DCE-MR as input images; and III) the separate phases of the DCE-MR as channels of one input image. The three input configurations are fed into a dilated fully convolutional network and into a small U-net. The CNNs were trained using 19 annotated DCE-MR series and tested on another 19 annotated DCE-MR series. The performance of the three input configurations for both networks is evaluated against manual annotations. The results show that both neural networks perform better when the separate phases of the DCE-MR series are used as channels of an input image in comparison to one phase as input image or the separate phases as input images. No significant difference between the performances of the two network architectures was found for the separate phases as channels of an input image., Submitted to SPIE Medical Imaging 2019

Volumetric Laser Endomicroscopy (VLE) is a promising balloon-based imaging technique for detecting early neoplasia in Barretts Esophagus. Especially Computer Aided Detection (CAD) techniques show great promise compared to medical doctors, who cannot reliably find disease patterns in the noisy VLE signal. However, an essential pre-processing step for the CAD system is tissue segmentation. At present, tissue is segmented manually but is not scalable for the entire VLE scan consisting of 1,200 frames of 4,096 × 2,048 pixels. Furthermore, the current CAD methods cannot use the VLE scans to their full potential, as only a small segment of the esophagus is selected for further processing, while an automated segmentation system results in significantly more available data. This paper explores the possibility of automatically segmenting relevant tissue for VLE scans using FusionNet and a domain-specific loss function. The contribution of this work is threefold. First, we propose a tissue segmentation algorithm for VLE scans. Second, we introduce a weighted ground truth that exploits the signal-to-noise ratio characteristics of the VLE data. Third, we compare our algorithm segmentation against two additional VLE experts. The results show that our algorithm annotations are indistinguishable from the expert annotations and therefore the algorithm can be used as a preprocessing step for further classification of the tissue.

Cardiovascular disease (CVD) is a leading cause of death in the lung cancer screening population. Chest CT scans made in lung cancer screening are suitable for identification of participants at risk of CVD. Existing methods analyzing CT images from lung cancer screening for prediction of CVD events or mortality use engineered features extracted from the images combined with patient information. In this work we propose a method that automatically predicts 5-year cardiovascular mortality directly from chest CT scans without the need for hand-crafting image features. A set of 1,583 participants of the National Lung Screening Trial was included (1,188 survivors, 395 non-survivors). Low-dose chest CT images acquired at baseline were analyzed and the follow-up time was 5 years. To limit the analysis to the heart region, the heart was first localized by our previously developed algorithm for organ localization exploiting convolutional neural networks. Thereafter, a convolutional autoencoder was used to encode the identified heart region. Finally, based on the extracted encodings subjects were classified into survivors or non-survivors using a support vector machine classifier. The performance of the method was assessed in eight cross-validation experiments with 1,433 images used for training, 50 for validation and 100 for testing. The method achieved a performance with an area under the ROC curve of 0.72. The results demonstrate that prediction of cardiovascular mortality directly from low-dose screening chest CT scans, without hand-crafted features, is feasible, allowing identification of subjects at risk of fatal CVD events., This work has been submitted to SPIE 2019 conference

Current state-of-the-art deep learning segmentation methods have not yet made a broad entrance into the clinical setting in spite of high demand for such automatic methods. One important reason is the lack of reliability caused by models that fail unnoticed and often locally produce anatomically implausible results that medical experts would not make. This paper presents an automatic image segmentation method based on (Bayesian) dilated convolutional networks (DCNN) that generate segmentation masks and spatial uncertainty maps for the input image at hand. The method was trained and evaluated using segmentation of the left ventricle (LV) cavity, right ventricle (RV) endocardium and myocardium (Myo) at end-diastole (ED) and end-systole (ES) in 100 cardiac 2D MR scans from the MICCAI 2017 Challenge (ACDC). Combining segmentations and uncertainty maps and employing a human-in-the-loop setting, we provide evidence that image areas indicated as highly uncertain regarding the obtained segmentation almost entirely cover regions of incorrect segmentations. The fused information can be harnessed to increase segmentation performance. Our results reveal that we can obtain valuable spatial uncertainty maps with low computational effort using DCNNs., This work has been submitted to SPIE 2019 conference

The Couinaud classification of hepatic anatomy partitions the liver into eight functionally independent segments. Detection and segmentation of the hepatic vein (HV), portal vein (PV) and inferior vena cava (IVC) plays an important role in the subsequent delineation of the liver segments. To facilitate pharmacokinetic modeling of the liver based on the same data, a 4D DCE-MR scan protocol was selected. This yields images with high temporal resolution but low spatial resolution. Since the liver's vasculature consists of many tiny branches, segmentation of these images is challenging. The proposed framework starts with registration of the 4D DCE-MRI series followed by region growing from manually annotated seeds in the main branches of key blood vessels in the liver. It calculates the Pearson correlation between the time intensity curves (TICs) of a seed and all voxels. A maximum correlation map for each vessel is obtained by combining the correlation maps for all branches of the same vessel through a maximum selection per voxel. The maximum correlation map is incorporated in a level set scheme to individually delineate the main vessels. Subsequently, the eight liver segments are segmented based on three vertical intersecting planes fit through the three skeleton branches of HV and IVC's center of mass as well as a horizontal plane fit through the skeleton of PV. Our segmentation regarding delineation of the vessels is more accurate than the results of two state-of-the-art techniques on five subjects in terms of the average symmetric surface distance (ASSD) and modified Hausdorff distance (MHD). Furthermore, the proposed liver partitioning achieves large overlap with manual reference segmentations (expressed in Dice Coefficient) in all but a small minority of segments (mean values between 87% and 94% for segments 2-8). The lower mean overlap for segment 1 (72%) is due to the limited spatial resolution of our DCE-MR scan protocol.

Currently, non-rigid image registration algorithms are too computationally intensive to use in time-critical applications. Existing implementations that focus on speed typically address this by either parallelization on GPU-hardware, or by introducing methodically novel techniques into CPU-oriented algorithms. Stochastic gradient descent (SGD) optimization and variations thereof have proven to drastically reduce the computational burden for CPU-based image registration, but have not been successfully applied in GPU hardware due to its stochastic nature. This paper proposes 1) NiftyRegSGD, a SGD optimization for the GPU-based image registration tool NiftyReg, 2) random chunk sampler, a new random sampling strategy that better utilizes the memory bandwidth of GPU hardware. Experiments have been performed on 3D lung CT data of 19 patients, which compared NiftyRegSGD (with and without random chunk sampler) with CPU-based elastix Fast Adaptive SGD (FASGD) and NiftyReg. The registration runtime was 21.5s, 4.4s and 2.8s for elastix-FASGD, NiftyRegSGD without, and NiftyRegSGD with random chunk sampling, respectively, while similar accuracy was obtained. Our method is publicly available at https://github.com/SuperElastix/NiftyRegSGD.

Taking a multi-objective optimization approach to deformable image registration has recently gained attention, because such an approach removes the requirement of manually tuning the weights of all the involved objectives. Especially for problems that require large complex deformations, this is a non-trivial task. From the resulting Pareto set of solutions one can then much more insightfully select a registration outcome that is most suitable for the problem at hand. To serve as an internal optimization engine, currently used multi-objective algorithms are competent, but rather inefficient. In this paper we largely improve upon this by introducing a multi-objective real-valued adaptation of the recently introduced Gene-pool Optimal Mixing Evolutionary Algorithm (GOMEA) for discrete optimization. In this work, GOMEA is tailored specifically to the problem of deformable image registration to obtain substantially improved efficiency. This improvement is achieved by exploiting a key strength of GOMEA: iteratively improving small parts of solutions, allowing to faster exploit the impact of such updates on the objectives at hand through partial evaluations. We performed experiments on three registration problems. In particular, an artificial problem containing a disappearing structure, a pair of pre- and post-operative breast CT scans, and a pair of breast MRI scans acquired in prone and supine position were considered. Results show that compared to the previously used evolutionary algorithm, GOMEA obtains a speed-up of up to a factor of ∼1600 on the tested registration problems while achieving registration outcomes of similar quality.

Optical coherence tomography (OCT) yields high-resolution, three-dimensional images of the retina. A better understanding of retinal nerve fiber bundle (RNFB) trajectories in combination with visual field data may be used for future diagnosis and monitoring of glaucoma. However, manual tracing of these bundles is a tedious task. In this work, we present an automatic technique to estimate the orientation of RNFBs from volumetric OCT scans. Our method consists of several steps, starting from automatic segmentation of the RNFL. Then, a stack of en face images around the posterior nerve fiber layer interface was extracted. The image showing the best visibility of RNFB trajectories was selected for further processing. After denoising the selected en face image, a semblance structure-oriented filter was applied to probe the strength of local linear structure in a discrete set of orientations creating an orientation space. Gaussian filtering along the orientation axis in this space is used to find the dominant orientation. Next, a confidence map was created to supplement the estimated orientation. This confidence map was used as pixel weight in normalized convolution to regularize the semblance filter response after which a new orientation estimate can be obtained. Finally, after several iterations an orientation field corresponding to the strongest local orientation was obtained. The RNFB orientations of six macular scans from three subjects were estimated. For all scans, visual inspection shows a good agreement between the estimated orientation fields and the RNFB trajectories in the en face images. Additionally, a good correlation between the orientation fields of two scans of the same subject was observed. Our method was also applied to a larger field of view around the macula. Manual tracing of the RNFB trajectories shows a good agreement with the automatically obtained streamlines obtained by fiber tracking.

In Dynamic Contrast-Enhanced MRI (DCE-MRI) of the liver, a series of images is acquired over a period of 20 minutes. Due to the patient's breathing, the liver is subject to a substantial displacement between acquisitions. Furthermore, due to its location in the abdomen, the liver also undergoes marked deformation. The large deformations combined with variation in image contrast make accurate liver registration challenging. We present a registration framework that incorporates a liver segmentation to improve the registration accuracy. The segmented liver serves as region-of-interest to our in-house developed registration method called ALOST (autocorrelation of local image structure). ALOST is a continuous optimization method that uses local phase features to overcome space-variant intensity distortions. The proposed framework can confine the solution field to the liver and allow for ALOST to obtain a more accurate solution. For the segmentation part, we use a level-set method to delineate the liver in a so-called contrast enhancement map. This map is obtained by computing the difference between the last and registered first volume from the DCE series. Subsequently, we slightly dilate the segmentation, and apply it as the mask to the other DCE-MRI volumes during registration. It is shown that the registration result becomes more accurate compared with the original ALOST approach.

Quantification of the spin-lattice relaxation time, T1, of tissues is important for characterization of tissues in clinical magnetic resonance imaging (MRI). In T1 mapping, T1 values are estimated from a set of T1-weighted MRI images. Due to the limited spatial resolution of the T1-weighted images, one voxel might consist of two tissues, causing partial volume effects (PVE). In conventional mono-exponential T1 estimation, these PVE result in systematic errors in the T1 map. To account for PVE, single-voxel bi-exponential estimators have been suggested. Unfortunately, in general, they suffer from low accuracy and precision. In this work, we propose a joint multi-voxel bi-exponential T1 estimator (JMBE) and compare its performance to a single-voxel bi-exponential T1 estimator (SBE). Results show that, in contrast to the SBE, and for clinically achievable single-voxel SNRs, the JMBE is accurate and efficient if four or more neighboring voxels are used in the joint estimation framework. This illustrates that, for clinically realistic SNRs, accurate results for quantitative biexponential T1 estimation are only achievable if information of neighboring voxels is incorporated.

Optical coherence tomography (OCT) is used to produce high-resolution three-dimensional images of the retina, which permit the investigation of retinal irregularities. In dry age-related macular degeneration (AMD), a chronic eye disease that causes central vision loss, disruptions such as drusen and changes in retinal layer thicknesses occur which could be used as biomarkers for disease monitoring and diagnosis. Due to the topology disrupting pathology, existing segmentation methods often fail. Here, we present a solution for the segmentation of retinal layers in dry AMD subjects by extending our previously presented loosely coupled level sets framework which operates on attenuation coefficients. In eyes affected by AMD, Bruch’s membrane becomes visible only below the drusen and our segmentation framework is adapted to delineate such a partially discernible interface. Furthermore, the initialization stage, which tentatively segments five interfaces, is modified to accommodate the appearance of drusen. This stage is based on Dijkstra's algorithm and combines prior knowledge on the shape of the interface, gradient and attenuation coefficient in the newly proposed cost function. This prior knowledge is incorporated by varying the weights for horizontal, diagonal and vertical edges. Finally, quantitative evaluation of the accuracy shows a good agreement between manual and automated segmentation.

International audience

International audience; The development of 3D ultrasonic probes and 3D ultrasound imaging offersnew functionalities that call for specific image processingdevelopments. In this paper, we propose an original method for thesegmentationof the utero-fetal unit (UFU) from 3D ultrasound (3DUS) volumes, acquired during the first trimester of gestation.UFU segmentation is required for a number of tasks, such as precise organ delineation, 3D modeling, quantitative measurements and evaluation of the clinical impact of 3D imaging.The segmentation problem is formulated as the optimization of a partition of theimage into two classes of tissues: the amniotic fluid and the fetaltissues. A Bayesian formulation of the partition problem integrates statistical models of theintensity distributions in each tissue class as homogeneity constraints and regularity constraints on the contours.An energy functional is minimized using a level set implementation of a deformable model to identify the optimal partition. We propose to combine Rayleigh, Normal, Exponential and Gamma distribution models to compute the region homogeneity constraints.We tested the segmentation method on a database of 19 ante-natal 3DUS images. Promising results wereobtained, showing the flexibility of the level set formulation and the interest of learning the most appropriatestatistical models according to the idiosyncrasies of the data and the tissues. The segmentation method was shown to be robust to different types of initialization and to provide accurate results, with an average overlap measure of 0.89 when comparing with manual segmentations.

Multimodal groupwise registration has been of growing interest to the image processing community due to developments in scanner technologies (e.g. multiparametric MRI, DCE-CT or PET-MR) that increased both the number of modalities and number of images under consideration. In this work a novel methodology is presented for multimodal groupwise registration that is based on Laplacian eigenmaps, a nonlinear dimensionality reduction technique. Compared to recently proposed dissimilarity metrics based on principal component analysis, the proposed metric should enable a better capture of the intensity relationships between different images in the group. The metric is constructed to be the second smallest eigenvalue from the eigenvector problem defined in Laplacian eigenmaps. The method was validated in three distinct experiments: a non-linear synthetic registration experiment, the registration of quantitative MRI data of the carotid artery, and the registration of multimodal data of the brain (RIRE). The results show increased accuracy and robustness compared to other state-of-the-art groupwise registration methodologies.

Purpose: To develop, test, evaluate and apply a novel tool for the white matter fiber-based analysis of T1w/T2w ratio maps quantifying myelin content. Background: The cerebral white matter in the human brain develops from a mostly non-myelinated state to a nearly fully mature white matter myelination within the first few years of life. High resolution T1w/T2w ratio maps are believed to be effective in quantitatively estimating myelin content on a voxel-wise basis. We propose the use of a fiber-tract-based analysis of such T1w/T2w ratio data, as it allows us to separate fiber bundles that a common regional analysis imprecisely groups together, and to associate effects to specific tracts rather than large, broad regions. Methods: We developed an intuitive, open source tool to facilitate such fiber-based studies of T1w/T2w ratio maps. Via its Graphical User Interface (GUI) the tool is accessible to non-technical users. The framework uses calibrated T1w/T2w ratio maps and a prior fiber atlas as an input to generate profiles of T1w/T2w values. The resulting fiber profiles are used in a statistical analysis that performs along-tract functional statistical analysis. We applied this approach to a preliminary study of early brain development in neonates. Results: We developed an open-source tool for the fiber based analysis of T1w/T2w ratio maps and tested it in a study of brain development.

Diffusion-weighted MR imaging (DWI) is a powerful tool to study brain tissue microstructure. DWI is sensitive to subtle changes in the white matter (WM), and can provide insight into abnormal brain changes in diseases such as Alzheimer’s disease (AD). In this study, we used 7-Tesla hybrid diffusion imaging (HYDI) to scan 3 transgenic rats (line TgF344-AD; that model the full clinico-pathological spectrum of the human disease) ex vivo at 10, 15 and 24 months. We acquired 300 DWI volumes across 5 q-sampling shells (b=1000, 3000, 4000, 8000, 12000 s/mm^2). From the top three b-value shells with highest signal-to-noise ratios, we reconstructed markers of WM disease, including indices of axon density and diameter in the corpus callosum (CC) – directly quantifying processes that occur in AD. As expected, apparent anisotropy progressively decreased with age; there were also decreases in the intra- and extra-axonal MR signal along axons. Axonal diameters were larger in segments of the CC (splenium and body, but not genu), possibly indicating neuritic dystrophy – characterized by enlarged axons and dendrites as previously observed at the ultrastructural level (see Cohen et al., J. Neurosci. 2013). This was further supported by increases in MR signals trapped in glial cells, CSF and possibly other small compartments in WM structures. Finally, tractography detected fewer fibers in the CC at 10 versus 24 months of age. These novel findings offer great potential to provide technical and scientific insight into the biology of brain disease.

In radiotherapy treatment planning that is only based on magnetic resonance imaging (MRI), the electron density information usually obtained from computed tomography (CT) must be derived from the MRI by synthesizing a so-called pseudo CT (pCT). This is a non-trivial task since MRI intensities are neither uniquely nor quantitatively related to electron density. Typical approaches involve either a classification or regression model requiring specialized MRI sequences to solve intensity ambiguities, or an atlas-based model necessitating multiple registrations between atlases and subject scans. In this work, we explore a machine learning approach for creating a pCT of the pelvic region from conventional MRI sequences without using atlases. We use a random forest provided with information about local texture, edges and spatial features derived from the MRI. This helps to solve intensity ambiguities. Furthermore, we use the concept of auto-context by sequentially training a number of classification forests to create and improve context features, which are finally used to train a regression forest for pCT prediction. We evaluate the pCT quality in terms of the voxel-wise error and the radiologic accuracy as measured by water-equivalent path lengths. We compare the performance of our method against two baseline pCT strategies, which either set all MRI voxels in the subject equal to the CT value of water, or in addition transfer the bone volume from the real CT. We show an improved performance compared to both baseline pCTs suggesting that our method may be useful for MRI-only radiotherapy.