Search

Fully convolutional neural networks (FCNs), and in particular U-Nets, have achieved state-of-the-art results in semantic segmentation for numerous medical imaging applications. Moreover, batch normalization and Dice loss have been used successfully to stabilize and accelerate training. However, these networks are poorly calibrated i.e. they tend to produce overconfident predictions both in correct and erroneous classifications, making them unreliable and hard to interpret. In this paper, we study predictive uncertainty estimation in FCNs for medical image segmentation. We make the following contributions: 1) We systematically compare cross entropy loss with Dice loss in terms of segmentation quality and uncertainty estimation of FCNs; 2) We propose model ensembling for confidence calibration of the FCNs trained with batch normalization and Dice loss; 3) We assess the ability of calibrated FCNs to predict segmentation quality of structures and detect out-of-distribution test examples. We conduct extensive experiments across three medical image segmentation applications of the brain, the heart, and the prostate to evaluate our contributions. The results of this study offer considerable insight into the predictive uncertainty estimation and out-of-distribution detection in medical image segmentation and provide practical recipes for confidence calibration. Moreover, we consistently demonstrate that model ensembling improves confidence calibration., Journal of IEEE Transactions on Medical Imaging

PURPOSE We present SlicerDMRI, an open-source software suite that enables research using diffusion magnetic resonance imaging (dMRI), the only modality that can map the white matter connections of the living human brain. SlicerDMRI enables analysis and visualization of dMRI data and is aimed at the needs of clinical research users. SlicerDMRI is built upon and deeply integrated with 3D Slicer, a National Institutes of Health–supported open-source platform for medical image informatics, image processing, and three-dimensional visualization. Integration with 3D Slicer provides many features of interest to cancer researchers, such as real-time integration with neuronavigation equipment, intraoperative imaging modalities, and multimodal data fusion. One key application of SlicerDMRI is in neurosurgery research, where brain mapping using dMRI can provide patient-specific maps of critical brain connections as well as insight into the tissue microstructure that surrounds brain tumors. PATIENTS AND METHODS In this article, we focus on a demonstration of SlicerDMRI as an informatics tool to enable end-to-end dMRI analyses in two retrospective imaging data sets from patients with high-grade glioma. Analyses demonstrated here include conventional diffusion tensor analysis, advanced multifiber tractography, automated identification of critical fiber tracts, and integration of multimodal imagery with dMRI. RESULTS We illustrate the ability of SlicerDMRI to perform both conventional and advanced dMRI analyses as well as to enable multimodal image analysis and visualization. We provide an overview of the clinical rationale for each analysis along with pointers to the SlicerDMRI tools used in each. CONCLUSION SlicerDMRI provides open-source and clinician-accessible research software tools for dMRI analysis. SlicerDMRI is available for easy automated installation through the 3D Slicer Extension Manager.

Background Brain metastases are manually identified during stereotactic radiosurgery (SRS) treatment planning, which is time consuming and potentially challenging. Purpose To develop and investigate deep learning (DL) methods for detecting brain metastasis with MRI to aid in treatment planning for SRS. Materials and Methods In this retrospective study, contrast material-enhanced three-dimensional T1-weighted gradient-echo MRI scans from patients who underwent gamma knife SRS from January 2011 to August 2018 were analyzed. Brain metastases were manually identified and contoured by neuroradiologists and treating radiation oncologists. DL single-shot detector (SSD) algorithms were constructed and trained to map axial MRI slices to a set of bounding box predictions encompassing metastases and associated detection confidences. Performances of different DL SSDs were compared for per-lesion metastasis-based detection sensitivity and positive predictive value (PPV) at a 50% confidence threshold. For the highest-performing model, detection performance was analyzed by using free-response receiver operating characteristic analysis. Results Two hundred sixty-six patients (mean age, 60 years ± 14 [standard deviation]; 148 women) were randomly split into 80% training and 20% testing groups (212 and 54 patients, respectively). For the testing group, sensitivity of the highest-performing (baseline) SSD was 81% (95% confidence interval [CI]: 80%, 82%; 190 of 234) and PPV was 36% (95% CI: 35%, 37%; 190 of 530). For metastases measuring at least 6 mm, sensitivity was 98% (95% CI: 97%, 99%; 130 of 132) and PPV was 36% (95% CI: 35%, 37%; 130 of 366). Other models (SSD with a ResNet50 backbone, SSD with focal loss, and RetinaNet) yielded lower sensitivities of 73% (95% CI: 72%, 74%; 171 of 234), 77% (95% CI: 76%, 78%; 180 of 234), and 79% (95% CI: 77%, 81%; 184 of 234), respectively, and lower PPVs of 29% (95% CI: 28%, 30%; 171 of 581), 26% (95% CI: 26%, 26%; 180 of 681), and 13% (95% CI: 12%, 14%; 184 of 1412). Conclusion Deep-learning single-shot detector models detected nearly all brain metastases that were 6 mm or larger with limited false-positive findings using postcontrast T1-weighted MRI. © RSNA, 2020 See also the editorial by Kikinis and Wells in this issue.

In this work, we propose a theoretical framework based on maximum profile likelihood for pairwise and groupwise registration. By an asymptotic analysis, we demonstrate that maximum profile likelihood registration minimizes an upper bound on the joint entropy of the distribution that generates the joint image data. Further, we derive the congealing method for groupwise registration by optimizing the profile likelihood in closed form, and using coordinate ascent, or iterative model refinement. We also describe a method for feature based registration in the same framework and demonstrate it on groupwise tractographic registration. In the second part of the article, we propose an approach to deep metric registration that implements maximum likelihood registration using deep discriminative classifiers. We show further that this approach can be used for maximum profile likelihood registration to discharge the need for well-registered training data, using iterative model refinement. We demonstrate that the method succeeds on a challenging registration problem where the standard mutual information approach does not perform well.

This paper presents a method for automatically calibrating and assessing the calibration quality of an externally tracked 2-D ultrasound (US) probe by scanning arbitrary, natural tissues, as opposed a specialized calibration phantom as is the typical practice. A generative topic model quantifies the posterior probability of calibration parameters conditioned on local 2-D image features arising from a generic underlying substrate. Auto-calibration is achieved by identifying the maximum a-posteriori image-to-probe transform, and calibration quality is assessed online in terms of the posterior probability of the current image-to-probe transform. Both are closely linked to the 3-D point reconstruction error (PRE) in aligning feature observations arising from the same underlying physical structure in different US images. The method is of practical importance in that it operates simply by scanning arbitrary textured echogenic structures, e.g., in-vivo tissues in the context of the US-guided procedures, without requiring specialized calibration procedures or equipment. Observed data take the form of local scale-invariant features that can be extracted and fit to the model in near real-time. Experiments demonstrate the method on a public data set of in vivo human brain scans of 14 unique subjects acquired in the context of neurosurgery. Online calibration assessment can be performed at approximately 3 Hz for the US images of $640\times 480$ pixels. Auto-calibration achieves an internal mean PRE of 1.2 mm and a discrepancy of [2 mm, 6 mm] in comparison to the calibration via a standard phantom-based method.

We propose and demonstrate a novel machine learning algorithm that assesses pulmonary edema severity from chest radiographs. While large publicly available datasets of chest radiographs and free-text radiology reports exist, only limited numerical edema severity labels can be extracted from radiology reports. This is a significant challenge in learning such models for image classification. To take advantage of the rich information present in the radiology reports, we develop a neural network model that is trained on both images and free-text to assess pulmonary edema severity from chest radiographs at inference time. Our experimental results suggest that the joint image-text representation learning improves the performance of pulmonary edema assessment compared to a supervised model trained on images only. We also show the use of the text for explaining the image classification by the joint model. To the best of our knowledge, our approach is the first to leverage free-text radiology reports for improving the image model performance in this application. Our code is available at: https://github.com/RayRuizhiLiao/joint_chestxray., NIH/NIBIB/NAC (Grant P41EB015902)

Neuroimaging studies typically adopt a common feature space for all data, which may obscure aspects of neuroanatomy only observable in subsets of a population, e.g. cortical folding patterns unique to individuals or shared by close relatives. Here, we propose to model individual variability using a distinctive keypoint signature: a set of unique, localized patterns, detected automatically in each image by a generic saliency operator. The similarity of an image pair is then quantified by the proportion of keypoints they share using a novel Jaccard-like measure of set overlap. Experiments demonstrate the keypoint method to be highly efficient and accurate, using a set of 7536 T1-weighted MRIs pooled from four public neuroimaging repositories, including twins, non-twin siblings, and 3334 unique subjects. All same-subject image pairs are identified by a similarity threshold despite confounds including aging and neurodegenerative disease progression. Outliers reveal previously unknown data labeling inconsistencies, demonstrating the usefulness of the keypoint signature as a computational tool for curating large neuroimage datasets.

Statistical shape analysis via diffeomorphic image registration is challenging due to the high-dimensional and nonlinear nature of the space of diffeomorphisms. In this chapter, we first introduce a low-dimensional representation of initial velocity fields (i.e., the tangent space of diffeomorphisms) in a bandlimited frequency domain, which leads to an efficient statistical analysis of anatomical shape changes characterized by diffeomorphic transformations. We then present a latent variable model for probabilistic principal geodesic analysis (PPGA) that quantifies shape variability using this new representation. We run our model on 3D brain MRI scans and demonstrate its efficiency in capturing the data variability across large populations. Experimental results show that our model estimates a more compact representation of group variation at a substantially lower computational cost than state-of-the-art methods, such as tangent space principal component analysis (TPCA) and PPGA using a vector field representation in the original high-dimensional image grid.

The loss function of an unsupervised multimodal image registration framework has two terms, i.e., a metric for similarity measure and regularization. In the deep learning era, researchers proposed many approaches to automatically learn the similarity metric, which has been shown effective in improving registration performance. However, for the regularization term, most existing multimodal registration approaches still use a hand-crafted formula to impose artificial properties on the estimated deformation field. In this work, we propose a unimodal cyclic regularization training pipeline, which learns task-specific prior knowledge from simpler unimodal registration, to constrain the deformation field of multimodal registration. In the experiment of abdominal CT-MR registration, the proposed method yields better results over conventional regularization methods, especially for severely deformed local regions., Comment: 5 pages, 5 figures

This paper presents an efficient approach to quantifying image registration uncertainty based on a low-dimensional representation of geometric deformations. In contrast to previous methods, we develop a Bayesian diffeomorphic registration framework in a bandlimited space, rather than a high-dimensional image space. We show that a dense posterior distribution on deformation fields can be fully characterized by much fewer parameters, which dramatically reduces the computational complexity of model inferences. To further avoid heavy computation loads introduced by random sampling algorithms, we approximate a marginal posterior by using Laplace’s method at the optimal solution of log-posterior distribution. Experimental results on both 2D synthetic data and real 3D brain magnetic resonance imaging (MRI) scans demonstrate that our method is significantly faster than the state-of-the-art diffeomorphic registration uncertainty quantification algorithms, while producing comparable results.

PURPOSE: The detection of clinically significant prostate cancer (PCa) is shown to greatly benefit from MRI–ultrasound fusion biopsy, which involves overlaying pre-biopsy MRI volumes (or targets) with real-time ultrasound images. In previous literature, machine learning models trained on either MRI or ultrasound data have been proposed to improve biopsy guidance and PCa detection. However, quantitative fusion of information from MRI and ultrasound has not been explored in depth in a large study. This paper investigates information fusion approaches between MRI and ultrasound to improve targeting of PCa foci in biopsies. METHODS: We build models of fully convolutional networks (FCN) using data from a newly proposed ultrasound modality, temporal enhanced ultrasound (TeUS), and apparent diffusion coefficient (ADC) from 107 patients with 145 biopsy cores. The architecture of our models is based on U-Net and U-Net with attention gates. Models are built using joint training through intermediate and late fusion of the data. We also build models with data from each modality, separately, to use as baseline. The performance is evaluated based on the area under the curve (AUC) for predicting clinically significant PCa. RESULTS: Using our proposed deep learning framework and intermediate fusion, integration of TeUS and ADC outperforms the individual modalities for cancer detection. We achieve an AUC of 0.76 for detection of all PCa foci, and 0.89 for PCa with larger foci. Results indicate a shared representation between multiple modalities outperforms the average unimodal predictions. CONCLUSION: We demonstrate the significant potential of multimodality integration of information from MRI and TeUS to improve PCa detection, which is essential for accurate targeting of cancer foci during biopsy. By using FCNs as the architecture of choice, we are able to predict the presence of clinically significant PCa in entire imaging planes immediately, without the need for region-based analysis. This reduces the overall computational time and enables future intra-operative deployment of this technology.

PURPOSE: Brain shift during tumor resection can progressively invalidate the accuracy of neuronavigation systems and affect neurosurgeons’ ability to achieve optimal resections. This paper compares two methods that have been presented in the literature to compensate for brain shift: a thin-plate spline deformation model and a finite element method (FEM). For this comparison, both methods are driven by identical sparse data. Specifically, both methods are driven by displacements between automatically detected and matched feature points from intraoperative 3D ultrasound (iUS). Both methods have been shown to be fast enough for intraoperative brain shift correction (Machado et al. in Int J Comput Assist Radiol Surg 13(10):1525–1538, 2018; Luo et al. in J Med Imaging (Bellingham) 4(3):035003, 2017). However, the spline method requires no preprocessing and ignores physical properties of the brain while the FEM method requires significant preprocessing and incorporates patient-specific physical and geometric constraints. The goal of this work was to explore the relative merits of these methods on recent clinical data. METHODS: Data acquired during 19 sequential tumor resections in Brigham and Women’s Hospital’s Advanced Multi-modal Image-Guided Operating Suite between December 2017 and October 2018 were considered for this retrospective study. Of these, 15 cases and a total of 24 iUS to iUS image pairs met inclusion requirements. Automatic feature detection (Machado et al. in Int J Comput Assist Radiol Surg 13(10):1525–1538, 2018) was used to detect and match features in each pair of iUS images. Displacements between matched features were then used to drive both the spline model and the FEM method to compensate for brain shift between image acquisitions. The accuracies of the resultant deformation models were measured by comparing the displacements of manually identified landmarks before and after deformation. RESULTS: The mean initial subcortical registration error between preoperative MRI and the first iUS image averaged 5.3 ± 0.75 mm. The mean subcortical brain shift, measured using displacements between manually identified landmarks in pairs of iUS images, was 2.5 ± 1.3 mm. Our results showed that FEM was able to reduce subcortical registration error by a small but statistically significant amount (from 2.46 to 2.02 mm). A large variability in the results of the spline method prevented us from demonstrating either a statistically significant reduction in subcortical registration error after applying the spline method or a statistically significant difference between the results of the two methods. CONCLUSIONS: In this study, we observed less subcortical brain shift than has previously been reported in the literature (Frisken et al., in: Miller (ed) Biomechanics of the brain, Springer, Cham, 2019). This may be due to the fact that we separated out the initial misregistration between preoperative MRI and the first iUS image from our brain shift measurements or it may be due to modern neurosurgical practices designed to reduce brain shift, including reduced craniotomy sizes and better control of intracranial pressure with the use of mannitol and other medications. It appears that the FEM method and its use of geometric and biomechanical constraints provided more consistent brain shift correction and better correction farther from the driving feature displacements than the simple spline model. The spline-based method was simpler and tended to give better results for small deformations. However, large variability in the spline results and relatively small brain shift prevented this study from demonstrating a statistically significant difference between the results of the two methods.

BACKGROUND AND PURPOSE: 3D reconstruction of a targeted area (“safe” triangle and Kambin triangle) may benefit the viability assessment of transforaminal epidural steroid injection, especially at the L5/S1 level. However, manual segmentation of lumbosacral nerves for 3D reconstruction is time-consuming. The aim of this study was to investigate the feasibility of deep learning–based segmentation of lumbosacral nerves on CT and the reconstruction of the safe triangle and Kambin triangle. MATERIALS AND METHODS: A total of 50 cases of spinal CT were manually labeled for lumbosacral nerves and bones using Slicer 4.8. The ratio of training/validation/testing was 32:8:10. A 3D U-Net was adopted to build the model SPINECT for automatic segmentations of lumbosacral structures. The Dice score, pixel accuracy, and Intersection over Union were computed to assess the segmentation performance of SPINECT. The areas of Kambin and safe triangles were measured to validate the 3D reconstruction. RESULTS: The results revealed successful segmentation of lumbosacral bone and nerve on CT. The average pixel accuracy for bone was 0.940, and for nerve, 0.918. The average Intersection over Union for bone was 0.897 and for nerve, 0.827. The Dice score for bone was 0.945, and for nerve, it was 0.905. There were no significant differences in the quantified Kambin triangle or safe triangle between manually segmented images and automatically segmented images (P > .05). CONCLUSIONS: Deep learning–based automatic segmentation of lumbosacral structures (nerves and bone) on routine CT is feasible, and SPINECT-based 3D reconstruction of safe and Kambin triangles is also validated.

This paper presents a novel approach to modeling the pos terior distribution in image registration that is computationally efficient for large deformation diffeomorphic metric mapping (LDDMM). We develop a Laplace approximation of Bayesian registration models entirely in a bandlimited space that fully describes the properties of diffeomorphic transformations. In contrast to current methods, we compute the inverse Hessian at the mode of the posterior distribution of diffeomorphisms directly in the low dimensional frequency domain. This dramatically reduces the computational complexity of approximating posterior marginals in the high dimensional imaging space. Experimental results show that our method is significantly faster than the state-of-the-art diffeomorphic image registration uncertainty quantification algorithms, while producing comparable results. The efficiency of our method strengthens the feasibility in prospective clinical applications, e.g., real- time image-guided navigation for brain surgery.

Multimodal integration between mass spectrometry imaging (MSI) and radiology-established modalities such as magnetic resonance imaging (MRI) would allow the investigations of key questions in complex biological systems such as the central nervous system. Such integration would provide complementary multiscale data to bridge the gap between molecular and anatomical phenotypes, potentially revealing new insights into molecular mechanisms underlying anatomical pathologies presented on MRI. Automatic coregistration between 3D MSI/MRI is a computationally challenging process due to dimensional complexity, MSI data sparsity, lack of direct spatial-correspondences, and nonlinear tissue deformation. Here, we present a new computational approach based on stochastic neighbor embedding to nonlinearly align 3D MSI to MRI data, identify and reconstruct biologically relevant molecular patterns in 3D, and fuse the MSI datacube to the MRI space. We demonstrate our method using multimodal high-spectral resolution matrix-assisted laser desorption ionization (MALDI) 9.4 T MSI and 7 T in vivo MRI data, acquired from a patient-derived, xenograft mouse brain model of glioblastoma following administration of the EGFR inhibitor drug of Erlotinib. Results show the distribution of some identified molecular ions of the EGFR inhibitor erlotinib, a phosphatidylcholine lipid, and cholesterol, which were reconstructed in 3D and mapped to the MRI space. The registration quality was evaluated on two normal mouse brains using the Dice coefficient for the regions of brainstem, hippocampus, and cortex. The method is generic and can therefore be applied to hyperspectral images from different mass spectrometers and integrated with other established in vivo imaging modalities such as computed tomography (CT) and positron emission tomography (PET).

To investigate the heating of EEG electrodes during magnetic resonance imaging (MRI) scans and to better understand the underlying physical mechanisms with a focus on the antenna effect.Gold cup and conductive plastic electrodes were placed on small watermelons with fiberoptic probes used to measure electrode temperature changes during a variety of 1.5T and 3T MRI scans. A subset of these experiments was repeated on a healthy human volunteer.The differences between gold and plastic electrodes did not appear to be practically significant. For both electrode types, we observed heating below 4°C for straight wires whose lengths were multiples of ½ the radiofrequency (RF) wavelength and stronger heating (over 15°C) for wire lengths that were odd multiples of ¼ RF wavelength, consistent with the antenna effect.The antenna effect, which has received little attention so far in the context of EEG-MRI safety, can play as significant a role as the loop effect (from electromagnetic induction) in the heating of EEG electrodes, and therefore wire lengths that are odd multiples of ¼ RF wavelength should be avoided. These results have important implications for the design of EEG electrodes and MRI studies as they help to minimize the risk to patients undergoing MRI with EEG electrodes in place.

Neurosurgery makes use of preoperative imaging to visualize pathology, inform surgical planning, and evaluate the safety of selected approaches. The utility of preoperative imaging for neuronavigation, however, is diminished by the well-characterized phenomenon of brain shift, in which the brain deforms intraoperatively as a result of craniotomy, swelling, gravity, tumor resection, cerebrospinal fluid (CSF) drainage, and many other factors. As such, there is a need for updated intraoperative information that accurately reflects intraoperative conditions. Since 1982, intraoperative ultrasound has allowed neurosurgeons to craft and update operative plans without ionizing radiation exposure or major workflow interruption. Continued evolution of ultrasound technology since its introduction has resulted in superior imaging quality, smaller probes, and more seamless integration with neuronavigation systems. Furthermore, the introduction of related imaging modalities, such as 3-dimensional ultrasound, contrast-enhanced ultrasound, high-frequency ultrasound, and ultrasound elastography, has dramatically expanded the options available to the neurosurgeon intraoperatively. In the context of these advances, we review the current state, potential, and challenges of intraoperative ultrasound for brain tumor resection. We begin by evaluating these ultrasound technologies and their relative advantages and disadvantages. We then review three specific applications of these ultrasound technologies to brain tumor resection: (1) intraoperative navigation, (2) assessment of extent of resection, and (3) brain shift monitoring and compensation. We conclude by identifying opportunities for future directions in the development of ultrasound technologies.

Purpose: Matching points that are derived from features or landmarks in image data is a key step in some medical imaging applications. Since most robust point matching algorithms claim to be able to deal with outliers, users may place high confidence in the matching result and use it without further examination. However, for tasks such as feature-based registration in image-guided neurosurgery, even a few mismatches, in the form of invalid displacement vectors, could cause serious consequences. As a result, having an effective tool by which operators can manually screen all matches for outliers could substantially benefit the outcome of those applications. Methods: We introduce a novel variogram-based outlier screening method for vectors. The variogram is a powerful geostatistical tool for characterizing the spatial dependence of stochastic processes. Since the spatial correlation of invalid displacement vectors, which are considered as vector outliers, tends to behave differently than normal displacement vectors, they can be efficiently identified on the variogram. Results: We validate the proposed method on 9 sets of clinically acquired ultrasound data. In the experiment, potential outliers are flagged on the variogram by one operator and further evaluated by 8 experienced medical imaging researchers. The matching quality of those potential outliers is approximately 1.5 lower, on a scale from 1 (bad) to 5 (good), than valid displacement vectors. Conclusion: The variogram is a simple yet informative tool. While being used extensively in geostatistical analysis, it has not received enough attention in the medical imaging field. We believe there is a good deal of potential for clinically applying the proposed outlier screening method. By way of this paper, we also expect researchers to find variogram useful in other medical applications that involve motion vectors analyses., National Institutes of Health (U.S.) (Grant P41-EB015898-09), National Institutes of Health (U.S.) (Grant P41-EB015902)

Image-guidance improves tissue sampling during biopsy by allowing the physician to visualize the tip and trajectory of the biopsy needle relative to the target in MRI, CT, ultrasound, or other relevant imagery. This paper reports a system for fast automatic needle tip and trajectory localization and visualization in MRI that has been developed and tested in the context of an active clinical research program in prostate biopsy. To the best of our knowledge, this is the first reported system for this clinical application, and also the first reported system that leverages deep neural networks for segmentation and localization of needles in MRI across biomedical applications. Needle tip and trajectory were annotated on 583 T2-weighted intra-procedural MRI scans acquired after needle insertion for 71 patients who underwent transperenial MRI-targeted biopsy procedure at our institution. The images were divided into two independent training-validation and test sets at the patient level. A deep 3-dimensional fully convolutional neural network model was developed, trained and deployed on these samples. The accuracy of the proposed method, as tested on previously unseen data, was 2.80 mm average in needle tip detection, and 0.98ᵒ in needle trajectory angle. An observer study was designed in which independent annotations by a second observer, blinded to the original observer, were compared to the output of the proposed method. The resultant error was comparable to the measured inter-observer concordance, reinforcing the clinical acceptability of the proposed method. The proposed system has the potential for deployment in clinical routine.

Purpose: The brain undergoes significant structural change over the course of neurosurgery, including highly nonlinear deformation and resection. It can be informative to recover the spatial mapping between structures identified in preoperative surgical planning and the intraoperative state of the brain. We present a novel feature-based method for achieving robust, fully automatic deformable registration of intraoperative neurosurgical ultrasound images. Methods: A sparse set of local image feature correspondences is first estimated between ultrasound image pairs, after which rigid, affine and thin-plate spline models are used to estimate dense mappings throughout the image. Correspondences are derived from 3D features, distinctive generic image patterns that are automatically extracted from 3D ultrasound images and characterized in terms of their geometry (i.e., location, scale, and orientation) and a descriptor of local image appearance. Feature correspondences between ultrasound images are achieved based on a nearest-neighbor descriptor matching and probabilistic voting model similar to the Hough transform. Results: Experiments demonstrate our method on intraoperative ultrasound images acquired before and after opening of the dura mater, during resection and after resection in nine clinical cases. A total of 1620 automatically extracted 3D feature correspondences were manually validated by eleven experts and used to guide the registration. Then, using manually labeled corresponding landmarks in the pre- and post-resection ultrasound images, we show that our feature-based registration reduces the mean target registration error from an initial value of 3.3 to 1.5 mm. Conclusions: This result demonstrates that the 3D features promise to offer a robust and accurate solution for 3D ultrasound registration and to correct for brain shift in image-guided neurosurgery., National Institutes of Health (U.S.) (Grant P41-EB015898-09), National Institutes of Health (U.S.) (Grant P41-EB015902), National Institutes of Health (U.S.) (Grant R01-NS049251), Portuguese Foundation for International Cooperation in Science, Technology and Higher Education (Grant PD/BD/105869/2014), Portuguese Foundation for International Cooperation in Science, Technology and Higher Education (Grant IDMEC/LAETA UID/EMS/50022/2013)

We address the problem of interpolating randomly non-uniformly spatiotemporally scattered uncertain motion measurements, which arises in the context of soft tissue motion estimation. Soft tissue motion estimation is of great interest in the field of image-guided soft-tissue intervention and surgery navigation, because it enables the registration of pre-interventional/pre-operative navigation information on deformable soft-tissue organs. To formally define the measurements as spatiotemporally scattered motion signal samples, we propose a novel motion field representation. To perform the interpolation of the motion measurements in an uncertainty-aware optimal unbiased fashion, we devise a novel Gaussian process (GP) regression model with a non-constant-mean prior and an anisotropic covariance function and show through an extensive evaluation that it outperforms the state-of-the-art GP models that have been deployed previously for similar tasks. The employment of GP regression enables the quantification of uncertainty in the interpolation result, which would allow the amount of uncertainty present in the registered navigation information governing the decisions of the surgeon or intervention specialist to be conveyed.

To determine the detection rate, clinical relevance, Gleason grade, and location of prostate cancer ( PCa prostate cancer ) diagnosed with and the safety of an in-bore transperineal 3-T magnetic resonance (MR) imaging-guided prostate biopsy in a clinically heterogeneous patient population.This prospective retrospectively analyzed study was HIPAA compliant and institutional review board approved, and informed consent was obtained. Eighty-seven men (mean age, 66.2 years ± 6.9) underwent multiparametric endorectal prostate MR imaging at 3 T and transperineal MR imaging-guided biopsy. Three subgroups of patients with at least one lesion suspicious for cancer were included: men with no prior PCa prostate cancer diagnosis, men with PCa prostate cancer who were undergoing active surveillance, and men with treated PCa prostate cancer and suspected recurrence. Exclusion criteria were prior prostatectomy and/or contraindication to 3-T MR imaging. The transperineal MR imaging-guided biopsy was performed in a 70-cm wide-bore 3-T device. Overall patient biopsy outcomes, cancer detection rates, Gleason grade, and location for each subgroup were evaluated and statistically compared by using χ(2) and one-way analysis of variance followed by Tukey honestly significant difference post hoc comparisons.Ninety biopsy procedures were performed with no serious adverse events, with a mean of 3.7 targets sampled per gland. Cancer was detected in 51 (56.7%) men: 48.1% (25 of 52) with no prior PCa prostate cancer , 61.5% (eight of 13) under active surveillance, and 72.0% (18 of 25) in whom recurrence was suspected. Gleason pattern 4 or higher was diagnosed in 78.1% (25 of 32) in the no prior PCa prostate cancer and active surveillance groups. Gleason scores were not assigned in the suspected recurrence group. MR targets located in the anterior prostate had the highest cancer yield (40 of 64, 62.5%) compared with those for the other parts of the prostate (P.001).In-bore 3-T transperineal MR imaging-guided biopsy, with a mean of 3.7 targets per gland, allowed detection of many clinically relevant cancers, many of which were located anteriorly.

Using electroencephalography (EEG) to elucidate the spontaneous activation of brain resting-state networks (RSNs) is nontrivial as the signal of interest is of low amplitude and it is difficult to distinguish the underlying neural sources. Using the principles of electric field topographical analysis, it is possible to estimate the meta-stable states of the brain (i.e., the resting-state topographies, so-called microstates). We estimated seven resting-state topographies explaining the EEG data set with k-means clustering (N = 164, 256 electrodes). Using a method specifically designed to localize the sources of broadband EEG scalp topographies by matching sensor and source space temporal patterns, we demonstrated that we can estimate the EEG RSNs reliably by measuring the reproducibility of our findings. After subtracting their mean from the seven EEG RSNs, we identified seven state-specific networks. The mean map includes regions known to be densely anatomically and functionally connected (superior frontal, superior parietal, insula, and anterior cingulate cortices). While the mean map can be interpreted as a “router,” crosslinking multiple functional networks, the seven state-specific RSNs partly resemble and extend previous functional magnetic resonance imaging-based networks estimated as the hemodynamic correlates of four canonical EEG microstates.

Standard image based segmentation approaches perform poorly when there is little or no contrast along boundaries of different regions. In such cases, segmentation is largely performed manually using prior knowledge of the shape and relative location of the underlying structures combined with partially discernible boundaries. We present an automated approach guided by covariant shape deformations of neighboring structures, which is an additional source of prior information. Captured by a shape atlas, these deformations are transformed into a statistical model using the logistic function. Structure boundaries, anatomical labels, and image inhomogeneities are estimated simultaneously within an Expectation-Maximization formulation of the maximum a posteriori probability estimation problem. We demonstrate the approach on 20 brain magnetic resonance images showing superior performance, particularly in cases where purely image based methods fail.

This paper presents a novel segmentation approach featuring shape constraints of multiple structures. A framework is developed combining statistical shape modeling with a maximum a posteriori segmentation problem. The shape is characterized by signed distance maps and its modes of variations are generated through principle component analysis. To solve the maximum a posteriori segmentation problem a robust Expectation Maximization implementation is used. The Expectation Maximization segmenter generates a label map, calculates image intensity inhomogeneities, and considers shape constraints for each structure of interest. Our approach enables high quality segmentations of structures with weak image boundaries which is demonstrated by automatically segmenting 32 brain MRIs into right and left thalami.

White matter segmentation is an essential step to study whole-brain structural connectivity via diffusion MRI white matter tractography. One important goal of segmentation methods is to improve consistency of the white matter segmentations across multiple subjects. In this study, we quantitatively compare two popular white matter segmentation strategies, i.e., a cortical-parcellation-based method and a groupwise fiber clustering method, to investigate their performance on consistency. Our experimental results indicate that the groupwise fiber clustering generated more consistent segmentations with lower variability across subjects. This suggests that the fiber clustering strategy could provide a potential alternative to the traditional cortical-parcellation-based brain connectivity modeling methods.

Purpose Reliably detecting MRI signals in the brain that are more tightly coupled to neural activity than blood-oxygen-level-dependent fMRI signals could not only prove valuable for basic scientific research but could also enhance clinical applications such as epilepsy presurgical mapping. This endeavor will likely benefit from an improved understanding of the behavior of ionic currents, the mediators of neural activity, in the presence of the strong magnetic fields that are typical of modern-day MRI scanners. Theory Of the various mechanisms that have been proposed to explain the behavior of ionic volume currents in a magnetic field, only one—magnetohydrodynamic flow—predicts a slow evolution of signals, on the order of a minute for normal saline in a typical MRI scanner. Methods This prediction was tested by scanning a volume-current phantom containing normal saline with gradient-echo-planar imaging at 3 T. Results Greater signal changes were observed in the phase of the images than in the magnitude, with the changes evolving on the order of a minute. Conclusion These results provide experimental support for the MHD flow hypothesis. Furthermore, MHD-driven cerebrospinal fluid flow could provide a novel fMRI contrast mechanism. Magn Reson Med 74:1145–1155, 2015. © 2014 Wiley Periodicals, Inc.

The Incidence of renal cell carcinoma in the United States has increased by 126% over the past 50 years (1), together with an increase in its associated mortality rates (2). Image-guided percutaneous cryoablation is one of the most promising and prevailing minimally invasive techniques for treating kidney cancer (3–5). Compared with computed tomography (CT), MRI is considered a superior intraprocedural imaging tool to visualize the ablation zone, referred to as the “iceball”, the tumor, and, nearby anatomical structures (6). The current cryoablation procedure relies on the interventionalist’s constant qualitative assessment of repeatedly acquired intraprocedural MR images during the 15-min freezing cycle. The interventionalist has to evaluate the coverage of tumor and surrounding normal parenchyma by the slowly growing iceball, while at the same time tending to other procedural needs, such as keeping the skin entry site warm and monitoring patient’s medical status. At times, this may lead to inaccurate or incomplete interpretation of the progress of the ablation. Also, it could cause deficient tumor coverage that would later lead to recurrence, and injury to the surrounding critical structures with subsequent complications (7). Hence, there is a need for a computerized monitoring tool to aid the interventionalist by providing quantitative tumor ablation metrics and three-dimensional (3D) visualization of the iceball. One key component of such a tool is fast and accurate segmentation of the 3D iceball configuration, which is displayed as an approximately ball-shaped signal void region in MR images. The accuracy of many semi-automatic segmentation tools, such as the ITK-SNAP (www.itksnap.org) (8) and the GrowCut module (9) in 3D Slicer (www.slicer.org) (10), is proportional to the manual operations involved, e.g., the inputting of fiducials and the extent of the segmented region growth, and interactive editing until segmentation is complete. The time required to perform these manual operations makes semiautomatic approaches impractical for intraprocedural segmentation during cryoablation. Automatic iceball segmentation is challenging, as the abdominal anatomy surrounding the iceball, the probe artifacts, and the iceball itself have similar intensity values. This causes the failure of segmentation methods relying solely on intensity information. The approach of Liu et al (11) using graph cut (12–16) with shape priors (17–19) generated from probe locations (named GC-prior) showed promise. However, directly using the modeled iceball shape based on probe locations in the baseline scan is vulnerable to errors, such as those caused by inaccurate probe localization, imperfection of iceball shape modeling, or misregistration between the baseline scan and images obtained during the freeze cycles due to patient’s movement. The purpose of this study is to develop an automatic iceball segmentation method that is robust to the predetected probes, so that the overall accuracy of the segmentation can be improved. Different from previous work (11), our segmentation method is initialized with semiautomatically detected 3D cryoablation probes rather than manually labeled 2D probes. The former probe localization method is considered more suitable for a clinical setting. We incorporated a pre-segmentation step using the Fuzzy C-means (FCM) classification paradigm (20), and a shape prior adaptation step using a Fast Marching propagation process (21,22). These steps provide an adapted iceball shape for modeling the priors, as well as initial intensity centroids, both to be input into the graph cut segmentation. In addition, we introduce intensity normalization and noise reduction steps for better segmentation results. Finally, this study reports results based on a wider selection of cases (13 subjects) compared with the two cases reported in (11).

Prostate cancer (PCa) remains a leading cause of cancer mortality among American men. Multi-parametric magnetic resonance imaging (mpMRI) is widely used to assist with detection of PCa and characterization of its aggressiveness. Computer-aided diagnosis (CADx) of PCa in MRI can be used as clinical decision support system to aid radiologists in interpretation and reporting of mpMRI. We report on the development of a convolution neural network (CNN) model to support CADx in PCa based on the appearance of prostate tissue in mpMRI, conducted as part of the SPIE-AAPM-NCI PROSTATEx challenge. The performance of different combinations of mpMRI inputs to CNN was assessed and the best result was achieved using DWI and DCE-MRI modalities together with the zonal information of the finding. On the test set, the model achieved an area under the receiver operating characteristic curve of 0.80.

© 2017 Elsevier B.V. We present an efficient probabilistic model of anatomical variability in a linear space of initial velocities of diffeomorphic transformations and demonstrate its benefits in clinical studies of brain anatomy. To overcome the computational challenges of the high dimensional deformation-based descriptors, we develop a latent variable model for principal geodesic analysis (PGA) based on a low dimensional shape descriptor that effectively captures the intrinsic variability in a population. We define a novel shape prior that explicitly represents principal modes as a multivariate complex Gaussian distribution on the initial velocities in a bandlimited space. We demonstrate the performance of our model on a set of 3D brain MRI scans from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Our model yields a more compact representation of group variation at substantially lower computational cost than the state-of-the-art method such as tangent space PCA (TPCA) and probabilistic principal geodesic analysis (PPGA) that operate in the high dimensional image space.

International audience; Current theories hold that brain function is highly related with long-range physical connections through axonal bundles, namely extrinsic connectivity. However, obtaining a groupwise cortical parcella-tion based on extrinsic connectivity remains challenging. Current par-cellation methods are computationally expensive; need tuning of several parameters or rely on ad-hoc constraints. Furthermore, none of these methods present a model for the cortical extrinsic connectivity. To tackle these problems, we propose a parsimonious model for the extrinsic con-nectivity and an efficient parcellation technique based on clustering of tractograms. Our technique allows the creation of single subject and groupwise parcellations of the whole cortex. The parcellations obtained with our technique are in agreement with anatomical and functional par-cellations in the literature. In particular, the motor and sensory cortex are subdivided in agreement with the human homunculus of Penfield. We illustrate this by comparing our resulting parcels with an anatomical atlas and the motor strip mapping included in the Human Connectome Project data.

Differences in scanning parameters or modalities can complicate image analysis based on supervised classification. This paper presents two representation learning approaches, based on autoencoders, that address this problem by learning representations that are similar across domains. Both approaches use, next to the data representation objective, a similarity objective to minimise the difference between representations of corresponding patches from each domain. We evaluated the methods in transfer learning experiments on multi-modal brain MRI data and on synthetic data. After transforming training and test data from different modalities to the common representations learned by our methods, we trained classifiers for each of pair of modalities. We found that adding the similarity term to the standard objective can produce representations that are more similar and can give a higher accuracy in these cross-modality classification experiments.

International audience; This paper presents a novel mathematical framework for representing uncertainty in large deformation diffeomorphic image registration. The Bayesian posterior distribution over the deformations aligning a moving and a fixed image is approximated via a variational formulation. A stochastic differential equation (SDE) modeling the deformations as the evolution of a time-varying velocity field leads to a prior density over deformations in the form of a Gaussian process. This permits estimating the full posterior distribution in order to represent uncertainty, in contrast to methods in which the posterior is approximated via Monte Carlo sampling or maximized in maximum a-posteriori (MAP) estimation. The frame-work is demonstrated in the case of landmark-based image registration, including simulated data and annotated pre and intra-operative 3D images.

We propose a method for the automated identification of key white matter fiber tracts for neurosurgical planning, and we apply the method in a retrospective study of 18 consecutive neurosurgical patients with brain tumors. Our method is designed to be relatively robust to challenges in neurosurgical tractography, which include peritumoral edema, displacement, and mass effect caused by mass lesions. The proposed method has two parts. First, we learn a data-driven white matter parcellation or fiber cluster atlas using groupwise registration and spectral clustering of multi-fiber tractography from healthy controls. Key fiber tract clusters are identified in the atlas. Next, patient-specific fiber tracts are automatically identified using tractography-based registration to the atlas and spectral embedding of patient tractography. Results indicate good generalization of the data-driven atlas to patients: 80% of the 800 fiber clusters were identified in all 18 patients, and 94% of the 800 fiber clusters were found in 16 or more of the 18 patients. Automated subject-specific tract identification was evaluated by quantitative comparison to subject-specific motor and language functional MRI, focusing on the arcuate fasciculus (language) and corticospinal tracts (motor), which were identified in all patients. Results indicate good colocalization: 89 of 95, or 94%, of patient-specific language and motor activations were intersected by the corresponding identified tract. All patient-specific activations were within 3mm of the corresponding language or motor tract. Overall, our results indicate the potential of an automated method for identifying fiber tracts of interest for neurosurgical planning, even in patients with mass lesions., Highlights • Spectral clustering machine learning approach for white matter tract identification • Data-driven white matter parcellation learned from healthy subjects tractography • White matter parcellation applied to 18 consecutive patients with brain tumors • Arcuate fasciculus and corticospinal tracts identified in all patients • All tracts within 3 mm of corresponding patient-specific functional activations

The delineation of functionally and structurally distinct regions as well as their connectivity can provide key knowledge towards understanding the brain’s behaviour and function. Cytoarchitecture has long been the gold standard for such parcellation tasks, but has poor scalability and cannot be mapped in vivo. Functional and diffusion magnetic resonance imaging allow in vivo mapping of brain’s connectivity and the parcellation of the brain based on local connectivity information. Several methods have been developed for single subject connectivity driven parcellation, but very few have tackled the task of group-wise parcellation, which is essential for uncovering group specific behaviours. In this paper, we propose a group-wise connectivity-driven parcellation method based on spectral clustering that captures local connectivity information at multiple scales and directly enforces correspondences between subjects. The method is applied to diffusion Magnetic Resonance Imaging driven parcellation on two independent groups of 50 subjects from the Human Connectome Project. Promising quantitative and qualitative results in terms of information loss, modality comparisons, group consistency and inter-group similarities demonstrate the potential of the method.

The National Alliance for Medical Image Computing (NA-MIC) was launched in 2004 with the goal of investigating and developing an open source software infrastructure for the extraction of information and knowledge from medical images using computational methods. Several leading research and engineering groups participated in this effort that was funded by the US National Institutes of Health through a variety of infrastructure grants. This effort transformed 3D Slicer from an internal, Boston-based, academic research software application into a professionally maintained, robust, open source platform with an international leadership and developer and user communities. Critical improvements to the widely used underlying open source libraries and tools—VTK, ITK, CMake, CDash, DCMTK—were an additional consequence of this effort. This project has contributed to close to a thousand peer-reviewed publications and a growing portfolio of US and international funded efforts expanding the use of these tools in new medical computing applications every year. In this editorial, we discuss what we believe are gaps in the way medical image computing is pursued today; how a well-executed research platform can enable discovery, innovation and reproducible science (“Open Science”); and how our quest to build such a software platform has evolved into a productive and rewarding social engineering exercise in building an open-access community with a shared vision.