Search

Self-training is an important class of unsupervised domain adaptation (UDA) approaches that are used to mitigate the problem of domain shift, when applying knowledge learned from a labeled source domain to unlabeled and heterogeneous target domains. While self-training-based UDA has shown considerable promise on discriminative tasks, including classification and segmentation, through reliable pseudo-label filtering based on the maximum softmax probability, there is a paucity of prior work on self-training-based UDA for generative tasks, including image modality translation. To fill this gap, in this work, we seek to develop a generative self-training (GST) framework for domain adaptive image translation with continuous value prediction and regression objectives. Specifically, we quantify both aleatoric and epistemic uncertainties within our GST using variational Bayes learning to measure the reliability of synthesized data. We also introduce a self-attention scheme that de-emphasizes the background region to prevent it from dominating the training process. The adaptation is then carried out by an alternating optimization scheme with target domain supervision that focuses attention on the regions with reliable pseudo-labels. We evaluated our framework on two cross-scanner/center, inter-subject translation tasks, including tagged-to-cine magnetic resonance (MR) image translation and T1-weighted MR-to-fractional anisotropy translation. Extensive validations with unpaired target domain data showed that our GST yielded superior synthesis performance in comparison to adversarial training UDA methods. [ABSTRACT FROM AUTHOR]

Diffusion magnetic resonance imaging (dMRI) has enabled in vivo investigation of white matter tracts. Fiber orientation (FO) estimation is a key step in tract reconstruction and has been a popular research topic in dMRI analysis. In particular, the sparsity assumption has been used in conjunction with a dictionary-based framework to achieve reliable FO estimation with a reduced number of gradient directions. Because image noise can have a deleterious effect on the accuracy of FO estimation, previous works have incorporated spatial consistency of FOs in the dictionary-based framework to improve the estimation. However, because FOs are only indirectly determined from the mixture fractions of dictionary atoms and not modeled as variables in the objective function, these methods do not incorporate FO smoothness directly, and their ability to produce smooth FOs could be limited. In this work, we propose an improvement to Fiber Orientation Reconstruction using Neighborhood Information (FORNI), which we call FORNI+; this method estimates FOs in a dictionary-based framework where FO smoothness is better enforced than in FORNI alone. We describe an objective function that explicitly models the actual FOs and the mixture fractions of dictionary atoms. Specifically, it consists of data fidelity between the observed signals and the signals represented by the dictionary, pairwise FO dissimilarity that encourages FO smoothness, and weighted ℓ 1 -norm terms that ensure the consistency between the actual FOs and the FO configuration suggested by the dictionary representation. The FOs and mixture fractions are then jointly estimated by minimizing the objective function using an iterative alternating optimization strategy. FORNI+ was evaluated on a simulation phantom, a physical phantom, and real brain dMRI data. In particular, in the real brain dMRI experiment, we have qualitatively and quantitatively evaluated the reproducibility of the proposed method. Results demonstrate that FORNI+ produces FOs with better quality compared with competing methods. [ABSTRACT FROM AUTHOR]

Image labeling is essential for analyzing morphometric features in medical imaging data. Labels can be obtained by either human interaction or automated segmentation algorithms, both of which suffer from errors. The Simultaneous Truth and Performance Level Estimation (STAPLE) algorithm for both discrete-valued and continuous-valued labels has been proposed to find the consensus fusion while simultaneously estimating rater performance. In this paper, we first show that the previously reported continuous STAPLE in which bias and variance are used to represent rater performance yields a maximum likelihood solution in which bias is indeterminate. We then analyze the major cause of the deficiency and evaluate two classes of auxiliary bias estimation processes, one that estimates the bias as part of the algorithm initialization and the other that uses a maximum a posteriori criterion with a priori probabilities on the rater bias. We compare the efficacy of six methods, three variants from each class, in simulations and through empirical human rater experiments. We comment on their properties, identify deficient methods, and propose effective methods as solution. [ABSTRACT FROM AUTHOR]

Image registration is a crucial step in many medical image analysis procedures such as image fusion, surgical planning, segmentation and labeling, and shape comparison in population or longitudinal studies. A new approach to volumetric intersubject deformable image registration is presented. The method, called Mjolnir, is an extension of the highly successful method HAMMER.New image features in order to better localize points of correspondence between the two images are introduced as well as a novel approach to generate a dense displacement field based upon the weighted diffusion of automatically derived feature correspondences. An extensive validation of the algorithm was performed on T1-weighted SPGR MR brain images from the NIREP evaluation database. The results were compared with results generated by HAMMER and are shown to yield significant improvements in cortical alignment as well as reduced computation time. [ABSTRACT FROM AUTHOR]

Abstract: A method to track a grid of cardiac material points in three dimensions using slice-following (SF) tagged magnetic resonance imaging (MRI) and harmonic-phase MRI is presented. A three-dimensional grid of material points on the lines of intersections of short-axis (SA) and long-axis (LA) planes is automatically tracked by combining two-dimensional pathlines that are computed on both SA and LA image planes. This process yields the true three-dimensional motion of points originating on the image plane intersections. Experimental data from normal volunteers, each obtained in four short breath-holds using the SF harmonic phase MRI pulse sequence, is presented. A validation of two-dimensional in-plane tracks using this pulse sequence on a moving phantom is also presented. [Copyright &y& Elsevier]

We outline an Eulerian framework for computing the thickness of tissues between two simply connected boundaries that does not require landmark points or parameterizations of either boundary. Thickness is defined as the length of correspondence trajectories, which run from one tissue boundary to the other, and which follow a smooth vector field constructed in the region between the boundaries. A pair of partial differential equations (PDEs) that are guided by this vector field are then solved over this region, and the sum of their solutions yields the thickness of the tissue region. Unlike other approaches, this approach does not require explicit construction of any correspondence trajectories. An efficient, stable, and computationally fast solution to these PDEs is found by careful selection of finite differences according to an upwinding condition. The behavior and performance of our method is demonstrated on two simulations and two magnetic resonance imaging data sets in two and three dimensions. These experiments reveal very good performance and show strong potential for application in tissue thickness visualization and quantification. [ABSTRACT FROM AUTHOR]

This editorial introduces the Special Issue on Simulation and Synthesis in Medical Imaging. In this editorial, we define so-far ambiguous terms of simulation and synthesis in medical imaging. We also briefly discuss the synergistic importance of mechanistic (hypothesis-driven) and phenomenological (data-driven) models of medical image generation. Finally, we introduce the twelve papers published in this issue covering both mechanistic (5) and phenomenological (7) medical image generation. This rich selection of papers covers applications in cardiology, retinopathy, histopathology, neurosciences, and oncology. It also covers all mainstream diagnostic medical imaging modalities. We conclude the editorial with a personal view on the field and highlight some existing challenges and future research opportunities. [ABSTRACT FROM PUBLISHER]

Abstract: Measurement of myocardial strain provides direct information about heart function that can be correlated with disease. We present an MRI pulse sequence that acquires in just six heartbeats both harmonic phase (HARP) and strain-encoded (SENC) images and provides dense measurements of radial, circumferential and longitudinal strains within a single short-axis slice. Normal volunteer data confirm the feasibility of this pulse sequence, and acquired data demonstrate the strain measurement reliability. [Copyright &y& Elsevier]

Image segmentation plays a crucial role in many medical-imaging applications, by automating or facilitating the delineation of anatomical structures and other regions of interest. We present a critical appraisal of the current status of semiautomated and automated methods for the segmentation of anatomical medical images. Terminology and important issues in image segmentation are first presented. Current segmentation approaches are then reviewed with an emphasis on the advantages and disadvantages of these methods for medical imaging applications. We conclude with a discussion on the future of image segmentation methods in biomedical research. [ABSTRACT FROM AUTHOR]

The Sørensen-Dice index (SDI) is a widely used measure for evaluating medical image segmentation algorithms. It offers a standardized measure of segmentation accuracy which has proven useful. However, it offers diminishing insight when the number of objects is unknown, such as in white matter lesion segmentation of multiple sclerosis (MS) patients. We present a refinement for finer grained parsing of SDI results in situations where the number of objects is unknown. We explore these ideas with two case studies showing what can be learned from our two presented studies. Our first study explores an inter-rater comparison, showing that smaller lesions cannot be reliably identified. In our second case study, we demonstrate fusing multiple MS lesion segmentation algorithms based on the insights into the algorithms provided by our analysis to generate a segmentation that exhibits improved performance. This work demonstrates the wealth of information that can be learned from refined analysis of medical image segmentations. [ABSTRACT FROM AUTHOR]

Magnetic resonance imaging (MRI) is a flexible medical imaging modality that often lacks reproducibility between protocols and scanners. It has been shown that even when care is taken to standardize acquisitions, any changes in hardware, software, or protocol design can lead to differences in quantitative results. This greatly impacts the quantitative utility of MRI in multi-site or long-term studies, where consistency is often valued over image quality. We propose a method of contrast harmonization, called DeepHarmony, which uses a U-Net-based deep learning architecture to produce images with consistent contrast. To provide training data, a small overlap cohort (n = 8) was scanned using two different protocols. Images harmonized with DeepHarmony showed significant improvement in consistency of volume quantification between scanning protocols. A longitudinal MRI dataset of patients with multiple sclerosis was also used to evaluate the effect of a protocol change on atrophy calculations in a clinical research setting. The results show that atrophy calculations were substantially and significantly affected by protocol change, whereas such changes have a less significant effect and substantially reduced overall difference when using DeepHarmony. This establishes that DeepHarmony can be used with an overlap cohort to reduce inconsistencies in segmentation caused by changes in scanner protocol, allowing for modernization of hardware and protocol design in long-term studies without invalidating previously acquired data. [ABSTRACT FROM AUTHOR]

Multi-modal deformable registration is important for many medical image analysis tasks such as atlas alignment, image fusion, and distortion correction. Whereas a conventional method would register images with different modalities using modality independent features or information theoretic metrics such as mutual information, this paper presents a new framework that addresses the problem using a two-channel registration algorithm capable of using mono-modal similarity measures such as sum of squared differences or cross-correlation. To make it possible to use these same-modality measures, image synthesis is used to create proxy images for the opposite modality as well as intensity-normalized images from each of the two available images. The new deformable registration framework was evaluated by performing intra-subject deformation recovery, intra-subject boundary alignment, and inter-subject label transfer experiments using multi-contrast magnetic resonance brain imaging data. Three different multi-channel registration algorithms were evaluated, revealing that the framework is robust to the multi-channel deformable registration algorithm that is used. With a single exception, all results demonstrated improvements when compared against single channel registrations using the same algorithm with mutual information. [ABSTRACT FROM AUTHOR]

Automatic processing of magnetic resonance images is a vital part of neuroscience research. Yet even the best and most widely used medical image processing methods will not produce consistent results when their input images are acquired with different pulse sequences. Although intensity standardization and image synthesis methods have been introduced to address this problem, their performance remains dependent on knowledge and consistency of the pulse sequences used to acquire the images. In this paper, an image synthesis approach that first estimates the pulse sequence parameters of the subject image is presented. The estimated parameters are then used with a collection of atlas or training images to generate a new atlas image having the same contrast as the subject image. This additional image provides an ideal source from which to synthesize any other target pulse sequence image contained in the atlas. In particular, a nonlinear regression intensity mapping is trained from the new atlas image to the target atlas image and then applied to the subject image to yield the particular target pulse sequence within the atlas. Both intensity standardization and synthesis of missing tissue contrasts can be achieved using this framework. The approach was evaluated on both simulated and real data, and shown to be superior in both intensity standardization and synthesis to other established methods. [ABSTRACT FROM AUTHOR]

Purpose: Accurate tissue motion tracking within the tongue can help professionals diagnose and treat vocal tract-related disorders, evaluate speech quality before and after surgery, and conduct various scientific studies. The authors compared tissue tracking results from 4 widely used deformable registration (DR) methods applied to cine magnetic resonance imaging (MRI) with harmonic phase (HARP)-based tracking applied to tagged MRI. Method: Ten subjects repeated the phrase "a geese" multiple times while sagittal images of the head were collected at 26 Hz, first in a tagged MRI data set and then in a cine MRI data set. HARP tracked the motion of 8 specified tissue points in the tagged data set. Four DR methods including diffeomorphic demons and free-form deformations based on cubic B-spline with 3 different similarity measures were used to track the same 8 points in the cine MRI data set. Individual points were tracked and length changes of several muscles were calculated using the DR- and HARP-based tracking methods. Results: The results showed that the DR tracking errors were nonsystematic and varied in direction, amount, and timing across speakers and within speakers. Comparison of HARP and DR tracking with manual tracking showed better tracking results for HARP except at the tongue surface, where mistracking caused greater errors in HARP than DR. Conclusions: Tissue point tracking using DR tracking methods contains nonsystematic tracking errors within and across subjects, making it less successful than tagged MRI tracking within the tongue. However, HARP sometimes mistracks points at the tongue surface of tagged MRI because of its limited bandpass filter and tag pattern fading, so that DR has better success measuring surface tissue points on cine MRI than HARP does. Therefore, a hybrid method is being explored. [ABSTRACT FROM AUTHOR]

Abstract: Diffusion tensor imaging (DTI) is widely used to characterize tissue micro-architecture and brain connectivity. In regions of crossing fibers, however, the tensor model fails because it cannot represent multiple, independent intra-voxel orientations. Most of the methods that have been proposed to resolve this problem require diffusion magnetic resonance imaging (MRI) data that comprise large numbers of angles and high b-values, making them problematic for routine clinical imaging and many scientific studies. We present a technique based on compressed sensing that can resolve crossing fibers using diffusion MRI data that can be rapidly and routinely acquired in the clinic (30 directions, b-value equal to 700s/mm2). The method assumes that the observed data can be well fit using a sparse linear combination of tensors taken from a fixed collection of possible tensors each having a different orientation. A fast algorithm for computing the best orientations based on a hierarchical compressed sensing algorithm and a novel metric for comparing estimated orientations are also proposed. The performance of this approach is demonstrated using both simulations and in vivo images. The method is observed to resolve crossing fibers using conventional data as well as a standard q-ball approach using much richer data that requires considerably more image acquisition time. [Copyright &y& Elsevier]

Image labeling and parcellation (i.e., assigning structure to a collection of voxels) are critical tasks for the assessment of volumetric and morphometric features in medical imaging data. The process of image labeling is inherently error prone as images are corrupted by noise and artifacts. Even expert interpretations are subject to subjectivity and the precision of the individual raters. Hence, all labels must be considered imperfect with some degree of inherent variability. One may seek multiple independent assessments to both reduce this variability and quantify the degree of uncertainty. Existing techniques have exploited maximum a posteriori statistics to combine data from multiple raters and simultaneously estimate rater reliabilities. Although quite successful, wide-scale application has been hampered by unstable estimation with practical datasets, for example, with label sets with small or thin objects to be labeled or with partial or limited datasets. As well, these approaches have required each rater to generate a complete dataset, which is often impossible given both human foibles and the typical turnover rate of raters in a research or clinical environment. Herein, we propose a robust approach to improve estimation performance with small anatomical structures, allow for missing data, account for repeated label sets, and utilize training/catch trial data. With this approach, numerous raters can label small, overlapping portions of a large dataset, and rater heterogeneity can be robustly controlled while simultaneously estimating a single, reliable label set and characterizing uncertainty. The proposed approach enables many individuals to collaborate in the construction of large datasets for labeling tasks (e.g., human parallel processing) and reduces the otherwise detrimental impact of rater unavailability [ABSTRACT FROM PUBLISHER]

Purpose: Design and optimization of medical imaging systems benefit from accurate theoretical modeling that identifies the physical factors governing image quality, particularly in the early stages of system development. This work extends Fourier metrics of imaging performance and detectability index (d′) to tomosynthesis and cone-beam CT (CBCT) and investigates the extent to which d′ is a valid descriptor of task-based imaging performance as assessed by human observers. Methods: The detectability index for tasks presented in 2D slices (dslice′) was derived from 3D cascaded systems analysis of tomosynthesis and CBCT. Anatomical background noise measured in a physical phantom presenting power-law spectral density was incorporated in the 'generalized' noise-equivalent quanta. Theoretical calculations of dslice′ were performed as a function of total angular extent (θtot) of source-detector orbit ranging 10°-360° under two acquisition schemes: (i) Constant angular separation between projections (constant-Δθ), giving variable number of projections (Nproj) and dose vs θtot and (ii) constant number of projections (constant-Nproj), giving constant dose (but variable angular sampling) with θtot. Five simple observer models were investigated: Prewhitening (PW), prewhitening with eye filter and internal noise (PWEi), nonprewhitening (NPW), nonprewhitening with eye filter (NPWE), and nonprewhitening with eye filter and internal noise (NPWEi). Human observer performance was measured in 9AFC tests for five simple imaging tasks presented within uniform and power-law clutter backgrounds. Measurements (from 9AFC tests) and theoretical calculations (from cascaded systems analysis of dslice′) were compared in terms of area under the ROC curve (Az) Results: Reasonable correspondence between theoretical calculations and human observer performance was achieved for all imaging tasks over the broad range of experimental conditions and acquisition schemes. The PW and PWEi observer models tended to overestimate detectability, while the various NPW models predicted observer performance fairly well, with NPWEi giving the best overall agreement. Detectability was shown to increase with θtot due to the reduction of out-of-plane clutter, reaching a plateau after a particular θtot that depended on the imaging task. Depending on the acquisition scheme, however (i.e., constant-Nproj or Δθ), detectability was seen in some cases to decline at higher θtot due to tradeoffs among quantum noise, background clutter, and view sampling. Conclusions: Generalized detectability index derived from a 3D cascaded systems model shows reasonable correspondence with human observer performance over a fairly broad range of imaging tasks and conditions, although discrepancies were observed in cases relating to orbits intermediate to 180° and 360°. The basic correspondence of theoretical and measured performance supports the application of such a theoretical framework for system design and optimization of tomosynthesis and CBCT. [ABSTRACT FROM AUTHOR]

The success of prostate brachytherapy critically depends on delivering adequate dose to the prostate gland. Intraoperative localization of the implanted seeds provides potential for dose evaluation and optimization during therapy. A reduced-dimensionality matching algorithm for prostate brachytherapy seed reconstruction (REDMAPS) that uses multiple X-ray fluoroscopy images obtained from different poses is proposed. The seed reconstruction problem is formulated as a combinatorial optimization problem, and REDMAPS finds a solution in a clinically acceptable amount of time using dimensionality reduction to create a smaller space of possible solutions. Dimensionality reduction is possible since the optimal solution has approximately zero cost when the poses of the acquired images are known to be within a small error. REDMAPS is also formulated to address the “hidden seed problem” in which seeds overlap on one or more observed images. REDMAPS uses a pruning algorithm to avoid unnecessary computation of cost metrics and the reduced problem is solved using linear programming. REDMAPS was first evaluated and its parameters tuned using simulations. It was then validated using five phantom and 21 patient datasets. REDMAPS was successful in reconstructing the seeds with an overall seed matching rate above 99% and a reconstruction error below 1 mm in less than 5 s. [ABSTRACT FROM AUTHOR]

• A novel formulation for domain adaptation without retraining the model nor using source training data. • Quick test-time adaptation to a single test subject at inference stage. • Applicable to both medical image synthesis and segmentation. [Display omitted] Deep neural networks have been successfully applied to medical image analysis tasks like segmentation and synthesis. However, even if a network is trained on a large dataset from the source domain, its performance on unseen test domains is not guaranteed. The performance drop on data obtained differently from the network's training data is a major problem (known as domain shift) in deploying deep learning in clinical practice. Existing work focuses on retraining the model with data from the test domain, or harmonizing the test domain's data to the network training data. A common practice is to distribute a carefully-trained model to multiple users (e.g., clinical centers), and then each user uses the model to process their own data, which may have a domain shift (e.g., varying imaging parameters and machines). However, the lack of availability of the source training data and the cost of training a new model often prevents the use of known methods to solve user-specific domain shifts. Here, we ask whether we can design a model that, once distributed to users, can quickly adapt itself to each new site without expensive retraining or access to the source training data? In this paper, we propose a model that can adapt based on a single test subject during inference. The model consists of three parts, which are all neural networks: a task model (T) which performs the image analysis task like segmentation; a set of autoencoders (AEs); and a set of adaptors (As). The task model and autoencoders are trained on the source dataset and can be computationally expensive. In the deployment stage, the adaptors are trained to transform the test image and its features to minimize the domain shift as measured by the autoencoders' reconstruction loss. Only the adaptors are optimized during the testing stage with a single test subject thus is computationally efficient. The method was validated on both retinal optical coherence tomography (OCT) image segmentation and magnetic resonance imaging (MRI) T1-weighted to T2-weighted image synthesis. Our method, with its short optimization time for the adaptors (10 iterations on a single test subject) and its additional required disk space for the autoencoders (around 15 MB), can achieve significant performance improvement. Our code is publicly available at: https://github.com/YufanHe/self-domain-adapted-network. [ABSTRACT FROM AUTHOR]