Search

As a follow-up to the first IEEE Transactions on Medical Imaging (TMI) special issue on the theme of deep tomographic reconstruction, the second special issue is assembled to reflect the status and momentum of this rapidly emerging field. In this editorial, we provide a brief background illustrating the motivation for the development of network-based, data-driven, and learning-oriented reconstruction methods, summarize the included papers, and report our verification of the shared deep learning codes. Finally, we discuss several important research topics to facilitate further investigation and collaboration. [ABSTRACT FROM AUTHOR]

Structured low-rank (SLR) algorithms, which exploit annihilation relations between the Fourier samples of a signal resulting from different properties, is a powerful image reconstruction framework in several applications. This scheme relies on low-rank matrix completion to estimate the annihilation relations from the measurements. The main challenge with this strategy is the high computational complexity of matrix completion. We introduce a deep learning (DL) approach to significantly reduce the computational complexity. Specifically, we use a convolutional neural network (CNN)-based filterbank that is trained to estimate the annihilation relations from imperfect (under-sampled and noisy) k-space measurements of Magnetic Resonance Imaging (MRI). The main reason for the computational efficiency is the pre-learning of the parameters of the non-linear CNN from exemplar data, compared to SLR schemes that learn the linear filterbank parameters from the dataset itself. Experimental comparisons show that the proposed scheme can enable calibration-less parallel MRI; it can offer performance similar to SLR schemes while reducing the runtime by around three orders of magnitude. Unlike pre-calibrated and self-calibrated approaches, the proposed uncalibrated approach is insensitive to motion errors and affords higher acceleration. The proposed scheme also incorporates image domain priors that are complementary, thus significantly improving the performance over that of SLR schemes. [ABSTRACT FROM AUTHOR]

We introduce a model-based deep learning architecture termed MoDL-MUSSELS for the correction of phase errors in multishot diffusion-weighted echo-planar MR images. The proposed algorithm is a generalization of the existing MUSSELS algorithm with similar performance but significantly reduced computational complexity. In this work, we show that an iterative re-weighted least-squares implementation of MUSSELS alternates between a multichannel filter bank and the enforcement of data consistency. The multichannel filter bank projects the data to the signal subspace, thus exploiting the annihilation relations between shots. Due to the high computational complexity of the self-learned filter bank, we propose replacing it with a convolutional neural network (CNN) whose parameters are learned from exemplary data. The proposed CNN is a hybrid model involving a multichannel CNN in the k-space and another CNN in the image space. The k-space CNN exploits the annihilation relations between the shot images, while the image domain network is used to project the data to an image manifold. The experiments show that the proposed scheme can yield reconstructions that are comparable to state-of-the-art methods while offering several orders of magnitude reduction in run-time. [ABSTRACT FROM AUTHOR]

We introduce a local manifold regularization approach to recover dynamic MRI data from highly undersampled measurements. The proposed scheme relies on the manifold structure of local image patches at the same spatial location in a free-breathing cardiac MRI dataset; this approach is a generalization of the SmooThness Regularization on Manifolds (SToRM) scheme that exploits the global manifold structure of images in the dataset. Since the manifold structure of the patches varies depending on the spatial location and is often considerably simpler than the global one, this approach significantly reduces the data demand, facilitating the recovery from shorter scans. Since the navigator-based estimation of manifold structure pursued in SToRM is not feasible in this setting, a reformulation of SToRM is introduced. Specifically, the regularization term of the cost function involves the sum of robust distances between images sub-patches in the dataset. The optimization algorithm alternates between updating the images and estimating the manifold structure of the image patches. The utility of the proposed scheme is demonstrated in the context of $\textit {in-vivo}$ prospective free-breathing cardiac CINE MRI imaging with multichannel acquisitions and simulated phantoms. The new framework facilitates a reduction in scan time, as compared to the SToRM strategy. [ABSTRACT FROM AUTHOR]

Recent theory of mapping an image into a structured low-rank Toeplitz or Hankel matrix has become an effective method to restore images. In this paper, we introduce a generalized structured low-rank algorithm to recover images from their undersampled Fourier coefficients using infimal convolution regularizations. The image is modeled as the superposition of a piecewise constant component and a piecewise linear component. The Fourier coefficients of each component satisfy an annihilation relation, which results in a structured Toeplitz matrix. We exploit the low-rank property of the matrices to formulate a combined regularized optimization problem. In order to solve the problem efficiently and to avoid the high-memory demand resulting from the large-scale Toeplitz matrices, we introduce a fast and a memory-efficient algorithm based on the half-circulant approximation of the Toeplitz matrix. We demonstrate our algorithm in the context of single and multi-channel MR images recovery. Numerical experiments indicate that the proposed algorithm provides improved recovery performance over the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

We introduce a model-based image reconstruction framework with a convolution neural network (CNN)-based regularization prior. The proposed formulation provides a systematic approach for deriving deep architectures for inverse problems with the arbitrary structure. Since the forward model is explicitly accounted for, a smaller network with fewer parameters is sufficient to capture the image information compared to direct inversion approaches. Thus, reducing the demand for training data and training time. Since we rely on end-to-end training with weight sharing across iterations, the CNN weights are customized to the forward model, thus offering improved performance over approaches that rely on pre-trained denoisers. Our experiments show that the decoupling of the number of iterations from the network complexity offered by this approach provides benefits, including lower demand for training data, reduced risk of overfitting, and implementations with significantly reduced memory footprint. We propose to enforce data-consistency by using numerical optimization blocks, such as conjugate gradients algorithm within the network. This approach offers faster convergence per iteration, compared to methods that rely on proximal gradients steps to enforce data consistency. Our experiments show that the faster convergence translates to improved performance, primarily when the available GPU memory restricts the number of iterations. [ABSTRACT FROM AUTHOR]

We introduce a structured low rank matrix completion algorithm to recover a series of images from their under-sampled measurements, where the signal along the parameter dimension at every pixel is described by a linear combination of exponentials. We exploit the exponential behavior of the signal at every pixel, along with the spatial smoothness of the exponential parameters to derive an annihilation relation in the Fourier domain. This relation translates to a low-rank property on a structured matrix constructed from the Fourier samples. We enforce the low-rank property of the structured matrix as a regularization prior to recover the images. Since the direct use of current low rank matrix recovery schemes to this problem is associated with high computational complexity and memory demand, we adopt an iterative re-weighted least squares algorithm, which facilitates the exploitation of the convolutional structure of the matrix. Novel approximations involving 2-D fast Fourier transforms are introduced to drastically reduce the memory demand and computational complexity, which facilitates the extension of structured low-rank methods to large scale 3-D problems. We demonstrate our algorithm in the MR parameter mapping setting and show improvement over the state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

The main focus of this work is to introduce a single free-breathing and ungated imaging protocol to jointly estimate cardiac function and myocardial T 1 maps. We reconstruct a time series of images corresponding to k-space data from a free-breathing and ungated inversion recovery gradient echo sequence using a manifold algorithm. We model each image in the time series as a non-linear function of three variables: cardiac and respiratory phases and inversion time. The non-linear function is realized using a convolutional neural networks (CNN) generator, while the CNN parameters, as well as the phase information, are estimated from the measured k-t space data. We use a dense conditional auto-encoder to estimate the cardiac and respiratory phases from the central multi-channel k-space samples acquired at each frame. The latent vectors of the auto-encoder are constrained to be bandlimited functions with appropriate frequency bands, which enables the disentanglement of the latent vectors into cardiac and respiratory phases, even when the data are acquired with intermittent inversion pulses. Once the phases are estimated, we pose the image recovery as the learning of the parameters of the CNN generator from the measured k-t space data. The learned CNN generator is used to generate synthetic data on demand by feeding it with appropriate latent vectors. The proposed approach capitalizes on the synergies between cine MRI and T 1 mapping to reduce the scan time and improve patient comfort. The framework also enables the generation of synthetic breath-held cine movies with different inversion contrasts, which improves the visualization of the myocardium. In addition, the approach also enables the estimation of the T 1 maps with specific phases, which is challenging with breath-held approaches. [ABSTRACT FROM AUTHOR]

Current deep learning-based manifold learning algorithms such as the variational autoencoder (VAE) require fully sampled data to learn the probability density of real-world datasets. However, fully sampled data is often unavailable in a variety of problems, including the recovery of dynamic and high-resolution magnetic resonance imaging (MRI). We introduce a novel variational approach to learn a manifold from undersampled data. The VAE uses a decoder fed by latent vectors, drawn from a conditional density estimated from the fully sampled images using an encoder. Since fully sampled images are not available in our setting, we approximate the conditional density of the latent vectors by a parametric model whose parameters are estimated from the undersampled measurements using back-propagation. We use the framework for the joint alignment and recovery of multi-slice free breathing and ungated cardiac MRI data from highly undersampled measurements. Experimental results demonstrate the utility of the proposed scheme in dynamic imaging alignment and reconstructions. [ABSTRACT FROM AUTHOR]

Bilinear models such as low-rank and dictionary methods, which decompose dynamic data to spatial and temporal factor matrices are powerful and memory-efficient tools for the recovery of dynamic MRI data. Current bilinear methods rely on sparsity and energy compaction priors on the factor matrices to regularize the recovery. Motivated by deep image prior, we introduce a novel bilinear model, whose factor matrices are generated using convolutional neural networks (CNNs). The CNN parameters, and equivalently the factors, are learned from the undersampled data of the specific subject. Unlike current unrolled deep learning methods that require the storage of all the time frames in the dataset, the proposed approach only requires the storage of the factors or compressed representation; this approach allows the direct use of this scheme to large-scale dynamic applications, including free breathing cardiac MRI considered in this work. To reduce the run time and to improve performance, we initialize the CNN parameters using existing factor methods. We use sparsity regularization of the network parameters to minimize the overfitting of the network to measurement noise. Our experiments on free-breathing and ungated cardiac cine data acquired using a navigated golden-angle gradient-echo radial sequence show the ability of our method to provide reduced spatial blurring as compared to classical bilinear methods as well as a recent unsupervised deep-learning approach. [ABSTRACT FROM AUTHOR]

We propose a novel deformation corrected compressed sensing (DC-CS) framework to recover contrast enhanced dynamic magnetic resonance images from undersampled measurements. We introduce a formulation that is capable of handling a wide class of sparsity/compactness priors on the deformation corrected dynamic signal. In this work, we consider example compactness priors such as sparsity in temporal Fourier domain, sparsity in temporal finite difference domain, and nuclear norm penalty to exploit low rank structure. Using variable splitting, we decouple the complex optimization problem to simpler and well understood sub problems; the resulting algorithm alternates between simple steps of shrinkage-based denoising, deformable registration, and a quadratic optimization step. Additionally, we employ efficient continuation strategies to reduce the risk of convergence to local minima. The decoupling enabled by the proposed scheme enables us to apply this scheme to contrast enhanced MRI applications. Through experiments on numerical phantom and in vivo myocardial perfusion MRI datasets, we observe superior image quality of the proposed DC-CS scheme in comparison to the classical k-t FOCUSS with motion estimation/correction scheme, and demonstrate reduced motion artifacts over classical compressed sensing schemes that utilize the compact priors on the original deformation uncorrected signal. [ABSTRACT FROM AUTHOR]

We introduce a generative smoothness regularization on manifolds (SToRM) model for the recovery of dynamic image data from highly undersampled measurements. The model assumes that the images in the dataset are non-linear mappings of low-dimensional latent vectors. We use the deep convolutional neural network (CNN) to represent the non-linear transformation. The parameters of the generator as well as the low-dimensional latent vectors are jointly estimated only from the undersampled measurements. This approach is different from traditional CNN approaches that require extensive fully sampled training data. We penalize the norm of the gradients of the non-linear mapping to constrain the manifold to be smooth, while temporal gradients of the latent vectors are penalized to obtain a smoothly varying time-series. The proposed scheme brings in the spatial regularization provided by the convolutional network. The main benefit of the proposed scheme is the improvement in image quality and the orders-of-magnitude reduction in memory demand compared to traditional manifold models. To minimize the computational complexity of the algorithm, we introduce an efficient progressive training-in-time approach and an approximate cost function. These approaches speed up the image reconstructions and offers better reconstruction performance. [ABSTRACT FROM AUTHOR]

We introduce a kernel low-rank algorithm to recover free-breathing and ungated dynamic MRI from spiral acquisitions without explicit k-space navigators. It is often challenging for low-rank methods to recover free-breathing and ungated images from undersampled measurements; extensive cardiac and respiratory motion often results in the Casorati matrix not being sufficiently low-rank. Therefore, we exploit the non-linear structure of the dynamic data, which gives the low-rank kernel matrix. Unlike prior work that rely on navigators to estimate the manifold structure, we propose a kernel low-rank matrix completion method to directly fill in the missing k-space data from variable density spiral acquisitions. We validate the proposed scheme using simulated data and in-vivo data. Our results show that the proposed scheme provides improved reconstructions compared to the classical methods such as low-rank and XD-GRASP. The comparison with breath-held cine data shows that the quantitative metrics agree, whereas the image quality is marginally lower. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) restoration is an important preprocessing step in HSI data analysis to improve the image quality for subsequent applications of HSI. In this article, we introduce a spatial–spectral patch-based nonconvex sparsity and low-rank regularization method for HSI restoration. In contrast to traditional approaches based on convex penalties or nonconvex spectral penalty alone, we consider the sparsity of HSI in the spatial–spectral domain and combine the nonconvex low-rank penalty and the nonconvex 3-D total variation (TV)-like sparsity regularization to fully exploit the correlations in both spatial–spectral dimensions of the HSI data set. In addition, we propose a fast iterative variable splitting-based algorithm to effectively solve the corresponding optimization problem. Numerical experiments on both simulated and real HSI data sets demonstrate that the proposed nonconvex low-rank and TV (NonLRTV) method significantly improves the recovered image quality compared with the state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]