Search

Diffusion models (DPMs) have demonstrated remarkable performance in image generation, often times outperforming other generative models. Since their introduction, the powerful noise-to-image denoising pipeline has been extended to various discriminative tasks, including image segmentation. In case of medical imaging, often times the images are large 3D scans, where segmenting one image using DPMs become extremely inefficient due to large memory consumption and time consuming iterative sampling process. In this work, we propose a novel conditional generative modeling framework (LDSeg) that performs diffusion in latent space for medical image segmentation. Our proposed framework leverages the learned inherent low-dimensional latent distribution of the target object shapes and source image embeddings. The conditional diffusion in latent space not only ensures accurate n-D image segmentation for multi-label objects, but also mitigates the major underlying problems of the traditional DPM based segmentation: (1) large memory consumption, (2) time consuming sampling process and (3) unnatural noise injection in forward/reverse process. LDSeg achieved state-of-the-art segmentation accuracy on three medical image datasets with different imaging modalities. Furthermore, we show that our proposed model is significantly more robust to noises, compared to the traditional deterministic segmentation models, which can be potential in solving the domain shift problems in the medical imaging domain. Codes are available at: https://github.com/LDSeg/LDSeg., Comment: 9 pages, 7 figures

Deep learning (DL) has recently emerged as a pivotal technology for enhancing magnetic resonance imaging (MRI), a critical tool in diagnostic radiology. This review paper provides a comprehensive overview of recent advances in DL for MRI reconstruction. It focuses on DL approaches and architectures designed to improve image quality, accelerate scans, and address data-related challenges. These include end-to-end neural networks, pre-trained networks, generative models, and self-supervised methods. The paper also discusses the role of DL in optimizing acquisition protocols, enhancing robustness against distribution shifts, and tackling subtle bias. Drawing on the extensive literature and practical insights, it outlines current successes, limitations, and future directions for leveraging DL in MRI reconstruction, while emphasizing the potential of DL to significantly impact clinical imaging practices.

End-to-End (E2E) unrolled optimization frameworks show promise for Magnetic Resonance (MR) image recovery, but suffer from high memory usage during training. In addition, these deterministic approaches do not offer opportunities for sampling from the posterior distribution. In this paper, we introduce a memory-efficient approach for E2E learning of the posterior distribution. We represent this distribution as the combination of a data-consistency-induced likelihood term and an energy model for the prior, parameterized by a Convolutional Neural Network (CNN). The CNN weights are learned from training data in an E2E fashion using maximum likelihood optimization. The learned model enables the recovery of images from undersampled measurements using the Maximum A Posteriori (MAP) optimization. In addition, the posterior model can be sampled to derive uncertainty maps about the reconstruction. Experiments on parallel MR image reconstruction show that our approach performs comparable to the memory-intensive E2E unrolled algorithm, performs better than its memory-efficient counterpart, and can provide uncertainty maps. Our framework paves the way towards MR image reconstruction in 3D and higher dimensions

Diffusion models have shown impressive performance for image generation, often times outperforming other generative models. Since their introduction, researchers have extended the powerful noise-to-image denoising pipeline to discriminative tasks, including image segmentation. In this work we propose a conditional score-based generative modeling framework for medical image segmentation which relies on a parametric surface representation for the segmentation masks. The surface re-parameterization allows the direct application of standard diffusion theory, as opposed to when the mask is represented as a binary mask. Moreover, we adapted an extended variant of the diffusion technique known as the "cold-diffusion" where the diffusion model can be constructed with deterministic perturbations instead of Gaussian noise, which facilitates significantly faster convergence in the reverse diffusion. We evaluated our method on the segmentation of the left ventricle from 65 transthoracic echocardiogram videos (2230 echo image frames) and compared its performance to the most popular and widely used image segmentation models. Our proposed model not only outperformed the compared methods in terms of segmentation accuracy, but also showed potential in estimating segmentation uncertainties for further downstream analyses due to its inherent generative nature., Comment: 5 pages, 5 figures, conference

The recovery of magnetic resonance (MR) images from undersampled measurements is a key problem that has seen extensive research in recent years. Unrolled approaches, which rely on end-to-end training of convolutional neural network (CNN) blocks within iterative reconstruction algorithms, offer state-of-the-art performance. These algorithms require a large amount of memory during training, making them difficult to employ in high-dimensional applications. Deep equilibrium (DEQ) models and the recent monotone operator learning (MOL) approach were introduced to eliminate the need for unrolling, thus reducing the memory demand during training. Both approaches require a Lipschitz constraint on the network to ensure that the forward and backpropagation iterations converge. Unfortunately, the constraint often results in reduced performance compared to unrolled methods. The main focus of this work is to relax the constraint on the CNN block in two different ways. Inspired by convex-non-convex regularization strategies, we now impose the monotone constraint on the sum of the gradient of the data term and the CNN block, rather than constrain the CNN itself to be a monotone operator. This approach enables the CNN to learn possibly non-monotone score functions, which can translate to improved performance. In addition, we only restrict the operator to be monotone in a local neighborhood around the image manifold. Our theoretical results show that the proposed algorithm is guaranteed to converge to the fixed point and that the solution is robust to input perturbations, provided that it is initialized close to the true solution. Our empirical results show that the relaxed constraints translate to improved performance and that the approach enjoys robustness to input perturbations similar to MOL., Comment: 10 pages, 7 figures

We propose an unsupervised deep learning algorithm for the motion-compensated reconstruction of 5D cardiac MRI data from 3D radial acquisitions. Ungated free-breathing 5D MRI simplifies the scan planning, improves patient comfort, and offers several clinical benefits over breath-held 2D exams, including isotropic spatial resolution and the ability to reslice the data to arbitrary views. However, the current reconstruction algorithms for 5D MRI take very long computational time, and their outcome is greatly dependent on the uniformity of the binning of the acquired data into different physiological phases. The proposed algorithm is a more data-efficient alternative to current motion-resolved reconstructions. This motion-compensated approach models the data in each cardiac/respiratory bin as Fourier samples of the deformed version of a 3D image template. The deformation maps are modeled by a convolutional neural network driven by the physiological phase information. The deformation maps and the template are then jointly estimated from the measured data. The cardiac and respiratory phases are estimated from 1D navigators using an auto-encoder. The proposed algorithm is validated on 5D bSSFP datasets acquired from two subjects., Comment: MICCAI 2023 conference paper

We introduce multi-scale energy models to learn the prior distribution of images, which can be used in inverse problems to derive the Maximum A Posteriori (MAP) estimate and to sample from the posterior distribution. Compared to the traditional single-scale energy models, the multi-scale strategy improves the estimation accuracy and convergence of the MAP algorithm, even when it is initialized far away from the solution. We propose two kinds of multi-scale strategies: a) the explicit (e-MuSE) framework, where we use a sequence of explicit energies, each corresponding to a smooth approximation of the original negative log-prior, and b) the implicit (i-MuSE), where we rely on a single energy function whose gradients at different scales closely match the corresponding e-MuSE gradients. Although both schemes improve convergence and accuracy, the e-MuSE MAP solution depends on the scheduling strategy, including the choice of intermediate scales and exit conditions. In contrast, the i-MuSE formulation is significantly simpler, resulting in faster convergence and improved performance. We compare the performance of the proposed MuSE models in the context of Magnetic Resonance (MR) image recovery. The results demonstrate that the multi-scale framework yields a MAP reconstruction comparable in quality to the End-to-End (E2E) trained models, while being relatively unaffected by the changes in the forward model. In addition, the i-MuSE scheme also allows the generation of samples from the posterior distribution, enabling us to estimate the uncertainty maps.

Purpose: The aim of this work is to introduce a single model-based deep network that can provide high-quality reconstructions from undersampled parallel MRI data acquired with multiple sequences, acquisition settings and field strengths. Methods: A single unrolled architecture, which offers good reconstructions for multiple acquisition settings, is introduced. The proposed scheme adapts the model to each setting by scaling the CNN features and the regularization parameter with appropriate weights. The scaling weights and regularization parameter are derived using a multi-layer perceptron model from conditional vectors, which represents the specific acquisition setting. The perceptron parameters and the CNN weights are jointly trained using data from multiple acquisition settings, including differences in field strengths, acceleration, and contrasts. The conditional network is validated using datasets acquired with different acquisition settings. Results: The comparison of the adaptive framework, which trains a single model using the data from all the settings, shows that it can offer consistently improved performance for each acquisition condition. The comparison of the proposed scheme with networks that are trained independently for each acquisition setting shows that it requires less training data per acquisition setting to offer good performance. Conclusion: The Ada-MoDL framework enables the use of a single model-based unrolled network for multiple acquisition settings. In addition to eliminating the need to train and store multiple networks for different acquisition settings, this approach reduces the training data needed for each acquisition setting.

Model-based deep learning methods that combine imaging physics with learned regularization priors have been emerging as powerful tools for parallel MRI acceleration. The main focus of this paper is to determine the utility of the monotone operator learning (MOL) framework in the parallel MRI setting. The MOL algorithm alternates between a gradient descent step using a monotone convolutional neural network (CNN) and a conjugate gradient algorithm to encourage data consistency. The benefits of this approach include similar guarantees as compressive sensing algorithms including uniqueness, convergence, and stability, while being significantly more memory efficient than unrolled methods. We validate the proposed scheme by comparing it with different unrolled algorithms in the context of accelerated parallel MRI for static and dynamic settings.

Recent quantitative parameter mapping methods including MR fingerprinting (MRF) collect a time series of images that capture the evolution of magnetization. The focus of this work is to introduce a novel approach termed as Deep Factor Model(DFM), which offers an efficient representation of the multi-contrast image time series. The higher efficiency of the representation enables the acquisition of the images in a highly undersampled fashion, which translates to reduced scan time in 3D high-resolution multi-contrast applications. The approach integrates motion estimation and compensation, making the approach robust to subject motion during the scan., Comment: 4 pages, 4 figures

We introduce a novel energy formulation for Plug- and-Play (PnP) image recovery. Traditional PnP methods that use a convolutional neural network (CNN) do not have an energy based formulation. The primary focus of this work is to introduce an energy-based PnP formulation, which relies on a CNN that learns the log of the image prior from training data. The score function is evaluated as the gradient of the energy model, which resembles a UNET with shared encoder and decoder weights. The proposed score function is thus constrained to a conservative vector field, which is the key difference with classical PnP models. The energy-based formulation offers algorithms with convergence guarantees, even when the learned score model is not a contraction. The relaxation of the contraction constraint allows the proposed model to learn more complex priors, thus offering improved performance over traditional PnP schemes. Our experiments in magnetic resonance image reconstruction demonstrates the improved performance offered by the proposed energy model over traditional PnP methods.

This work proposes a self-navigated variable density spiral(VDS) based manifold regularization scheme to prospectively improve dynamic speech MRI at 3T. Short readout 1.3ms spirals were used to minimize off-resonance. A custom 16-channel speech coil was used for improved parallel imaging of vocal tract. The manifold model leveraged similarities between frames sharing similar speech postures without explicit motion binning. The self-navigating capability of VDS was leveraged to learn the Laplacian matrix of the manifold. Reconstruction was posed as a SENSE-based non-local soft weighted temporal regularization scheme. Our approach was compared against view-sharing, low-rank, finite difference, extra-dimension-based sparsity reconstruction constraints. Under-sampling experiments were conducted on five volunteers performing repetitive and arbitrary speaking tasks at different speaking rates. Quantitative evaluation in terms of mean square error over moving edges were performed in a retrospectively under-sampled data. For prospective under-sampling, blinded image quality evaluation in the categories of alias artifacts, spatial blurring, and temporal blurring were performed by three voice research experts. Region of interest analysis at articulator boundaries were performed to assess articulatory motion. Our scheme provided improved reconstruction over the others. With prospective under-sampling, a spatial resolution of 2.4mm2/pixel and a temporal resolution of 17.4 ms/frame for single slice imaging, and 52.2 ms/frame for 3-slice imaging were achieved. We demonstrated implicit motion binning by analyzing the mechanics of the Laplacian matrix. Our method demonstrated superior image quality scores in reducing spatial and temporal blurring. While it exhibited faint alias artifacts similar to temporal finite-difference, it provided statistically significant improvements over remaining constraints., Comment: 32 pages, 10 figures

Computational imaging has been revolutionized by compressed sensing algorithms, which offer guaranteed uniqueness, convergence, and stability properties. Model-based deep learning methods that combine imaging physics with learned regularization priors have emerged as more powerful alternatives for image recovery. The main focus of this paper is to introduce a memory efficient model-based algorithm with similar theoretical guarantees as CS methods. The proposed iterative algorithm alternates between a gradient descent involving the score function and a conjugate gradient algorithm to encourage data consistency. The score function is modeled as a monotone convolutional neural network. Our analysis shows that the monotone constraint is necessary and sufficient to enforce the uniqueness of the fixed point in arbitrary inverse problems. In addition, it also guarantees the convergence to a fixed point, which is robust to input perturbations. We introduce two implementations of the proposed MOL framework, which differ in the way the monotone property is imposed. The first approach enforces a strict monotone constraint, while the second one relies on an approximation. The guarantees are not valid for the second approach in the strict sense. However, our empirical studies show that the convergence and robustness of both approaches are comparable, while the less constrained approximate implementation offers better performance. The proposed deep equilibrium formulation is significantly more memory efficient than unrolled methods, which allows us to apply it to 3D or 2D+time problems that current unrolled algorithms cannot handle.

In this work, we proposed a continuous-acquisition strategy using a gradient echo (GRE) inversion recovery sequence based on spiral trajectories to simultaneously obtain the $T_1$ mapping and CINE imaging. The acquisition is using a free-breathing and ungated fashion. An approach based on variational auto-encoder(VAE) is used for the motion estimation from the centered k-space data. The motion signal is then used to train a deep manifold reconstruction algorithm for image reconstruction. Once the network is trained, we can excite the latent vectors (the estimated motion signals and the contrast signal) in any way as we wanted to generate the image frames in the time series. We can estimate the $T_1$ mapping using the generated image frames where only contrast is varying. We can also generate the breath-hold CINE in different contrast.

We introduce an unsupervised motion-compensated reconstruction scheme for high-resolution free-breathing pulmonary MRI. We model the image frames in the time series as the deformed version of the 3D template image volume. We assume the deformation maps to be points on a smooth manifold in high-dimensional space. Specifically, we model the deformation map at each time instant as the output of a CNN-based generator that has the same weight for all time-frames, driven by a low-dimensional latent vector. The time series of latent vectors account for the dynamics in the dataset, including respiratory motion and bulk motion. The template image volume, the parameters of the generator, and the latent vectors are learned directly from the k-t space data in an unsupervised fashion. Our experimental results show improved reconstructions compared to state-of-the-art methods, especially in the context of bulk motion during the scans.

Model-based deep learning (MoDL) algorithms that rely on unrolling are emerging as powerful tools for image recovery. In this work, we introduce a novel monotone operator learning framework to overcome some of the challenges associated with current unrolled frameworks, including high memory cost, lack of guarantees on robustness to perturbations, and low interpretability. Unlike current unrolled architectures that use finite number of iterations, we use the deep equilibrium (DEQ) framework to iterate the algorithm to convergence and to evaluate the gradient of the convolutional neural network blocks using Jacobian iterations. This approach significantly reduces the memory demand, facilitating the extension of MoDL algorithms to high dimensional problems. We constrain the CNN to be a monotone operator, which allows us to introduce algorithms with guaranteed convergence properties and robustness guarantees. We demonstrate the utility of the proposed scheme in the context of parallel MRI.

Deep learning algorithms that rely on extensive training data are revolutionizing image recovery from ill-posed measurements. Training data is scarce in many imaging applications, including ultra-high-resolution imaging. The deep image prior (DIP) algorithm was introduced for single-shot image recovery, completely eliminating the need for training data. A challenge with this scheme is the need for early stopping to minimize the overfitting of the CNN parameters to the noise in the measurements. We introduce a generalized Stein's unbiased risk estimate (GSURE) loss metric to minimize the overfitting. Our experiments show that the SURE-DIP approach minimizes the overfitting issues, thus offering significantly improved performance over classical DIP schemes. We also use the SURE-DIP approach with model-based unrolling architectures, which offers improved performance over direct inversion schemes.

Free-breathing cardiac MRI schemes are emerging as competitive alternatives to breath-held cine MRI protocols, enabling applicability to pediatric and other population groups that cannot hold their breath. Because the data from the slices are acquired sequentially, the cardiac/respiratory motion patterns may be different for each slice; current free-breathing approaches perform independent recovery of each slice. In addition to not being able to exploit the inter-slice redundancies, manual intervention or sophisticated post-processing methods are needed to align the images post-recovery for quantification. To overcome these challenges, we propose an unsupervised variational deep manifold learning scheme for the joint alignment and reconstruction of multislice dynamic MRI. The proposed scheme jointly learns the parameters of the deep network as well as the latent vectors for each slice, which capture the motion-induced dynamic variations, from the k-t space data of the specific subject. The variational framework minimizes the non-uniqueness in the representation, thus offering improved alignment and reconstructions.

Bilinear models that decompose dynamic data to spatial and temporal factors are powerful and memory-efficient tools for the recovery of dynamic MRI data. These methods rely on sparsity and energy compaction priors on the factors to regularize the recovery. The quality of the recovered images depend on the specific priors. Motivated by deep image prior, we introduce a novel bilinear model whose factors are represented using convolutional neural networks (CNNs). The CNN parameters are learned from the undersampled data off the same subject. To reduce the run time and to improve performance, we initialize the CNN parameters. We use sparsity regularization of the network parameters to minimize the overfitting of the network to measurement noise. Our experiments on free breathing and ungated cardiac cine data acquired using a navigated golden-angle gradient-echo radial sequence show the ability of our method to provide reduced spatial blurring as compared to low-rank and SToRM reconstructions.

The volume estimation of brain regions from MRI data is a key problem in many clinical applications, where the acquisition of data at high spatial resolution is desirable. While parallel MRI and constrained image reconstruction algorithms can accelerate the scans, image reconstruction artifacts are inevitable, especially at high acceleration factors. We introduce a novel image domain deep-learning framework for calibrationless parallel MRI reconstruction, coupled with a segmentation network to improve image quality and to reduce the vulnerability of current segmentation algorithms to image artifacts resulting from acceleration. The combination of the proposed image domain deep calibrationless approach with the segmentation algorithm offers improved image quality, while increasing the accuracy of the segmentations. The novel architecture with an encoder shared between the reconstruction and segmentation tasks is seen to reduce the need for segmented training datasets. In particular, the proposed few-shot training strategy requires only 10% of segmented datasets to offer good performance.

The main focus of this work is a novel framework for the joint reconstruction and segmentation of parallel MRI (PMRI) brain data. We introduce an image domain deep network for calibrationless recovery of undersampled PMRI data. The proposed approach is the deep-learning (DL) based generalization of local low-rank based approaches for uncalibrated PMRI recovery including CLEAR [6]. Since the image domain approach exploits additional annihilation relations compared to k-space based approaches, we expect it to offer improved performance. To minimize segmentation errors resulting from undersampling artifacts, we combined the proposed scheme with a segmentation network and trained it in an end-to-end fashion. In addition to reducing segmentation errors, this approach also offers improved reconstruction performance by reducing overfitting; the reconstructed images exhibit reduced blurring and sharper edges than independently trained reconstruction network.

Deep learning image reconstruction algorithms often suffer from model mismatches when the acquisition scheme differs significantly from the forward model used during training. We introduce a Generalized Stein's Unbiased Risk Estimate (GSURE) loss metric to adapt the network to the measured k-space data and minimize model misfit impact. Unlike current methods that rely on the mean square error in kspace, the proposed metric accounts for noise in the measurements. This makes the approach less vulnerable to overfitting, thus offering improved reconstruction quality compared to schemes that rely on mean-square error. This approach may be useful to rapidly adapt pre-trained models to new acquisition settings (e.g., multi-site) and different contrasts than training data

We introduce a generative smoothness regularization on manifolds (SToRM) model for the recovery of dynamic image data from highly undersampled measurements. The model assumes that the images in the dataset are non-linear mappings of low-dimensional latent vectors. We use the deep convolutional neural network (CNN) to represent the non-linear transformation. The parameters of the generator as well as the low-dimensional latent vectors are jointly estimated only from the undersampled measurements. This approach is different from traditional CNN approaches that require extensive fully sampled training data. We penalize the norm of the gradients of the non-linear mapping to constrain the manifold to be smooth, while temporal gradients of the latent vectors are penalized to obtain a smoothly varying time-series. The proposed scheme brings in the spatial regularization provided by the convolutional network. The main benefit of the proposed scheme is the improvement in image quality and the orders-of-magnitude reduction in memory demand compared to traditional manifold models. To minimize the computational complexity of the algorithm, we introduce an efficient progressive training-in-time approach and an approximate cost function. These approaches speed up the image reconstructions and offers better reconstruction performance.

We introduce a novel generative smoothness regularization on manifolds (SToRM) model for the recovery of dynamic image data from highly undersampled measurements. The proposed generative framework represents the image time series as a smooth non-linear function of low-dimensional latent vectors that capture the cardiac and respiratory phases. The non-linear function is represented using a deep convolutional neural network (CNN). Unlike the popular CNN approaches that require extensive fully-sampled training data that is not available in this setting, the parameters of the CNN generator as well as the latent vectors are jointly estimated from the undersampled measurements using stochastic gradient descent. We penalize the norm of the gradient of the generator to encourage the learning of a smooth surface/manifold, while temporal gradients of the latent vectors are penalized to encourage the time series to be smooth. The main benefits of the proposed scheme are (a) the quite significant reduction in memory demand compared to the analysis based SToRM model, and (b) the spatial regularization brought in by the CNN model. We also introduce efficient progressive approaches to minimize the computational complexity of the algorithm.

Current deep learning-based manifold learning algorithms such as the variational autoencoder (VAE) require fully sampled data to learn the probability density of real-world datasets. Once learned, the density can be used for a variety of tasks, including data imputation. However, fully sampled data is often unavailable in a variety of problems, including the recovery of dynamic and high-resolution MRI data considered in this work. To overcome this problem, we introduce a novel variational approach to learn a manifold from undersampled data. The VAE uses a decoder fed by latent vectors, drawn from a conditional density estimated from the fully sampled images using an encoder. Since fully sampled images are not available in our setting, we approximate the conditional density of the latent vectors by a parametric model whose parameters are estimated from the undersampled measurements using back-propagation. We use the framework for the joint alignment and recovery of multislice free breathing and ungated cardiac MRI data from highly undersampled measurements. Most of the current self-gating and manifold cardiac MRI approaches consider the independent recovery of images from each slice; these methods are not capable of exploiting the inter-slice redundancies in the datasets and require sophisticated post-processing or manual approaches to align the images from different slices. By contrast, the proposed scheme is able to align the multislice data and exploit the redundancies. Experimental results demonstrate the utility of the proposed scheme in dynamic imaging alignment and reconstructions.

Image reconstruction using deep learning algorithms offers improved reconstruction quality and lower reconstruction time than classical compressed sensing and model-based algorithms. Unfortunately, clean and fully sampled ground-truth data to train the deep networks is often unavailable in several applications, restricting the applicability of the above methods. We introduce a novel metric termed the ENsemble Stein's Unbiased Risk Estimate (ENSURE) framework, which can be used to train deep image reconstruction algorithms without fully sampled and noise-free images. The proposed framework is the generalization of the classical SURE and GSURE formulation to the setting where the images are sampled by different measurement operators, chosen randomly from a set. We evaluate the expectation of the GSURE loss functions over the sampling patterns to obtain the ENSURE loss function. We show that this loss is an unbiased estimate for the true mean-square error, which offers a better alternative to GSURE, which only offers an unbiased estimate for the projected error. Our experiments show that the networks trained with this loss function can offer reconstructions comparable to the supervised setting. While we demonstrate this framework in the context of MR image recovery, the ENSURE framework is generally applicable to arbitrary inverse problems.

Several imaging algorithms including patch-based image denoising, image time series recovery, and convolutional neural networks can be thought of as methods that exploit the manifold structure of signals. While the empirical performance of these algorithms is impressive, the understanding of recovery of the signals and functions that live on manifold is less understood. In this paper, we focus on the recovery of signals that live on a union of surfaces. In particular, we consider signals living on a union of smooth band-limited surfaces in high dimensions. We show that an exponential mapping transforms the data to a union of low-dimensional subspaces. Using this relation, we introduce a sampling theoretical framework for the recovery of smooth surfaces from few samples and the learning of functions living on smooth surfaces. The low-rank property of the features is used to determine the number of measurements needed to recover the surface. Moreover, the low-rank property of the features also provides an efficient approach, which resembles a neural network, for the local representation of multidimensional functions on the surface.

We propose a model-based deep learning architecture for the reconstruction of highly accelerated diffusion magnetic resonance imaging (MRI) that enables high resolution imaging. The proposed reconstruction jointly recovers all the diffusion weighted images in a single step from a joint k-q under-sampled acquisition in a parallel MRI setting. We propose the novel use of a pre-trained denoiser as a regularizer in a model-based reconstruction for the recovery of highly under-sampled data. Specifically, we designed the denoiser based on a general diffusion MRI tissue microstructure model for multi-compartmental modeling. By using a wide range of biologically plausible parameter values for the multi-compartmental microstructure model, we simulated diffusion signal that spans the entire microstructure parameter space. A neural network was trained in an unsupervised manner using an autoencoder to learn the diffusion MRI signal subspace. We employed the autoencoder in a model-based reconstruction and show that the autoencoder provides a strong denoising prior to recover the q-space signal. We show reconstruction results on a simulated brain dataset that shows high acceleration capabilities of the proposed method., Comment: 4 pages, 4 figures, and 1 table

Structured low-rank (SLR) algorithms, which exploit annihilation relations between the Fourier samples of a signal resulting from different properties, is a powerful image reconstruction framework in several applications. This scheme relies on low-rank matrix completion to estimate the annihilation relations from the measurements. The main challenge with this strategy is the high computational complexity of matrix completion. We introduce a deep learning (DL) approach to significantly reduce the computational complexity. Specifically, we use a convolutional neural network (CNN)-based filterbank that is trained to estimate the annihilation relations from imperfect (under-sampled and noisy) k-space measurements of Magnetic Resonance Imaging (MRI). The main reason for the computational efficiency is the pre-learning of the parameters of the non-linear CNN from exemplar data, compared to SLR schemes that learn the linear filterbank parameters from the dataset itself. Experimental comparisons show that the proposed scheme can enable calibration-less parallel MRI; it can offer performance similar to SLR schemes while reducing the runtime by around three orders of magnitude. Unlike pre-calibrated and self-calibrated approaches, the proposed uncalibrated approach is insensitive to motion errors and affords higher acceleration. The proposed scheme also incorporates image domain priors that are complementary, thus significantly improving the performance over that of SLR schemes.

We introduce a fast model based deep learning approach for calibrationless parallel MRI reconstruction. The proposed scheme is a non-linear generalization of structured low rank (SLR) methods that self learn linear annihilation filters from the same subject. It pre-learns non-linear annihilation relations in the Fourier domain from exemplar data. The pre-learning strategy significantly reduces the computational complexity, making the proposed scheme three orders of magnitude faster than SLR schemes. The proposed framework also allows the use of a complementary spatial domain prior; the hybrid regularization scheme offers improved performance over calibrated image domain MoDL approach. The calibrationless strategy minimizes potential mismatches between calibration data and the main scan, while eliminating the need for a fully sampled calibration region.

We propose a deep self-learning algorithm to learn the manifold structure of free-breathing and ungated cardiac data and to recover the cardiac CINE MRI from highly undersampled measurements. Our method learns the manifold structure in the dynamic data from navigators using autoencoder network. The trained autoencoder is then used as a prior in the image reconstruction framework. We have tested the proposed method on free-breathing and ungated cardiac CINE data, which is acquired using a navigated golden-angle gradient-echo radial sequence. Results show the ability of our method to better capture the manifold structure, thus providing us reduced spatial and temporal blurring as compared to the SToRM reconstruction.

Modern MRI schemes, which rely on compressed sensing or deep learning algorithms to recover MRI data from undersampled multichannel Fourier measurements, are widely used to reduce scan time. The image quality of these approaches is heavily dependent on the sampling pattern. We introduce a continuous strategy to jointly optimize the sampling pattern and network parameters. We use a multichannel forward model, consisting of a non-uniform Fourier transform with continuously defined sampling locations, to realize the data consistency block within a model-based deep learning image reconstruction scheme. This approach facilitates the joint and continuous optimization of the sampling pattern and the CNN parameters to improve image quality. We observe that the joint optimization of the sampling patterns and the reconstruction module significantly improves the performance of most deep learning reconstruction algorithms. The source code of the proposed joint learning framework is available at https://github.com/hkaggarwal/J-MoDL.

In this survey, we provide a detailed review of recent advances in the recovery of continuous domain multidimensional signals from their few non-uniform (multichannel) measurements using structured low-rank matrix completion formulation. This framework is centered on the fundamental duality between the compactness (e.g., sparsity) of the continuous signal and the rank of a structured matrix, whose entries are functions of the signal. This property enables the reformulation of the signal recovery as a low-rank structured matrix completion, which comes with performance guarantees. We will also review fast algorithms that are comparable in complexity to current compressed sensing methods, which enables the application of the framework to large-scale magnetic resonance (MR) recovery problems. The remarkable flexibility of the formulation can be used to exploit signal properties that are difficult to capture by current sparse and low-rank optimization strategies. We demonstrate the utility of the framework in a wide range of MR imaging (MRI) applications, including highly accelerated imaging, calibration-free acquisition, MR artifact correction, and ungated dynamic MRI., Comment: Accepted for IEEE Signal Processing Magazine

Purpose: To introduce a fast and improved direct reconstruction method for multi-shot diffusion weighted (msDW) scans for high-resolution studies. Methods:Multi-shot EPI methods can enable higher spatial resolution for diffusion MRI studies. Traditionally, such acquisitions required specialized reconstructions involving phase compensation to correct for inter-shot motion artifacts. The recently proposed MUSSELS reconstruction belongs to a new class of parallel imaging-based methods that recover artifact-free DWIs from msDW data without needing phase compensation. However, computational demands of the MUSSELS reconstruction scales as the matrix size and the number of shots increases, which hinders its practical utility for high-resolution applications. In this work, we propose a computationally efficient formulation using iterative reweighted least squares (IRLS) method. The new formulation is not only fast but it enables to accommodate additional priors such as conjugate symmetry property of the k-space data to improve the reconstruction. Using whole-brain in-vivo data, we show the utility of the new formulation for routine high-resolution studies with minimal computational burden. Results: The IRLS formulation provides about six times faster reconstruction for matrix sizes 192x192 and 256x256, compared to the original implementations. The reconstruction quality is improved by the addition of conjugate symmetry priors that reduce blurring and preserves the high-resolution details from partial Fourier acquisitions. Conclusion: The proposed method is shown to be computationally efficient to enable routine high-resolution studies. The computational complexity matches the traditional msDWI reconstruction methods and provides improved reconstruction results.

We introduce a sampling theoretic framework for the recovery of smooth surfaces and functions living on smooth surfaces from few samples. The proposed approach can be thought of as a nonlinear generalization of union of subspace models widely used in signal processing. This scheme relies on an exponential lifting of the original data points to feature space, where the features live on union of subspaces. The low-rank property of the features are used to recover the surfaces as well as to determine the number of measurements needed to recover the surface. The low-rank property of the features also provides an efficient approach which resembles a neural network for the local representation of multidimensional functions on the surface; the significantly reduced number of parameters make the computational structure attractive for learning inference from limited labeled training data.

We introduce a kernel low-rank algorithm to recover free-breathing and ungated dynamic MRI from spiral acquisitions without explicit k-space navigators. It is often challenging for low-rank methods to recover free-breathing and ungated images from undersampled measurements; extensive cardiac and respiratory motion often results in the Casorati matrix not being sufficiently low-rank. Therefore, we exploit the non-linear structure of the dynamic data, which gives the low-rank kernel matrix. Unlike prior work that rely on navigators to estimate the manifold structure, we propose a kernel low-rank matrix completion method to directly fill in the missing k-space data from variable density spiral acquisitions. We validate the proposed scheme using simulated data and in-vivo data. Our results show that the proposed scheme provides improved reconstructions compared to the classical methods such as low-rank and XD-GRASP. The comparison with breath-held cine data shows that the quantitative metrics agree, whereas the image quality is marginally lower., Comment: Accepted in IEEE Transactions on Medical Imaging

Purpose: To introduce a novel reconstruction method for simultaneous multi-slice (SMS) accelerated multi shot diffusion weighted imaging (ms-DWI). Methods: SMS acceleration using blipped CAIPI schemes have been proposed to speed up the acquisition of ms-DWIs. The reconstruction of the above data requires (i) phase compensation to combine data from different shots and (ii) slice unfolding to separate the data of different slices. The traditional approach is to first estimate the phase maps corresponding to each shot from a navigator or from the slice-aliased individual shot data. The phase maps are subsequently fed to a iterative reconstruction scheme to recover the slice unfolded DWIs. We propose a novel reconstruction method to jointly recover the slice-unfolded k-space data of the multiple shots. The proposed reconstruction is enabled by the low-rank property inherent in the k-space samples of a ms-DW acquisition. Specifically, we recover the missing samples of the multi-shot acquisition by enforcing a low-rank penalty on the block-Hankel matrix formed by the k-space data of each shot for each slice. The joint recovery of the slice-unfolded k-space data is then performed using a SENSE-based slice-unfolding subject to the low-rank constraint. The proposed joint recovery scheme is tested on simulated and in-vivo data and compared to similar un-navigated methods at slice acceleration factors of 2 and 3. Results: Our experiments show effective slice unaliasing and successful recovery of DWIs with minimal phase artifacts using the proposed method. The performance is comparable to existing methods at low acceleration factors and better than existing methods as the acceleration factor increase. Conclusion: For the slice acceleration factors considered in this study, the proposed method can successfully recover DWIs from SMS-accelerated ms-DWI acquisitions.

We introduce a model based off-the-grid image reconstruction algorithm using deep learned priors. The main difference of the proposed scheme with current deep learning strategies is the learning of non-linear annihilation relations in Fourier space. We rely on a model based framework, which allows us to use a significantly smaller deep network, compared to direct approaches that also learn how to invert the forward model. Preliminary comparisons against image domain MoDL approach demonstrates the potential of the off-the-grid formulation. The main benefit of the proposed scheme compared to structured low-rank methods is the quite significant reduction in computational complexity., Comment: ISBI 2019

We introduce a model-based deep learning architecture termed MoDL-MUSSELS for the correction of phase errors in multishot diffusion-weighted echo-planar MRI images. The proposed algorithm is a generalization of existing MUSSELS algorithm with similar performance but with significantly reduced computational complexity. In this work, we show that an iterative re-weighted least-squares implementation of MUSSELS alternates between a multichannel filter bank and the enforcement of data consistency. The multichannel filter bank projects the data to the signal subspace thus exploiting the phase relations between shots. Due to the high computational complexity of self-learned filter bank, we propose to replace it with a convolutional neural network (CNN) whose parameters are learned from exemplary data. The proposed CNN is a hybrid model involving a multichannel CNN in the k-space and another CNN in the image space. The k-space CNN exploits the phase relations between the shot images, while the image domain network is used to project the data to an image manifold. The experiments show that the proposed scheme can yield reconstructions that are comparable to state of the art methods while offering several orders of magnitude reduction in run-time.

Recent theory of mapping an image into a structured low-rank Toeplitz or Hankel matrix has become an effective method to restore images. In this paper, we introduce a generalized structured low-rank algorithm to recover images from their undersampled Fourier coefficients using infimal convolution regularizations. The image is modeled as the superposition of a piecewise constant component and a piecewise linear component. The Fourier coefficients of each component satisfy an annihilation relation, which results in a structured Toeplitz matrix, respectively. We exploit the low-rank property of the matrices to formulate a combined regularized optimization problem. In order to solve the problem efficiently and to avoid the high memory demand resulting from the large-scale Toeplitz matrices, we introduce a fast and memory efficient algorithm based on the half-circulant approximation of the Toeplitz matrix. We demonstrate our algorithm in the context of single and multi-channel MR images recovery. Numerical experiments indicate that the proposed algorithm provides improved recovery performance over the state-of-the-art approaches.

We introduce a continuous domain framework for the recovery of a planar curve from a few samples. We model the curve as the zero level set of a trigonometric polynomial. We show that the exponential feature maps of the points on the curve lie on a low-dimensional subspace. We show that the null-space vector of the feature matrix can be used to uniquely identify the curve, given a sufficient number of samples. The worst-case theoretical guarantees show that the number of samples required for unique recovery depends on the bandwidth of the underlying trigonometric polynomial, which is a measure of the complexity of the curve. We introduce an iterative algorithm that relies on the low-rank property of the feature maps to recover the curves when the samples are noisy or when the true bandwidth of the curve is unknown. We also demonstrate the preliminary utility of the proposed curve representation in the context of image segmentation.

We introduce a model-based reconstruction framework with deep learned (DL) and smoothness regularization on manifolds (STORM) priors to recover free breathing and ungated (FBU) cardiac MRI from highly undersampled measurements. The DL priors enable us to exploit the local correlations, while the STORM prior enables us to make use of the extensive non-local similarities that are subject dependent. We introduce a novel model-based formulation that allows the seamless integration of deep learning methods with available prior information, which current deep learning algorithms are not capable of. The experimental results demonstrate the preliminary potential of this work in accelerating FBU cardiac MRI.

We introduce an adaptive structured low rank algorithm to recover MR images from their undersampled Fourier coefficients. The image is modeled as a combination of a piecewise constant component and a piecewise linear component. The Fourier coefficients of each component satisfy an annihilation relation, which results in a structured Toeplitz matrix. We exploit the low rank property of the matrices to formulate a combined regularized optimization problem, which can be solved efficiently. Numerical experiments indicate that the proposed algorithm provides improved recovery performance over the previously proposed algorithms.

We introduce a structured low rank algorithm for the calibration-free compensation of field inhomogeneity artifacts in Echo Planar Imaging (EPI) MRI data. We acquire the data using two EPI readouts that differ in echo-time (TE). Using time segmentation, we reformulate the field inhomogeneity compensation problem as the recovery of an image time series from highly undersampled Fourier measurements. The temporal profile at each pixel is modeled as a single exponential, which is exploited to fill in the missing entries. We show that the exponential behavior at each pixel, along with the spatial smoothness of the exponential parameters, can be exploited to derive a 3D annihilation relation in the Fourier domain. This relation translates to a low rank property on a structured multi-fold Toeplitz matrix, whose entries correspond to the measured k-space samples. We introduce a fast two-step algorithm for the completion of the Toeplitz matrix from the available samples. In the first step, we estimate the null space vectors of the Toeplitz matrix using only its fully sampled rows. The null space is then used to estimate the signal subspace, which facilitates the efficient recovery of the time series of images. We finally demonstrate the proposed approach on spherical MR phantom data and human data and show that the artifacts are significantly reduced. The proposed approach could potentially be used to compensate for time varying field map variations in dynamic applications such as functional MRI.

We introduce a novel bandlimited manifold framework and an algorithm to recover freebreathing and ungated cardiac MR images from highly undersampled measurements. The image frames in the free breathing and ungated dataset are assumed to be points on a bandlimited manifold. We introduce a novel kernel low-rank algorithm to estimate the manifold structure (Laplacian) from a navigator-based acquisition scheme. The structure of the manifold is then used to recover the images from highly undersampled measurements. A computationally efficient algorithm, which relies on the bandlimited approximation of the Laplacian matrix, is used to recover the images. The proposed scheme is demonstrated on several patients with different breathing patterns and cardiac rates, without requiring the need for manually tuning the reconstruction parameters in each case. The proposed scheme enabled the recovery of free-breathing and ungated data, providing reconstructions that are qualitatively similar to breath-held scans performed on the same patients. This shows the potential of the technique as a clinical protocol for free-breathing cardiac scans.

Purpose: To reconstruct artifact-free images from measured k-space data, when the actual k-space trajectory deviates from the nominal trajectory due to gradient imperfections. Methods: Trajectory errors arising from eddy currents and gradient delays introduce phase inconsistencies in several fast scanning MR pulse sequences, resulting in image artifacts. The proposed algorithm provides a novel framework to compensate for this phase distortion. The algorithm relies on the construction of a multi-block Hankel matrix, where each block is constructed from k-space segments with the same phase distortion. In the presence of spatially smooth phase distortions between the segments, the complete block-Hankel matrix is known to be highly low-rank. Since each k-space segment is only acquiring part of the k-space data, the reconstruction of the phase compensated image from their partially parallel measurements is posed as a structured low-rank matrix optimization problem, assuming the coil sensitivities to be known. Results: The proposed formulation is tested on radial acquisitions in several settings including partial Fourier and golden-angle acquisitions. The experiments demonstrate the ability of the algorithm to successfully remove the artifacts arising from the trajectory errors, without the need for trajectory or phase calibration. The quality of the reconstruction was comparable to corrections achieved using the Trajectory Auto-Corrected Image Reconstruction (TrACR) for radial acquisitions. Conclusion: The proposed method provides a general framework for the recovery of artifact-free images from radial trajectories without the need for trajectory calibration., Comment: References fixed

The analysis of large datasets is often complicated by the presence of missing entries, mainly because most of the current machine learning algorithms are designed to work with full data. The main focus of this work is to introduce a clustering algorithm, that will provide good clustering even in the presence of missing data. The proposed technique solves an $\ell_0$ fusion penalty based optimization problem to recover the clusters. We theoretically analyze the conditions needed for the successful recovery of the clusters. We also propose an algorithm to solve a relaxation of this problem using saturating non-convex fusion penalties. The method is demonstrated on simulated and real datasets, and is observed to perform well in the presence of large fractions of missing entries., Comment: arXiv admin note: substantial text overlap with arXiv:1709.01870

We introduce a framework for the recovery of points on a smooth surface in high-dimensional space, with application to dynamic imaging. We assume the surface to be the zero-level set of a bandlimited function. We show that the exponential maps of the points on the surface satisfy annihilation relations, implying that they lie in a finite dimensional subspace. We rely on nuclear norm minimization of the maps to recover the points from noisy and undersampled measurements. Since this direct approach suffers from the curse of dimensionality, we introduce an iterative reweighted algorithm that uses the "kernel trick". The resulting algorithm has similarities to iterative algorithms used in graph signal processing (GSP); this framework can be seen as a continuous domain alternative to discrete GSP theory. The use of the algorithm in recovering free breathing and ungated cardiac data shows the potential of this framework in practical applications.

We introduce a continuous domain framework for the recovery of points on a surface in high dimensional space, represented as the zero-level set of a bandlimited function. We show that the exponential maps of the points on the surface satisfy annihilation relations, implying that they lie in a finite dimensional subspace. The subspace properties are used to derive sampling conditions, which will guarantee the perfect recovery of the surface from finite number of points. We rely on nuclear norm minimization to exploit the low-rank structure of the maps to recover the points from noisy measurements. Since the direct estimation of the surface is computationally prohibitive in very high dimensions, we propose an iterative reweighted algorithm using the "kernel trick". The iterative algorithm reveals deep links to Laplacian based algorithms widely used in graph signal processing; the theory and the sampling conditions can serve as a basis for discrete-continuous domain processing of signals on a graph., Comment: arXiv admin note: text overlap with arXiv:1801.00886