Search

Segmentation of cardiac magnetic resonance images (MRI) is crucial for the analysis and assessment of cardiac function, helping to diagnose and treat various cardiovascular diseases. Most recent techniques rely on deep learning and usually require an extensive amount of labeled data. To overcome this problem, few-shot learning has the capability of reducing data dependency on labeled data. In this work, we introduce a new method that merges few-shot learning with a U-Net architecture and Gaussian Process Emulators (GPEs), enhancing data integration from a support set for improved performance. GPEs are trained to learn the relation between the support images and the corresponding masks in latent space, facilitating the segmentation of unseen query images given only a small labeled support set at inference. We test our model with the M&Ms-2 public dataset to assess its ability to segment the heart in cardiac magnetic resonance imaging from different orientations, and compare it with state-of-the-art unsupervised and few-shot methods. Our architecture shows higher DICE coefficients compared to these methods, especially in the more challenging setups where the size of the support set is considerably small., Comment: Accepted at Statistical Atlases and Computational Modeling of the Heart (STACOM) Workshop 2024

This paper addresses the problem of uniqueness in learning physical laws for systems of partial differential equations (PDEs). Contrary to most existing approaches, it considers a framework of structured model learning, where existing, approximately correct physical models are augmented with components that are learned from data. The main result of the paper is a uniqueness result that covers a large class of PDEs and a suitable class of neural networks used for approximating the unknown model components. The uniqueness result shows that, in the idealized setting of full, noiseless measurements, a unique identification of the unknown model components is possible as regularization-minimizing solution of the PDE system. Furthermore, the paper provides a convergence result showing that model components learned on the basis of incomplete, noisy measurements approximate the ground truth model component in the limit. These results are possible under specific properties of the approximating neural networks and due to a dedicated choice of regularization. With this, a practical contribution of this analytic paper is to provide a class of model learning frameworks different to standard settings where uniqueness can be expected in the limit of full measurements.

This work focuses on the analysis of fully connected feed forward ReLU neural networks as they approximate a given, smooth function. In contrast to conventionally studied universal approximation properties under increasing architectures, e.g., in terms of width or depth of the networks, we are concerned with the asymptotic growth of the parameters of approximating networks. Such results are of interest, e.g., for error analysis or consistency results for neural network training. The main result of our work is that, for a ReLU architecture with state of the art approximation error, the realizing parameters grow at most polynomially. The obtained rate with respect to a normalized network size is compared to existing results and is shown to be superior in most cases, in particular for high dimensional input.

This review provides an introduction to - and overview of - the current state of the art in neural-network based regularization methods for inverse problems in imaging. It aims to introduce readers with a solid knowledge in applied mathematics and a basic understanding of neural networks to different concepts of applying neural networks for regularizing inverse problems in imaging. Distinguishing features of this review are, among others, an easily accessible introduction to learned generators and learned priors, in particular diffusion models, for inverse problems, and a section focusing explicitly on existing results in function space analysis of neural-network-based approaches in this context.

This paper is concerned with sampling from probability distributions $\pi$ on $\mathbb{R}^d$ admitting a density of the form $\pi(x) \propto e^{-U(x)}$, where $U(x)=F(x)+G(Kx)$ with $K$ being a linear operator and $G$ being non-differentiable. Two different methods are proposed, both employing a subgradient step with respect to $G\circ K$, but, depending on the regularity of $F$, either an explicit or an implicit gradient step with respect to $F$ can be implemented. For both methods, non-asymptotic convergence proofs are provided, with improved convergence results for more regular $F$. Further, numerical experiments are conducted for simple 2D examples, illustrating the convergence rates, and for examples of Bayesian imaging, showing the practical feasibility of the proposed methods for high dimensional data.

This work is concerned with the identifiability of metabolic parameters from multi-region measurement data in quantitative PET imaging. It shows that, for the frequently used two-tissue compartment model and under reasonable assumptions, it is possible to uniquely identify metabolic tissue parameters from standard PET measurements, without the need of additional concentration measurements from blood samples. This result, which holds in the idealized, noiseless scenario, indicates that costly concentration measurements from blood samples in quantitative PET imaging can be avoided in principle. The connection to noisy measurement data is made via a consistency result, showing that exact reconstruction is maintained in the vanishing noise limit. Numerical experiments with a regularization approach are further carried out to support these analytic results in an application example.

In this paper we introduce the class of infinite infimal convolution functionals and apply these functionals to the regularization of ill-posed inverse problems. The proposed regularization involves an infimal convolution of a continuously parametrized family of convex, positively one-homogeneous functionals defined on a common Banach space $X$. We show that, under mild assumptions, this functional admits an equivalent convex lifting in the space of measures with values in $X$. This reformulation allows us to prove well-posedness of a Tikhonov regularized inverse problem and opens the door to a sparse analysis of the solutions. In the case of finite-dimensional measurements we prove a representer theorem, showing that there exists a solution of the inverse problem that is sparse, in the sense that it can be represented as a linear combination of the extremal points of the ball of the lifted infinite infimal convolution functional. Then, we design a generalized conditional gradient method for computing solutions of the inverse problem without relying on an a priori discretization of the parameter space and of the Banach space $X$. The iterates are constructed as linear combinations of the extremal points of the lifted infinite infimal convolution functional. We prove a sublinear rate of convergence for our algorithm and apply it to denoising of signals and images using, as regularizer, infinite infimal convolutions of fractional-Laplacian-type operators with adaptive orders of smoothness and anisotropies.

In this work, a method for obtaining pixel-wise error bounds in Bayesian regularization of inverse imaging problems is introduced. The proposed method employs estimates of the posterior variance together with techniques from conformal prediction in order to obtain coverage guarantees for the error bounds, without making any assumption on the underlying data distribution. It is generally applicable to Bayesian regularization approaches, independent, e.g., of the concrete choice of the prior. Furthermore, the coverage guarantees can also be obtained in case only approximate sampling from the posterior is possible. With this in particular, the proposed framework is able to incorporate any learned prior in a black-box manner. Guaranteed coverage without assumptions on the underlying distributions is only achievable since the magnitude of the error bounds is, in general, unknown in advance. Nevertheless, experiments with multiple regularization approaches presented in the paper confirm that in practice, the obtained error bounds are rather tight. For realizing the numerical experiments, also a novel primal-dual Langevin algorithm for sampling from non-smooth distributions is introduced in this work.

In this work, a method for unsupervised energy disaggregation in private households equipped with smart meters is proposed. This method aims to classify power consumption as active or passive, granting the ability to report on the residents' activity and presence without direct interaction. This lays the foundation for applications like non-intrusive health monitoring of private homes. The proposed method is based on minimizing a suitable energy functional, for which the iPALM (inertial proximal alternating linearized minimization) algorithm is employed, demonstrating that various conditions guaranteeing convergence are satisfied. In order to confirm feasibility of the proposed method, experiments on semi-synthetic test data sets and a comparison to existing, supervised methods are provided., Comment: 9 pages, 2 figures, 3 tables

It is well-known that point sources with sufficient mutual distance can be reconstructed exactly from finitely many Fourier measurements by solving a convex optimization problem with Tikhonov-regularization (this property is sometimes termed superresolution). In case of noisy measurements one can bound the reconstruction error in unbalanced Wasserstein distances or weak Sobolev-type norms. A natural question is to what other settings the phenomenon of superresolution extends. We here keep the same measurement operator, but replace the regularizer to anisotropic total variation, which is particularly suitable for regularizing piecewise constant images with horizontal and vertical edges. Under sufficient mutual distance between the horizontal and vertical edges we prove exact reconstruction results and $L^1$ error bounds in terms of the measurement noise.

Isolating different types of motion in video data is a highly relevant problem in video analysis. Applications can be found, for example, in dynamic medical or biological imaging, where the analysis and further processing of the dynamics of interest is often complicated by additional, unwanted dynamics, such as motion of the measurement subject. In this work, it is empirically shown that a representation of video data via untrained generator networks, together with a specific technique for latent space disentanglement that uses minimal, one-dimensional information on some of the underlying dynamics, allows to efficiently isolate different, highly non-linear motion types. In particular, such a representation allows to freeze any selection of motion types, and to obtain accurate independent representations of other dynamics of interest. Obtaining such a representation does not require any pre-training on a training data set, i.e., all parameters of the generator network are learned directly from a single video.

Objective: Exploit accelerometry data for an automatic, reliable, and prompt detection of spontaneous circulation during cardiac arrest, as this is both vital for patient survival and practically challenging. Methods: We developed a machine learning algorithm to automatically predict the circulatory state during cardiopulmonary resuscitation from 4-second-long snippets of accelerometry and electrocardiogram (ECG) data from pauses of chest compressions of real-world defibrillator records. The algorithm was trained based on 422 cases from the German Resuscitation Registry, for which ground truth labels were created by a manual annotation of physicians. It uses a kernelized Support Vector Machine classifier based on 49 features, which partially reflect the correlation between accelerometry and electrocardiogram data. Results: Evaluating 50 different test-training data splits, the proposed algorithm exhibits a balanced accuracy of 81.2%, a sensitivity of 80.6%, and a specificity of 81.8%, whereas using only ECG leads to a balanced accuracy of 76.5%, a sensitivity of 80.2%, and a specificity of 72.8%. Conclusion: The first method employing accelerometry for pulse/no-pulse decision yields a significant increase in performance compared to single ECG-signal usage. Significance: This shows that accelerometry provides relevant information for pulse/no-pulse decisions. In application, such an algorithm may be used to simplify retrospective annotation for quality management and, moreover, to support clinicians to assess circulatory state during cardiac arrest treatment., Comment: 12 pages, 7 figures

The regularity of images generated by convolutional neural networks, such as the U-net, generative networks, or the deep image prior, is analyzed. In a resolution-independent, infinite dimensional setting, it is shown that such images, represented as functions, are always continuous and, in some circumstances, even continuously differentiable, contradicting the widely accepted modeling of sharp edges in images via jump discontinuities. While such statements require an infinite dimensional setting, the connection to (discretized) neural networks used in practice is made by considering the limit as the resolution approaches infinity. As practical consequence, the results of this paper in particular provide analytical evidence that basic L2 regularization of network weights might lead to over-smoothed outputs.

We introduce and analyze a method of learning-informed parameter identification for partial differential equations (PDEs) in an all-at-once framework. The underlying PDE model is formulated in a rather general setting with three unknowns: physical parameter, state and nonlinearity. Inspired by advances in machine learning, we approximate the nonlinearity via a neural network, whose parameters are learned from measurement data. The later is assumed to be given as noisy observations of the unknown state, and both the state and the physical parameters are identified simultaneously with the parameters of the neural network. Moreover, diverging from the classical approach, the proposed all-at-once setting avoids constructing the parameter-to-state map by explicitly handling the state as additional variable. The practical feasibility of the proposed method is confirmed with experiments using two different algorithmic settings: A function-space algorithm based on analytic adjoints as well as a purely discretized setting using standard machine learning algorithms.

Complete time of flight (TOF) sinograms of state-of-the-art TOF PET scanners have a large memory footprint. Currently, they contain ~4e9 data bins which amount to ~17GB in 32bit floating point precision. Using iterative algorithms to reconstruct such enormous TOF sinograms becomes increasingly challenging due to the memory requirements and the computation time needed to evaluate the forward model for every data bin. This is especially true for more advanced optimization algorithms such as the SPDHG algorithm which allows for the use of non-smooth priors using subsets with guaranteed convergence. SPDHG requires the storage of additional sinograms in memory, which severely limits its application to data sets from state-of-the-art TOF PET systems. Motivated by the generally sparse nature of the TOF sinograms, we propose and analyze a new listmode (LM) extension of the SPDHG algorithm for reconstruction of sparse data following a Poisson distribution. The new algorithm is evaluated based on 2D and 3D simulations, and a real dataset acquired on a recent TOF PET/CT system. The performance of the newly proposed LM SPDHG algorithm is compared against the conventional sinogram SPDHG and the listmode EM-TV algorithm. We show that the speed of convergence of LM-SPDHG is equivalent the original SPDHG using binned data. However, we find that for a TOF PET system with 400ps TOF resolution, LM-SPDHG reduces the required memory from ~56GB to 0.7GB for a short dynamic frame with 1e7 counts and to 12.4GB for a long static acquisition with 5e8 counts. In contrast to SPDHG, the reduced memory requirements of LM-SPDHG enable a pure GPU implementation on state-of-the-art GPUs which will substantially accelerate reconstruction times. This in turn will allow the application of LM-SPDHG in routine clinical practice where short reconstruction times are crucial., Comment: accepted for publication

An important theme in modern inverse problems is the reconstruction of time-dependent data from only finitely many measurements. To obtain satisfactory reconstruction results in this setting it is essential to strongly exploit temporal consistency between the different measurement times. The strongest consistency can be achieved by reconstructing data directly in phase space, the space of positions and velocities. However, this space is usually too high-dimensional for feasible computations. We introduce a novel dimension reduction technique, based on projections of phase space onto lower-dimensional subspaces, which provably circumvents this curse of dimensionality: Indeed, in the exemplary framework of superresolution we prove that known exact reconstruction results stay true after dimension reduction, and we additionally prove new error estimates of reconstructions from noisy data in optimal transport metrics which are of the same quality as one would obtain in the non-dimension-reduced case., Comment: 73 pages, 3 figures

We introduce and study a mathematical framework for a broad class of regularization functionals for ill-posed inverse problems: Regularization Graphs. Regularization graphs allow to construct functionals using as building blocks linear operators and convex functionals, assembled by means of operators that can be seen as generalizations of classical infimal convolution operators. This class of functionals exhaustively covers existing regularization approaches and it is flexible enough to craft new ones in a simple and constructive way. We provide well-posedness and convergence results with the proposed class of functionals in a general setting. Further, we consider a bilevel optimization approach to learn optimal weights for such regularization graphs from training data. We demonstrate that this approach is capable of optimizing the structure and the complexity of a regularization graph, allowing, for example, to automatically select a combination of regularizers that is optimal for given training data.

This paper is concerned with the development, analysis and numerical realization of a novel variational model for the regularization of inverse problems in imaging. The proposed model is inspired by the architecture of generative convolutional neural networks; it aims to generate the unknown from variables in a latent space via multi-layer convolutions and non-linear penalties, and penalizes an associated cost. In contrast to conventional neural-network-based approaches, however, the convolution kernels are learned directly from the measured data such that no training is required. The present work provides a mathematical analysis of the proposed model in a function space setting, including proofs for regularity and existence/stability of solutions, and convergence for vanishing noise. Moreover, in a discretized setting, a numerical algorithm for solving various types of inverse problems with the proposed model is derived. Numerical results are provided for applications in inpainting, denoising, deblurring under noise, super-resolution and JPEG decompression with multiple test images.

Over the last decades, the total variation (TV) evolved to one of the most broadly-used regularisation functionals for inverse problems, in particular for imaging applications. When first introduced as a regulariser, higher-order generalisations of TV were soon proposed and studied with increasing interest, which led to a variety of different approaches being available today. We review several of these approaches, discussing aspects ranging from functional-analytic foundations to regularisation theory for linear inverse problems in Banach space, and provide a unified framework concerning well-posedness and convergence for vanishing noise level for respective Tikhonov regularisation. This includes general higher orders of TV, additive and infimal-convolution multi-order total variation, total generalised variation (TGV), and beyond. Further, numerical optimisation algorithms are developed and discussed that are suitable for solving the Tikhonov minimisation problem for all presented models. Focus is laid in particular on covering the whole pipeline starting at the discretisation of the problem and ending at concrete, implementable iterative procedures. A major part of this review is finally concerned with presenting examples and applications where higher-order TV approaches turned out to be beneficial. These applications range from classical inverse problems in imaging such as denoising, deconvolution, compressed sensing, optical-flow estimation and decompression, to image reconstruction in medical imaging and beyond, including magnetic resonance imaging (MRI), computed tomography (CT), magnetic-resonance positron emission tomography (MR-PET), and electron tomography.

Many methods for processing scalar and vector valued images, volumes and other data in the context of inverse problems are based on variational formulations. Such formulations require appropriate regularization functionals that model expected properties of the object to reconstruct. Prominent examples of regularization functionals in a vector-space context are the total variation (TV) and the Mumford-Shah functional, as well as higher-order schemes such as total generalized variation models. Driven by applications where the signals or data live in nonlinear manifolds, there has been quite some interest in developing analogous methods for nonlinear, manifold-valued data recently. In this chapter, we consider various variational regularization methods for manifold-valued data. In particular, we consider TV minimization as well as higher order models such as total generalized variation (TGV). Also, we discuss (discrete) Mumford-Shah models and related methods for piecewise constant data. We develop discrete energies for denoising and report on algorithmic approaches to minimize them. Further, we also deal with the extension of such methods to incorporate indirect measurement terms, thus addressing the inverse problem setup. Finally, we discuss wavelet sparse regularization for manifold-valued data.

We consider a class of regularization methods for inverse problems where a coupled regularization is employed for the simultaneous reconstruction of data from multiple sources. Applications for such a setting can be found in multi-spectral or multi-modality inverse problems, but also in inverse problems with dynamic data. We consider this setting in a rather general framework and derive stability and convergence results, including convergence rates. In particular, we show how parameter choice strategies adapted to the interplay of different data channels allow to improve upon convergence rates that would be obtained by treating all channels equally. Motivated by concrete applications, our results are obtained under rather general assumptions that allow to include the Kullback-Leibler divergence as data discrepancy term. To simplify their application to concrete settings, we further elaborate several practically relevant special cases in detail. To complement the analytical results, we also provide an algorithmic framework and source code that allows to solve a class of jointly regularized inverse problems with any number of data discrepancies. As concrete applications, we show numerical results for multi-contrast MR and joint MR-PET reconstruction.

In this work, we introduce a function space setting for a wide class of structural/weighted total variation (TV) regularization methods motivated by their applications in inverse problems. In particular, we consider a regularizer that is the appropriate lower semi-continuous envelope (relaxation) of a suitable total variation type functional initially defined for sufficiently smooth functions. We study examples where this relaxation can be expressed explicitly, and we also provide refinements for weighted total variation for a wide range of weights. Since an integral characterization of the relaxation in function space is, in general, not always available, we show that, for a rather general linear inverse problems setting, instead of the classical Tikhonov regularization problem, one can equivalently solve a saddle-point problem where no a priori knowledge of an explicit formulation of the structural TV functional is needed. In particular, motivated by concrete applications, we deduce corresponding results for linear inverse problems with norm and Poisson log-likelihood data discrepancy terms. Finally, we provide proof-of-concept numerical examples where we solve the saddle-point problem for weighted TV denoising as well as for MR guided PET image reconstruction.