Search

We consider probabilistic models for sequential observations which exhibit gradual transitions among a finite number of states. We are particularly motivated by applications such as human activity analysis where observed accelerometer time series contains segments representing distinct activities, which we call pure states, as well as periods characterized by continuous transition among these pure states. To capture this transitory behavior, the dynamical Wasserstein barycenter (DWB) model of Cheng et al. in 2021 [1] associates with each pure state a data-generating distribution and models the continuous transitions among these states as a Wasserstein barycenter of these distributions with dynamically evolving weights. Focusing on the univariate case where Wasserstein distances and barycenters can be computed in closed form, we extend [1] specifically relaxing the parameterization of the pure states as Gaussian distributions. We highlight issues related to the uniqueness in identifying the model parameters as well as uncertainties induced when estimating a dynamically evolving distribution from a limited number of samples. To ameliorate non-uniqueness, we introduce regularization that imposes temporal smoothness on the dynamics of the barycentric weights. A quantile-based approximation of the pure state distributions yields a finite dimensional estimation problem which we numerically solve using cyclic descent alternating between updates to the pure-state quantile functions and the barycentric weights. We demonstrate the utility of the proposed algorithm in segmenting both simulated and real world human activity time series.

We pursue tractable Bayesian analysis of generalized linear models (GLMs) for categorical data. Thus far, GLMs are difficult to scale to more than a few dozen categories due to non-conjugacy or strong posterior dependencies when using conjugate auxiliary variable methods. We define a new class of GLMs for categorical data called categorical-from-binary (CB) models. Each CB model has a likelihood that is bounded by the product of binary likelihoods, suggesting a natural posterior approximation. This approximation makes inference straightforward and fast; using well-known auxiliary variables for probit or logistic regression, the product of binary models admits conjugate closed-form variational inference that is embarrassingly parallel across categories and invariant to category ordering. Moreover, an independent binary model simultaneously approximates multiple CB models. Bayesian model averaging over these can improve the quality of the approximation for any given dataset. We show that our approach scales to thousands of categories, outperforming posterior estimation competitors like Automatic Differentiation Variational Inference (ADVI) and No U-Turn Sampling (NUTS) in the time required to achieve fixed prediction quality., Comment: to appear at ICML 2022

Many time series can be modeled as a sequence of segments representing high-level discrete states, such as running and walking in a human activity application. Flexible models should describe the system state and observations in stationary "pure-state" periods as well as transition periods between adjacent segments, such as a gradual slowdown between running and walking. However, most prior work assumes instantaneous transitions between pure discrete states. We propose a dynamical Wasserstein barycentric (DWB) model that estimates the system state over time as well as the data-generating distributions of pure states in an unsupervised manner. Our model assumes each pure state generates data from a multivariate normal distribution, and characterizes transitions between states via displacement-interpolation specified by the Wasserstein barycenter. The system state is represented by a barycentric weight vector which evolves over time via a random walk on the simplex. Parameter learning leverages the natural Riemannian geometry of Gaussian distributions under the Wasserstein distance, which leads to improved convergence speeds. Experiments on several human activity datasets show that our proposed DWB model accurately learns the generating distribution of pure states while improving state estimation for transition periods compared to the commonly used linear interpolation mixture models., Comment: To appear at Neurips 2021

Here we introduce a new reconstruction technique for two-dimensional Bragg Scattering Tomography (BST), based on the Radon transform models of [arXiv preprint, arXiv:2004.10961 (2020)]. Our method uses a combination of ideas from multibang control and microlocal analysis to construct an objective function which can regularize the BST artifacts; specifically the boundary artifacts due to sharp cutoff in sinogram space (as observed in [arXiv preprint, arXiv:2007.00208 (2020)]), and artifacts arising from approximations made in constructing the model used for inversion. We then test our algorithm in a variety of Monte Carlo (MC) simulated examples of practical interest in airport baggage screening and threat detection. The data used in our studies is generated with a novel Monte-Carlo code presented here. The model, which is available from the authors upon request, captures both the Bragg scatter effects described by BST as well as beam attenuation and Compton scatter., Comment: 26 pages, 17 figures

Non-parametric and distribution-free two-sample tests have been the foundation of many change point detection algorithms. However, randomness in the test statistic as a function of time makes them susceptible to false positives and localization ambiguity. We address these issues by deriving and applying filters matched to the expected temporal signatures of a change for various sliding window, two-sample tests under IID assumptions on the data. These filters are derived asymptotically with respect to the window size for the Wasserstein quantile test, the Wasserstein-1 distance test, Maximum Mean Discrepancy squared (MMD^2), and the Kolmogorov-Smirnov (KS) test. The matched filters are shown to have two important properties. First, they are distribution-free, and thus can be applied without prior knowledge of the underlying data distributions. Second, they are peak-preserving, which allows the filtered signal produced by our methods to maintain expected statistical significance. Through experiments on synthetic data as well as activity recognition benchmarks, we demonstrate the utility of this approach for mitigating false positives and improving the test precision. Our method allows for the localization of change points without the use of ad-hoc post-processing to remove redundant detections common to current methods. We further highlight the performance of statistical tests based on the Quantile-Quantile (Q-Q) function and show how the invariance property of the Q-Q function to order-preserving transformations allows these tests to detect change points of different scales with a single threshold within the same dataset.

Here we introduce a new forward model and imaging modality for Bragg Scattering Tomography (BST). The model we propose is based on an X-ray portal scanner with linear detector collimation, currently being developed for use in airport baggage screening. The geometry under consideration leads us to a novel two-dimensional inverse problem, where we aim to reconstruct the Bragg scattering differential cross section function from its integrals over a set of symmetric $C^2$ curves in the plane. The integral transform which describes the forward problem in BST is a new type of Radon transform, which we introduce and denote as the Bragg transform. We provide new injectivity results for the Bragg transform here, and describe how the conditions of our theorems can be applied to assist in the machine design of the portal scanner. Further we provide an extension of our results to $n$-dimensions, where a generalization of the Bragg transform is introduced. Here we aim to reconstruct a real valued function on $\mathbb{R}^{n+1}$ from its integrals over $n$-dimensional surfaces of revolution of $C^2$ curves embedded in $\mathbb{R}^{n+1}$. Injectivity proofs are provided also for the generalized Bragg transform., Comment: 39 pages, 13 figures

Here we present new joint reconstruction and regularization techniques inspired by ideas in microlocal analysis and lambda tomography, for the simultaneous reconstruction of the attenuation coefficient and electron density from X-ray transmission (i.e., X-ray CT) and backscattered data (assumed to be primarily Compton scattered). To demonstrate our theory and reconstruction methods, we consider the "parallel line segment" acquisition geometry of Webber and Miller ("Compton scattering tomography in translational geometries." Inverse Problems 36, no. 2 (2020): 025007), which is motivated by system architectures currently under development for airport security screening. We first present a novel microlocal analysis of the parallel line geometry which explains the nature of image artefacts when the attenuation coefficient and electron density are reconstructed separately. We next introduce a new joint reconstruction scheme for low effective $Z$ (atomic number) imaging ($Z<20$) characterized by a regularization strategy whose structure is derived from lambda tomography principles and motivated directly by the microlocal analytic results. Finally we show the effectiveness of our method in combating noise and image artefacts on simulated phantoms., Comment: 32 pages, 15 figures

We present a new inner-outer iterative algorithm for edge enhancement in imaging problems. At each outer iteration, we formulate a Tikhonov-regularized problem where the penalization is expressed in the 2-norm and involves a regularization operator designed to improve edge resolution as the outer iterations progress, through an adaptive process. An efficient hybrid regularization method is used to project the Tikhonov-regularized problem onto approximation subspaces of increasing dimensions (inner iterations), while conveniently choosing the regularization parameter (by applying well-known techniques, such as the discrepancy principle or the ${\mathcal L}$-curve criterion, to the projected problem). This procedure results in an automated algorithm for edge recovery that does not involve regularization parameter tuning by the user, nor repeated calls to sophisticated optimization algorithms, and is therefore particularly attractive from a computational point of view. A key to the success of the new algorithm is the design of the regularization operator through the use of an adaptive diagonal weighting matrix that effectively enforces smoothness only where needed. We demonstrate the value of our approach on applications in X-ray CT image reconstruction and in image deblurring, and show that it can be computationally much more attractive than other well-known strategies for edge preservation, while providing solutions of greater or equal quality.

Two common problems in time series analysis are the decomposition of the data stream into disjoint segments that are each in some sense "homogeneous" - a problem known as Change Point Detection (CPD) - and the grouping of similar nonadjacent segments, a problem that we call Time Series Segment Clustering (TSSC). Building upon recent theoretical advances characterizing the limiting distribution-free behavior of the Wasserstein two-sample test (Ramdas et al. 2015), we propose a novel algorithm for unsupervised, distribution-free CPD which is amenable to both offline and online settings. We also introduce a method to mitigate false positives in CPD and address TSSC by using the Wasserstein distance between the detected segments to build an affinity matrix to which we apply spectral clustering. Results on both synthetic and real data sets show the benefits of the approach.

Multi-task/Multi-output learning seeks to exploit correlation among tasks to enhance performance over learning or solving each task independently. In this paper, we investigate this problem in the context of Gaussian Processes (GPs) and propose a new model which learns a mixture of latent processes by decomposing the covariance matrix into a sum of structured hidden components each of which is controlled by a latent GP over input features and a "weight" over tasks. From this sum structure, we propose a parallelizable parameter learning algorithm with a predetermined initialization for the "weights". We also notice that an ensemble parameter learning approach using mini-batches of training data not only reduces the computation complexity of learning but also improves the regression performance. We evaluate our model on two datasets, the smaller Swiss Jura dataset and another relatively larger ATMS dataset from NOAA. Substantial improvements are observed compared with established alternatives., Comment: main body: 9 pages, supplemental material: 7 pages. This version corrected a few typos in previous version

In this paper we demonstrate the utility of fusing energy-resolved observations of Compton scattered photons with traditional attenuation data for the joint recovery of mass density and photoelectric absorption in the context of limited view tomographic imaging applications. We begin with the development of a physical and associated numerical model for the Compton scatter process. Using this model, we propose a variational approach recovering these two material properties. In addition to the typical data-fidelity terms, the optimization functional includes regularization for both the mass density and photoelectric coefficients. We consider a novel edge-preserving method in the case of mass density. To aid in the recovery of the photoelectric information, we draw on our recent method in \cite{r15} and employ a non-local regularization scheme that builds on the fact that mass density is more stably imaged. Simulation results demonstrate clear advantages associated with the use of both scattered photon data and energy resolved information in mapping the two material properties of interest. Specifically, comparing images obtained using only conventional attenuation data with those where we employ only Compton scatter photons and images formed from the combination of the two, shows that taking advantage of both types of data for reconstruction provides far more accurate results.

Recent years have seen growing interest in exploiting dual- and multi-energy measurements in computed tomography (CT) in order to characterize material properties as well as object shape. Material characterization is performed by decomposing the scene into constitutive basis functions, such as Compton scatter and photoelectric absorption functions. While well motivated physically, the joint recovery of the spatial distribution of photoelectric and Compton properties is severely complicated by the fact that the data are several orders of magnitude more sensitive to Compton scatter coefficients than to photoelectric absorption, so small errors in Compton estimates can create large artifacts in the photoelectric estimate. To address these issues, we propose a model-based iterative approach which uses patch-based regularization terms to stabilize inversion of photoelectric coefficients, and solve the resulting problem though use of computationally attractive Alternating Direction Method of Multipliers (ADMM) solution techniques. Using simulations and experimental data acquired on a commercial scanner, we demonstrate that the proposed processing can lead to more stable material property estimates which should aid materials characterization in future dual- and multi-energy CT systems.

The development of energy selective, photon counting X-ray detectors allows for a wide range of new possibilities in the area of computed tomographic image formation. Under the assumption of perfect energy resolution, here we propose a tensor-based iterative algorithm that simultaneously reconstructs the X-ray attenuation distribution for each energy. We use a multi-linear image model rather than a more standard "stacked vector" representation in order to develop novel tensor-based regularizers. Specifically, we model the multi-spectral unknown as a 3-way tensor where the first two dimensions are space and the third dimension is energy. This approach allows for the design of tensor nuclear norm regularizers, which like its two dimensional counterpart, is a convex function of the multi-spectral unknown. The solution to the resulting convex optimization problem is obtained using an alternating direction method of multipliers (ADMM) approach. Simulation results shows that the generalized tensor nuclear norm can be used as a stand alone regularization technique for the energy selective (spectral) computed tomography (CT) problem and when combined with total variation regularization it enhances the regularization capabilities especially at low energy images where the effects of noise are most prominent.

This paper presents several strategies for spectral de-noising of hyperspectral images and hypercube reconstruction from a limited number of tomographic measurements. In particular we show that the non-noisy spectral data, when stacked across the spectral dimension, exhibits low-rank. On the other hand, under the same representation, the spectral noise exhibits a banded structure. Motivated by this we show that the de-noised spectral data and the unknown spectral noise and the respective bands can be simultaneously estimated through the use of a low-rank and simultaneous sparse minimization operation without prior knowledge of the noisy bands. This result is novel for for hyperspectral imaging applications. In addition, we show that imaging for the Computed Tomography Imaging Systems (CTIS) can be improved under limited angle tomography by using low-rank penalization. For both of these cases we exploit the recent results in the theory of low-rank matrix completion using nuclear norm minimization.

This paper presents a new joint inversion approach to shape-based inverse problems. Given two sets of data from distinct physical models, the main objective is to obtain a unified characterization of inclusions within the spatial domain of the physical properties to be reconstructed. Although our proposed method generally applies to many types of inversion problems, the main motivation here is to characterize subsurface contaminant source-zones by processing down gradient hydrological data and cross-gradient electrical resistance tomography (ERT) observations. Inspired by Newton's method for multi-objective optimization, we present an iterative inversion scheme that suggests taking descent steps that can simultaneously reduce both data-model misfit terms. Such an approach, however, requires solving a non-smooth convex problem at every iteration, which is computationally expensive for a pixel-based inversion over the whole domain. Instead, we employ a parametric level set (PaLS) technique that substantially reduces the number of underlying parameters, making the inversion computationally tractable. The performance of the technique is examined and discussed through the reconstruction of source zone architectures that are representative of dense non-aqueous phase liquid (DNAPL) contaminant release in a statistically homogenous sandy aquifer. In these examples, the geometric configuration of the DNAPL mass is considered along with additional information about its spatial variability within the contaminated zone, such as the identification of low and high saturation regions. Comparison of the reconstructions with the true DNAPL architectures highlights the superior performance of the model-based technique and joint inversion scheme.

We present a novel approach for the inverse problem in electrical impedance tomography based on regularized quadratic regression. Our contribution introduces a new formulation for the forward model in the form of a nonlinear integral transform, that maps changes in the electrical properties of a domain to their respective variations in boundary data. Using perturbation theory the transform is approximated to yield a high-order misfit unction which is then used to derive a regularized inverse problem. In particular, we consider the nonlinear problem to second-order accuracy, hence our approximation method improves upon the local linearization of the forward mapping. The inverse problem is approached using Newton's iterative algorithm and results from simulated experiments are presented. With a moderate increase in computational complexity, the method yields superior results compared to those of regularized linear regression and can be implemented to address the nonlinear inverse problem.

Adjoint field methods are both elegant and efficient for calculating sensitivity information required across a wide range of physics-based inverse problems. Here we provide a unified approach to the derivation of such methods for problems whose physics are provided by Poisson's equation. Unlike existing approaches in the literature, we consider in detail and explicitly the role of general boundary conditions in the derivation of the associated adjoint field-based sensitivities. We highlight the relationship between the adjoint field computations required for both gradient decent and Gauss-Newton approaches to image formation. Our derivation is based on standard results from vector calculus coupled with transparent manipulation of the underlying partial different equations thereby making the concepts employed here easily adaptable to other systems of interest.

Dual energy computerized tomography has gained great interest because of its ability to characterize the chemical composition of a material rather than simply providing relative attenuation images as in conventional tomography. The purpose of this paper is to introduce a novel polychromatic dual energy processing algorithm with an emphasis on detection and characterization of piecewise constant objects embedded in an unknown, cluttered background. Physical properties of the objects, specifically the Compton scattering and photoelectric absorption coefficients, are assumed to be known with some level of uncertainty. Our approach is based on a level-set representation of the characteristic function of the object and encompasses a number of regularization techniques for addressing both the prior information we have concerning the physical properties of the object as well as fundamental, physics-based limitations associated with our ability to jointly recover the Compton scattering and photoelectric absorption properties of the scene. In the absence of an object with appropriate physical properties, our approach returns a null characteristic function and thus can be viewed as simultaneously solving the detection and characterization problems. Unlike the vast majority of methods which define the level set function non-parametrically, i.e., as a dense set of pixel values), we define our level set parametrically via radial basis functions (RBF's) and employ a Gauss-Newton type algorithm for cost minimization. Numerical results show that the algorithm successfully detects objects of interest, finds their shape and location, and gives a adequate reconstruction of the background.

The problem studied in this paper is ultrasound image reconstruction from frequency-domain measurements of the scattered field from an object with contrast in attenuation and sound speed. The case where the object has uniform but unknown contrast in these properties relative to the background is considered. Background clutter is taken into account in a physically realistic manner by considering an exact scattering model for randomly located small scatterers that vary in sound speed. The resulting statistical characteristics of the interference is incorporated into the imaging solution, which includes applying a total-variation minimization based approach where the relative effect of perturbation in sound speed to attenuation is included as a parameter. Convex optimization methods provide the basis for the reconstruction algorithm. Numerical data for inversion examples are generated by solving the discretized Lippman-Schwinger equation for the object and speckle-forming scatterers in the background. A statistical model based on the Born approximation is used for reconstruction of the object profile. Results are presented for a two dimensional problem in terms of classification performance and compared to minimum-l2-norm reconstruction. Classification using the proposed method is shown to be robust down to a signal-to-clutter ratio of less than 1 dB., Comment: 25 pages, 10 figures

In this paper, a parametric level set method for reconstruction of obstacles in general inverse problems is considered. General evolution equations for the reconstruction of unknown obstacles are derived in terms of the underlying level set parameters. We show that using the appropriate form of parameterizing the level set function results a significantly lower dimensional problem, which bypasses many difficulties with traditional level set methods, such as regularization, re-initialization and use of signed distance function. Moreover, we show that from a computational point of view, low order representation of the problem paves the path for easier use of Newton and quasi-Newton methods. Specifically for the purposes of this paper, we parameterize the level set function in terms of adaptive compactly supported radial basis functions, which used in the proposed manner provides flexibility in presenting a larger class of shapes with fewer terms. Also they provide a "narrow-banding" advantage which can further reduce the number of active unknowns at each step of the evolution. The performance of the proposed approach is examined in three examples of inverse problems, i.e., electrical resistance tomography, X-ray computed tomography and diffuse optical tomography.