Search

Suppose that we are given a time series where consecutive samples are believed to come from a probabilistic source, that the source changes from time to time and that the total number of sources is fixed. Our objective is to estimate the distributions of the sources. A standard approach to this problem is to model the data as a hidden Markov model (HMM). However, since the data often lacks the Markov or the stationarity properties of an HMM, one can ask whether this approach is still suitable or perhaps another approach is required. In this paper, we show that a maximum likelihood HMM estimator can be used to approximate the source distributions in a much larger class of models than HMMs. Specifically, we propose a natural and fairly general non-stationary model of the data, where the only restriction is that the sources do not change too often. Our main result shows that for this model, a maximum-likelihood HMM estimator produces the correct second moment of the data, and the results can be extended to higher moments. [ABSTRACT FROM AUTHOR]

This paper considers the asymptotic properties of the recursive maximum-likelihood estimator for hidden Markov models. The paper is focused on the analytic properties of the asymptotic log-likelihood and on the point-convergence and convergence rate of the recursive maximum-likelihood estimator. Using the principle of analytic continuation, the analyticity of the asymptotic log-likelihood is shown for analytically parameterized hidden Markov models. Relying on this fact and some results from differential geometry (Lojasiewicz inequality), the almost sure point convergence of the recursive maximum-likelihood algorithm is demonstrated, and relatively tight bounds on the convergence rate are derived. As opposed to the existing result on the asymptotic behavior of maximum-likelihood estimation in hidden Markov models, the results of this paper are obtained without assuming that the log-likelihood function has an isolated maximum at which the Hessian is strictly negative definite. [ABSTRACT FROM AUTHOR]

This paper examines a general class of noisy matrix completion tasks, where the goal is to estimate a matrix from observations obtained at a subset of its entries, each of which is subject to random noise or corruption. Our specific focus is on settings where the matrix to be estimated is well-approximated by a product of two (a priori unknown) matrices, one of which is sparse. Such structural models—referred to here as sparse factor models—have been widely used, for example, in subspace clustering applications, as well as in contemporary sparse modeling and dictionary learning tasks. Our main theoretical contributions are estimation error bounds for sparsity-regularized maximum likelihood estimators for the problems of this form, which are applicable to a number of different observation noise or corruption models. Several specific implications are examined, including scenarios where observations are corrupted by additive Gaussian noise or additive heavier-tailed (Laplace) noise, Poisson-distributed observations, and highly quantized (e.g., 1 b) observations. We also propose a simple algorithmic approach based on the alternating direction method of multipliers for these tasks, and provide experimental evidence to support our error analyses. [ABSTRACT FROM AUTHOR]

This paper studies the partial estimation of Gaussian graphical models from high-dimensional empirical observations. We derive a convex formulation for this problem using \ell1-regularized maximum-likelihood estimation, which can be solved via a smoothing approximation algorithm. Statistical estimation performance can be established for our method. The proposed approach has competitive empirical performance compared with existing methods, as demonstrated by various experiments on synthetic and real data sets. [ABSTRACT FROM AUTHOR]

This paper presents the theoretical foundation for time-domain, phase-based estimation of the frequency and phase of a single sinusoid in additive white Gaussian noise (AWGN), analogous to the theoretical foundation provided by Rife and Boorstyn for frequency-domain, Fourier-transform-based estimation. It is shown from the maximum a posteriori probability (MAP) and the maximum likelihood (ML) estimation principles that with the additive observation phase noise (AOPN), due to the AWGN, being described by its a posteriori distribution conditioned on the received signal magnitude, the received signal phase is a sufficient statistic for estimating the single-sinusoid angle parameters. Using a geometric approach, the exact statistical model for the AOPN is derived, where the a posteriori probability density function (pdf) and the corresponding a priori pdf are given by explicit, closed-form expressions that are valid for arbitrary signal-to-noise ratios (SNRs). The a posteriori pdf is Tikhonov, and is of particular interest as it establishes the AOPN model for phase-based frequency/phase MAP/ML estimation in the time domain. It is further illustrated that the results derived can yield various AOPN models as special cases, and the underlying physical insights and interconnections that exist among these models are revealed. It is shown that the model derived by Tretter is an ultimate specialization in the high SNR limit of the AOPN models developed here. For high SNR, the a posteriori Tikhonov pdf can be accurately approximated by a Gaussian distribution, which leads to the best linearized AOPN model. The applications of these AOPN models to the design of linear estimators, including the linear minimum mean square error (LMMSE) estimator, the linear minimum variance estimator, and the LMMSE implementation of the weighted phase averager are presented, and their estimation performances are compared through computer simulations, with the Cramer–Rao lower bound (CRLB) and the Bayesian CRLB as the benchmark. To facilitate estimator design, the a priori statistical models of the frequency and phase are proposed from the information-theoretic perspective, and an improved phase unwrapping algorithm over that given by Fu and Kam is presented. It is shown that by incorporating all the information available in the AOPN, the estimation accuracy can be much improved. [ABSTRACT FROM PUBLISHER]

In this paper, a new structured approach for obtaining uniformly best non-Bayesian biased estimators, which attain minimum-mean-square-error performance at any point in the parameter space, is established. We show that if a uniformly best biased (UBB) estimator exists, then it is unique, and it can be directly obtained from any locally best biased (LBB) estimator. A necessary and sufficient condition for the existence of a UBB estimator is derived. It is shown that if there exists an optimal bias, such that this condition is satisfied, then it is unique, and its closed-form expression is obtained. The proposed approach is exemplified in two nonlinear estimation problems, where uniformly minimum-variance-unbiased estimators do not exist. In the considered examples, we show that the UBB estimators outperform the corresponding maximum-likelihood estimators in the MSE sense. [ABSTRACT FROM AUTHOR]