Search

This paper deals with Elliptical Wishart distributions - which generalize the Wishart distribution - in the context of signal processing and machine learning. Two algorithms to compute the maximum likelihood estimator (MLE) are proposed: a fixed point algorithm and a Riemannian optimization method based on the derived information geometry of Elliptical Wishart distributions. The existence and uniqueness of the MLE are characterized as well as the convergence of both estimation algorithms. Statistical properties of the MLE are also investigated such as consistency, asymptotic normality and an intrinsic version of Fisher efficiency. On the statistical learning side, novel classification and clustering methods are designed. For the $t$-Wishart distribution, the performance of the MLE and statistical learning algorithms are evaluated on both simulated and real EEG and hyperspectral data, showcasing the interest of our proposed methods.

In this study, we consider the realm of covariance matrices in machine learning, particularly focusing on computing Fr\'echet means on the manifold of symmetric positive definite matrices, commonly referred to as Karcher or geometric means. Such means are leveraged in numerous machine-learning tasks. Relying on advanced statistical tools, we introduce a random matrix theory-based method that estimates Fr\'echet means, which is particularly beneficial when dealing with low sample support and a high number of matrices to average. Our experimental evaluation, involving both synthetic and real-world EEG and hyperspectral datasets, shows that we largely outperform state-of-the-art methods.

This paper deals with the Elliptical Wishart and Inverse Elliptical Wishart distributions, which play a major role when handling covariance matrices. Similarly to multivariate elliptical distributions, these form a large family of covariance distributions, encompassing, e.g., the Wishart or \textit{t}-Wishart ones. Our first major contribution is to derive a stochastic representation for Elliptical Wishart and Inverse Elliptical Wishart matrices. This later enables us to obtain various key statistical properties of Elliptical Wishart and Inverse Elliptical Wishart distributions such as expectations, variances, and Kronecker moments up to any orders. The stochastic representation also allows us to provide an efficient method to generate random matrices from Elliptical Wishart and Inverse Elliptical Wishart distributions. Finally, the practical interest of Elliptical Wishart distributions - in particular the \textit{t}-Wishart one - is demonstrated through a fitting experiment on real electroencephalographic data. This showcases their effectiveness in accurately modeling real covariance matrices.

We develop the information geometry of scaled Gaussian distributions for which the covariance matrix exhibits a Kronecker product structure. This model and its geometry are then used to propose an online change detection (CD) algorithm for multivariate image times series (MITS). The proposed approach relies mainly on the online estimation of the structured covariance matrix under the null hypothesis, which is performed through a recursive (natural) Riemannian gradient descent. This approach exhibits a practical interest compared to the corresponding offline version, as its computational cost remains constant for each new image added in the time series. Simulations show that the proposed recursive estimators reach the Intrinsic Cram\'er-Rao bound. The interest of the proposed online CD approach is demonstrated on both simulated and real data.

This paper presents a new performance bound for estimation problems where the parameter to estimate lies in a Riemannian manifold (a smooth manifold endowed with a Riemannian metric) and follows a given prior distribution. In this setup, the chosen Riemannian metric induces a geometry for the parameter manifold, as well as an intrinsic notion of the estimation error measure. Performance bound for such error measure were previously obtained in the non-Bayesian case (when the unknown parameter is assumed to deterministic), and referred to as \textit{intrinsic} Cram\'er-Rao bound. The presented result then appears either as: \textit{a}) an extension of the intrinsic Cram\'er-Rao bound to the Bayesian estimation framework; \textit{b}) a generalization of the Van-Trees inequality (Bayesian Cram\'er-Rao bound) that accounts for the aforementioned geometric structures. In a second part, we leverage this formalism to study the problem of covariance matrix estimation when the data follow a Gaussian distribution, and whose covariance matrix is drawn from an inverse Wishart distribution. Performance bounds for this problem are obtained for both the mean squared error (Euclidean metric) and the natural Riemannian distance for Hermitian positive definite matrices (affine invariant metric). Numerical simulation illustrate that assessing the error with the affine invariant metric is revealing of interesting properties of the maximum a posteriori and minimum mean square error estimator, which are not observed when using the Euclidean metric.

When dealing with a parametric statistical model, a Riemannian manifold can naturally appear by endowing the parameter space with the Fisher information metric. The geometry induced on the parameters by this metric is then referred to as the Fisher-Rao information geometry. Interestingly, this yields a point of view that allows for leveragingmany tools from differential geometry. After a brief introduction about these concepts, we will present some practical uses of these geometric tools in the framework of elliptical distributions. This second part of the exposition is divided into three main axes: Riemannian optimization for covariance matrix estimation, Intrinsic Cram\'er-Rao bounds, and classification using Riemannian distances.

Graphical models and factor analysis are well-established tools in multivariate statistics. While these models can be both linked to structures exhibited by covariance and precision matrices, they are generally not jointly leveraged within graph learning processes. This paper therefore addresses this issue by proposing a flexible algorithmic framework for graph learning under low-rank structural constraints on the covariance matrix. The problem is expressed as penalized maximum likelihood estimation of an elliptical distribution (a generalization of Gaussian graphical models to possibly heavy-tailed distributions), where the covariance matrix is optionally constrained to be structured as low-rank plus diagonal (low-rank factor model). The resolution of this class of problems is then tackled with Riemannian optimization, where we leverage geometries of positive definite matrices and positive semi-definite matrices of fixed rank that are well suited to elliptical models. Numerical experiments on real-world data sets illustrate the effectiveness of the proposed approach., Comment: 30 pages, 11 figures, 1 table, accepted at ECML PKDD 2023

This paper proposes a strategy to handle missing data for the classification of electroencephalograms using covariance matrices. It relies on the observed-data likelihood within an expectation-maximization algorithm. This approach is compared to two existing state-of-the-art methods: (i) covariance matrices computed with imputed data; (ii) Riemannian averages of partially observed covariance matrix. All approaches are combined with the minimum distance to Riemannian mean classifier and applied to a classification task of two widely known paradigms of brain-computer interfaces. In addition to be applicable for a wider range of missing data scenarios, the proposed strategy generally performs better than other methods on the considered real EEG data.

A new Riemannian geometry for the Compound Gaussian distribution is proposed. In particular, the Fisher information metric is obtained, along with corresponding geodesics and distance function. This new geometry is applied on a change detection problem on Multivariate Image Times Series: a recursive approach based on Riemannian optimization is developed. As shown on simulated data, it allows to reach optimal performance while being computationally more efficient., Comment: 13 pages

This paper proposes an original Riemmanian geometry for low-rank structured elliptical models, i.e., when samples are elliptically distributed with a covariance matrix that has a low-rank plus identity structure. The considered geometry is the one induced by the product of the Stiefel manifold and the manifold of Hermitian positive definite matrices, quotiented by the unitary group. One of the main contribution is to consider an original Riemannian metric, leading to new representations of tangent spaces and geodesics. From this geometry, we derive a new Riemannian optimization framework for robust covariance estimation, which is leveraged to minimize the popular Tyler's cost function on the considered quotient manifold. We also obtain a new divergence function, which is exploited to define a geometrical error measure on the quotient, and the corresponding intrinsic Cram\'er-Rao lower bound is derived. Thanks to the structure of the chosen parametrization, we further consider the subspace estimation error on the Grassmann manifold and provide its intrinsic Cram\'er-Rao lower bound. Our theoretical results are illustrated on some numerical experiments, showing the interest of the proposed optimization framework and that performance bounds can be reached.

Relying on recent advances in statistical estimation of covariance distances based on random matrix theory, this article proposes an improved covariance and precision matrix estimation for a wide family of metrics. The method is shown to largely outperform the sample covariance matrix estimate and to compete with state-of-the-art methods, while at the same time being computationally simpler. Applications to linear and quadratic discriminant analyses also demonstrate significant gains, therefore suggesting practical interest to statistical machine learning.