Search

Spin glasses are useless. Even the most imaginative physicists, submitted to grant pressure, could not find applications for these materials. Yet their study, triggered by pure intellectual interest, has created a formidable new branch of statistical physics distinguished this year by the Nobel prize attributed to Giorgio Parisi.

Factorization of matrices where the rank of the two factors diverges linearly with their sizes has many applications in diverse areas such as unsupervised representation learning, dictionary learning or sparse coding. We consider a setting where the two factors are generated from known component-wise independent prior distributions, and the statistician observes a (possibly noisy) component-wise function of their matrix product. In the limit where the dimensions of the matrices tend to infinity, but their ratios remain fixed, we expect to be able to derive closed form expressions for the optimal mean squared error on the estimation of the two factors. However, this remains a very involved mathematical and algorithmic problem. A related, but simpler, problem is extensive-rank matrix denoising, where one aims to reconstruct a matrix with extensive but usually small rank from noisy measurements. In this paper, we approach both these problems using high-temperature expansions at fixed order parameters. This allows to clarify how previous attempts at solving these problems failed at finding an asymptotically exact solution. We provide a systematic way to derive the corrections to these existing approximations, taking into account the structure of correlations particular to the problem. Finally, we illustrate our approach in detail on the case of extensive-rank matrix denoising. We compare our results with known optimal rotationally-invariant estimators, and show how exact asymptotic calculations of the minimal error can be performed using extensive-rank matrix integrals., Journal of Statistical Mechanics: Theory and Experiment, 2022 (8), ISSN:1742-5468

A major challenge of contemporary statistical inference is the large-scale limit, where one wants to discover the values of many hidden parameters, using large amount of data. In recent years, ideas from statistical physics of disordered systems have helped to develop new algorithms for important inference problems, ranging from community detection to compressed sensing, machine learning (notably neural networks), tomography and generalized linear regression. The talk will review these developments and explain how they can be used, to develop new types of algorithms and identify phase transitions.

By now, everybody should know that the recent progress of Artificial Intelligence (AI) is about to have a dramatic impact on many sectors of human activity. In the last ten years, we have seen spectacular breakthroughs on applications of AI, and much more is to come, but should we speak of “intelligence”?