1. Fundamental Limits of Weak Recovery with Applications to Phase Retrieval
- Author
-
Marco Mondelli and Andrea Montanari
- Subjects
FOS: Computer and information sciences ,Computer Science - Information Theory ,Information Theory (cs.IT) ,Applied Mathematics ,Gaussian ,Estimator ,Machine Learning (stat.ML) ,010103 numerical & computational mathematics ,Free probability ,01 natural sciences ,Upper and lower bounds ,Combinatorics ,Computational Mathematics ,symbols.namesake ,Computational Theory and Mathematics ,Statistics - Machine Learning ,symbols ,0101 mathematics ,Phase retrieval ,Spectral method ,Random matrix ,Analysis ,Eigenvalues and eigenvectors ,Mathematics - Abstract
In phase retrieval we want to recover an unknown signal $\boldsymbol x\in\mathbb C^d$ from $n$ quadratic measurements of the form $y_i = |\langle{\boldsymbol a}_i,{\boldsymbol x}\rangle|^2+w_i$ where $\boldsymbol a_i\in \mathbb C^d$ are known sensing vectors and $w_i$ is measurement noise. We ask the following weak recovery question: what is the minimum number of measurements $n$ needed to produce an estimator $\hat{\boldsymbol x}(\boldsymbol y)$ that is positively correlated with the signal $\boldsymbol x$? We consider the case of Gaussian vectors $\boldsymbol a_i$. We prove that - in the high-dimensional limit - a sharp phase transition takes place, and we locate the threshold in the regime of vanishingly small noise. For $n\le d-o(d)$ no estimator can do significantly better than random and achieve a strictly positive correlation. For $n\ge d+o(d)$ a simple spectral estimator achieves a positive correlation. Surprisingly, numerical simulations with the same spectral estimator demonstrate promising performance with realistic sensing matrices. Spectral methods are used to initialize non-convex optimization algorithms in phase retrieval, and our approach can boost the performance in this setting as well. Our impossibility result is based on classical information-theory arguments. The spectral algorithm computes the leading eigenvector of a weighted empirical covariance matrix. We obtain a sharp characterization of the spectral properties of this random matrix using tools from free probability and generalizing a recent result by Lu and Li. Both the upper and lower bound generalize beyond phase retrieval to measurements $y_i$ produced according to a generalized linear model. As a byproduct of our analysis, we compare the threshold of the proposed spectral method with that of a message passing algorithm., Comment: 63 pages, 3 figures, presented at COLT'18 and accepted at Foundations of Computational Mathematics
- Published
- 2018
- Full Text
- View/download PDF