1. Passive sampling in reproducing kernel Hilbert spaces using leverage scores
- Author
-
Pere Giménez-Febrer, Alba Pagès-Zamora, Ignacio Santamaría, Universitat Politècnica de Catalunya. Departament de Teoria del Senyal i Comunicacions, and Universitat Politècnica de Catalunya. SPCOM - Grup de Recerca de Processament del Senyal i Comunicacions
- Subjects
Vector spaces ,Nyström approximation ,Matemàtiques i estadística::Àlgebra [Àrees temàtiques de la UPC] ,Hilbert algebras ,Passive sampling ,Control and Systems Engineering ,Reproducing kernel Hilbert space ,Signal Processing ,Anàlisi de regressió ,Leverage score ,Kernel ridge regression ,Computer Vision and Pattern Recognition ,Espais vectorials ,Electrical and Electronic Engineering ,Regression analysis ,Software ,Hilbert, Àlgebres de - Abstract
This paper deals with the selection of the training dataset in kernel-based methods for function reconstruction, with a focus on kernel ridge regression. A functional analysis is performed which, in the absence of noise, links the optimal sampling distribution to the one minimizing the difference between the kernel matrix and its low-rank Nyström approximation. From this standpoint, a statistical passive sampling approach is derived which uses the leverage scores of the columns of the kernel matrix to design a sampling distribution that minimizes an upper bound of the risk function. The proposed approach constitutes a passive method, able to select the optimal subset of training samples using only information provided by the input set and the kernel, but without needing to know the values of the function to be approximated. Furthermore, the proposed approach is backed up by numerical tests on real datasets. This work has been funded by the Ministerio de Ciencia e Innovación (MICINN) of the Spanish Government and by the Agencia Estatal de Investigación (AEI/10.13039/501100011033) and ERDF funds (PID 2019-104958RB-C41/C43, RED2018-102668-T); and by the Catalan Government (2017 SGR 578).
- Published
- 2022