Passive sampling in reproducing kernel Hilbert spaces using leverage scores

This paper deals with the selection of the training dataset in kernel-based methods for function reconstruction, with a focus on kernel ridge regression. A functional analysis is performed which, in the absence of noise, links the optimal sampling distribution to the one minimizing the difference between the kernel matrix and its low-rank Nyström approximation. From this standpoint, a statistical passive sampling approach is derived which uses the leverage scores of the columns of the kernel matrix to design a sampling distribution that minimizes an upper bound of the risk function. The proposed approach constitutes a passive method, able to select the optimal subset of training samples using only information provided by the input set and the kernel, but without needing to know the values of the function to be approximated. Furthermore, the proposed approach is backed up by numerical tests on real datasets. This work has been funded by the Ministerio de Ciencia e Innovación (MICINN) of the Spanish Government and by the Agencia Estatal de Investigación (AEI/10.13039/501100011033) and ERDF funds (PID 2019-104958RB-C41/C43, RED2018-102668-T); and by the Catalan Government (2017 SGR 578).