On Asymptotic Distribution of Generalized Cross Validation Hyper-parameter Estimator for Regularized System Identification

Asymptotic theory is one of the core subjects in system identification theory and often used to assess properties of model estimators. In this paper, we focus on the asymptotic theory for the kernel-based regularized system identification and study the convergence in distribution of the generalized cross validation (GCV) based hyper-parameter estimator. It is shown that the difference between the GCV based hyper-parameter estimator and the optimal hyper-parameter estimator that minimizes the mean square error scaled by 1/root N converges in distribution to a zero mean Gaussian distribution, where N is the sample size and an expression of covariance matrix is obtained. In particular, for the ridge regression case, a closed-form expression of the variance is obtained and shows the influence of the limit of the regression matrix on the asymptotic distribution. For illustration, Monte Carlo numerical simulations are run to test our theoretical results.
Funding Agencies|Thousand Youth Talents Plan funded by the central government of China - NSFC [61773329]; Shenzhen Science and Technology Innovation Council [Ji-20170189, JCY20170411102101881]; Robotic Discipline Development Fund [20161418]; Shenzhen Government [2014.0003.23]; CUHKSZ; Swedish Research Council [2019-04956]