Search

Huber regression plays a critical role in robust learning owing to its robustness to outliers and heavy-tailed noises. In this paper, we investigate the theoretical foundation of an Ivanov regularized Huber regression within a Reproducing Kernel Hilbert Space (RKHS) framework. By imposing a weak moment condition on the conditional distribution, we show that Huber regression estimator with the scale parameter adapted to sample size can learn the underlying regression function very well. Specifically, we establish the explicit convergence rates of the prediction error even when the regression function is outside the associated RKHS. Our statistical analysis could greatly improve the understanding of the learning behavior of Huber regression in the absence of light-tailed assumption on the conditional distribution. [ABSTRACT FROM AUTHOR]

To cope with the challenges of memory bottleneck and algorithmic scalability when massive data sets are involved, we propose a distributed least squares procedure in the framework of functional linear model and reproducing kernel Hilbert space. This approach divides the big data set into multiple subsets, applies regularized least squares regression on each of them, and then averages the individual outputs as a final prediction. We establish the non-asymptotic prediction error bounds for the proposed learning strategy under some regularity conditions. When the target function only has weak regularity, we also introduce some unlabelled data to construct a semi-supervised approach to enlarge the number of the partitioned subsets. Results in present paper provide a theoretical guarantee that the distributed algorithm can achieve the optimal rate of convergence while allowing the whole data set to be partitioned into a large number of subsets for parallel processing. [ABSTRACT FROM AUTHOR]

We study a class of spectral learning methods with dependent observations, including popular ridge regression, Landweber iteration, spectral cut-off and so on. We derive an explicit risk bound in terms of the correlation of the observations, regularity of the regression function, and effective dimension of the reproducing kernel Hilbert space. By appropriately choosing regularization parameter according to the sample size, the risk bound yields a nearly optimal learning rate with a logarithmic term for strongly mixing sequences. We thus extend the applicable range of spectral algorithm to non-i.i.d. sampling process. Particularly, it is shown that the learning rates for i.i.d. samples in the literature refer to our special case, i.e., the mixing condition parameter tends to zero. [ABSTRACT FROM AUTHOR]

Uniform boundedness of output variables is a standard assumption in most theoretical analysis of regression algorithms. This standard assumption has recently been weaken to a moment hypothesis in least square regression (LSR) setting. Although there has been a large literature on error analysis for LSR under the moment hypothesis, very little is known about the statistical properties of support vector machines regression with unbounded sampling. In this paper, we fill the gap in the literature. Without any restriction on the boundedness of the output sampling, we establish an ad hoc convergence analysis for support vector machines regression under very mild conditions. [ABSTRACT FROM AUTHOR]