Search

Least Squares Regression (LSR) is a powerful machine learning method for image classification and feature selection. In this study, a framework approach is introduced for the multi-classification problem based on the $L_{2,p}$ -norm, utilizing more general loss functions and regularization terms, which is a robust sparse kernel-free quadratic surface least squares regression (RSQSLSR). The nonlinear relationship between features is addressed using a quadratic kernel-free technique combined with $\epsilon $ -dragging technology and manifold regularization to learn soft labels, which can achieve the goal of feature selection and classification, simultaneously. This model utilizes K quadratic surfaces mapping samples from the input space to the label space, preserving the local structure of the samples. To enhance practical applications, such as image classification, a simplified version of the method is proposed. An iterative algorithm for RSQSLSR is designed and its convergence is proved theoretically. The salient features and theoretical analysis of our proposed method are comprehensively discussed in this paper. Extensive experiments on synthetic and real datasets validate the effectiveness of our method, surpassing other state-of-the-art methods in terms of classification accuracy and feature selection performance.

Abstract $$\nu$$ -one-class support vector classification ( $$\nu$$ -OCSVC) has garnered significant attention for its remarkable performance in handling single-class classification and anomaly detection. Nonetheless, the model does not yield a unique decision boundary, and potentially compromises learning performance when the training data is contaminated by some outliers or mislabeled observations. This paper presents a novel C-parameter version of bounded one-class support vector classification (C-BOCSVC) to determine a unique decision boundary. The distance from the origin to decision boundary is the geometrical margin in the space $$\hat{\mathcal {H}}$$ (higher 1-dimension than feature space), and its maximization corresponds to the structural risk minimization (SRM) principle inscribed by an $$L_2$$ -norm regularization term both on the normal direction and bias of the decision boundary. To enhance the anti-noise and anti-outlier abilities of C-BOCSVC, the alternative robust version (C-RBOCSVC) is also developed, which incorporates the k-nearest neighbor relative density to assign varying weights to observations and mitigate the negative impact of outliers on the optimal decision boundary. The theoretical properties of the proposed method are successively derived, including the relationship between the solutions to the primal and dual problems, the connections between our C-BOCSVC and $$\nu$$ -OCSVC and the computational complexity. Experimental results over massive datasets demonstrate the feasibility and reliability of our C-BOCSVC, and highlight the superior performance of C-RBOCSVC compared to other state-of-the-art one-class classifiers when data is contaminated. The demo code of this work is publicly available at https://github.com/Zhangmath1122/C-RBOCSVC .