Search

In subspace methods, the subspace associated with a class is represented by a small number of vectors called dictionaries and using the dictionaries the similarity measure is defined and an input is classified into the class with the highest similarity. Usually, each dictionary is given an equal weight. But if subspaces of different classes overlap, the similarity measures for the overlapping regions will not give useful information for classification. In this paper, we propose optimizing the weights for the dictionaries using the idea of support vector machines (SVMs). Namely, first we map the input space into the empirical feature space, perform kernel principal component analysis (KPCA) for each class, and define a similarity measure. Then considering that the similarity measure corresponds to the hyperplane, we formulate the optimization problem as maximizing the margin between the class associated with the dictionaries and the remaining classes. The optimization problem results in all-at-once formulation of linear SVMs. We demonstrate the effectiveness of the proposed method with that of the conventional methods for two-class problems.

In this paper, we discuss subspace-based support vector machines (SS-SVMs), in which an input vector is classified into the class with the maximum similarity. Namely, for each class we define the weighted similarity measure using the vectors called dictionaries that represent the class, and optimize the weights so that the margin between classes is maximized. Because the similarity measure is defined for each class, for a data sample the similarity measure to which the data sample belongs needs to be the largest among all the similarity measures. Introducing slack variables, we define these constraints either by equality constraints or inequality constraints. As a result we obtain subspace-based least squares SVMs (SSLS-SVMs) and subspace-based linear programming SVMs (SSLP-SVMs). To speedup training of SSLS-SVMs, which are similar to LS-SVMs by all-at-once formulation, we also propose SSLS-SVMs by one-against-all formulation, which optimize each similarity measure separately. Using two-class problems, we clarify the difference of SSLS-SVMs and SSLP-SVMs and evaluate the effectiveness of the proposed methods over the conventional methods with equal weights and with weights equal to eigenvalues.

In this paper, we propose two methods for subspace based support vector machines (SS-SVMs) which are subspace based least squares support vector machines (SSLS-SVMs) and subspace based linear programming support vector machines (SSLP-SVMs): 1) optimum selection of the dictionaries of each class subspace from the standpoint of classification separability, and 2) speeding up training SS-SVMs. In method 1), for SSLS-SVMs, we select the dictionaries with optimized weights, and for SSLP-SVMs, we select the dictionaries without non-negative constraints. In method 2), the empirical feature space is obtained by using only the training data belonging to a class instead of using all the training data. Thus the dimension of the empirical feature space and training cost become lower. We demonstrate the effectiveness of the proposed methods over the conventional method for two-class bench mark datasets.