Improved Multi-class Support Vector Machines Using Novel Methods of Model Selection and Feature Extraction

In this paper, to improve the generalization capability of multi-class SVMs, we propose 1 a novel model selection and 2 feature extraction by SVMs. In 1, unlike the conventional model selection in multi-class SVMs, we determine hyper-parameters, which are kernel parameter and margin parameter, for each separating hyper-plane, separately. Namely, for each separating hyper-plane, we estimate the generalization capability and select optimal values of the hyper-parameters, separately. In 2, we define the weighted vectors of decision functions determined by training multi-class SVMs as the basis vector of the subspace, and we determine the separating hyper-planes in the subspace. Thus, we can determine the new separating hyper-planes during considering the all separating hyper-planes. Using multi-class benchmark data sets, we evaluate the effectiveness of the proposed methods over the conventional method.