Feature Extraction Using Support Vector Machines

We discuss feature extraction by support vector machines (SVMs). Because the coefficient vector of the hyperplane is orthogonal to the hyperplane, the vector works as a projection vector. To obtain more projection vectors that are orthogonal to the already obtained projection vectors, we train the SVM in the complementary space of the space spanned by the already obtained projection vectors. This is done by modifying the kernel function. We demonstrate the validity of this method using two-class benchmark data sets.