Search

Support vector machines (SVMs), which were originally designed for binary classifications, are an excellent tool for machine learning. For the multiclass classifications, they are usually converted into binary ones before they can be used to classify the examples. In the one-against-one algorithm with SVMs, there exists an unclassifiable region where the data samples cannot be classified by its decision function. This paper extends the one-against-one algorithm to handle this problem. We also give the convergence and computational complexity analysis of the proposed method. Finally, one-against-one, fuzzy decision function (FDF), and decision-directed acyclic graph (DDAG) algorithms and our proposed method are compared using five University of California at Irvine (UCI) data sets. The results report that the proposed method can handle the unclassifiable region better than others. [ABSTRACT FROM AUTHOR]

Support vector machines (SVMs), which were originally designed for binary classifications, are an excellent tool for machine learning. For the multiclass classifications, they are usually converted into binary ones before they can be used to classify the examples. In the one-against-one algorithm with SVMs, there exists an unclassifiable region where the data samples cannot be classified by its decision function. This paper extends the one-against-one algorithm to handle this problem. We also give the convergence and computational complexity analysis of the proposed method. Finally, one-against-one, fuzzy decision function (FDF), and decision-directed acyclic graph (DDAG) algorithms and our proposed method are compared using five University of California at Irvine (UCI) data sets. The results report that the proposed method can handle the unclassifiable region better than others.

The support vector machine (SVM) has been demonstrated to be a very effective classifier in many applications, but its performance is still limited as the data distribution information is underutilized in determining the decision hyperplane. Most of the existing kernels employed in nonlinear SVMs measure the similarity between a pair of pattern images based on the Euclidean inner product or the Euclidean distance of corresponding input patterns, which ignores data distribution tendency and makes the SVM essentially a "local" classifier. In this paper, we provide a step toward a paradigm of kernels by incorporating data specific knowledge into existing kernels. We first find the data structure for each class adaptively in the input space via agglomerative hierarchical clustering (AHC), and then construct the weighted Mahalanobis distance (WMD) kernels using the detected data distribution information. In WMD kernels, the similarity between two pattern images is determined not only by the Mahalanobis distance (MD) between their corresponding input patterns but also by the sizes of the clusters they reside in. Although WMD kernels are not guaranteed to be positive definite (pd) or conditionally positive definite (cpd), satisfactory classification results can still be achieved because regularizers in SVMs with WMD kernels are empirically positive in pseudo-Euclidean (pE) spaces. Experimental results on both synthetic and real-world data sets show the effectiveness of "plugging" data structure into existing kernels.

The generalization error bounds found by current error models using the number of effective parameters of a classifier and the number of training samples are usually very loose. These bounds are intended for the entire input space. However, support vector machine (SVM), radial basis function neural network (RBFNN), and multilayer perceptron neural network (MLPNN) are local learning machines for solving problems and treat unseen samples near the training samples to be more important. In this paper, we propose a localized generalization error model which bounds from above the generalization error within a neighborhood of the training samples using stochastic sensitivity measure. It is then used to develop an architecture selection technique for a classifier with maximal coverage of unseen samples by specifying a generalization error threshold. Experiments using 17 University of California at Irvine (UCI) data sets show that, in comparison with cross validation (CV), sequential learning, and two other ad hoc methods, our technique consistently yields the best testing classification accuracy with fewer hidden neurons and less training time.