Formulating Ensemble Learning of SVMs Into a Single SVM Formulation by Negative Agreement Learning

When a fixed number of support vector machines (SVMs) are taken as the base learners, an attempt to diversify them should be encouraged to achieve a satisfactory ensemble. In this article, by means of a negative agreement learning (NAL) strategy, a new SVM-based ensemble framework is proposed to simultaneously enhance the diversity of SVMs in the ensemble and suppress the training error of the ensemble. The proposed ensemble framework is theoretically derived to have distinctive merits: 1) the ensemble and each of its individual SVM base learner are trained in a joint manner rather than in an independent manner and 2) the NAL strategy facilitates the formulation of the ensemble of SVMs as one single SVM; thus, abundant advances in the training of SVM can be conveniently applied to the proposed ensemble learning of SVMs and there is no need to design special optimization techniques for the involved ensemble learning. Extensive experimental studies demonstrate the effectiveness of the proposed ensemble framework of SVMs.