Search

For ensemble learning, how to select and combine the candidate classifiers are two key issues which influence the performance of the ensemble system dramatically. Random vector functional link networks (RVFL) without direct input-to-output links is one of suitable base-classifiers for ensemble systems because of its fast learning speed, simple structure and good generalization performance. In this paper, to obtain a more compact ensemble system with improved convergence performance, an improved ensemble of RVFL based on attractive and repulsive particle swarm optimization (ARPSO) with double optimization strategy is proposed. In the proposed method, ARPSO is applied to select and combine the candidate RVFL. As for using ARPSO to select the optimal base RVFL, ARPSO considers both the convergence accuracy on the validation data and the diversity of the candidate ensemble system to build the RVFL ensembles. In the process of combining RVFL, the ensemble weights corresponding to the base RVFL are initialized by the minimum norm least-square method and then further optimized by ARPSO. Finally, a few redundant RVFL is pruned, and thus the more compact ensemble of RVFL is obtained. Moreover, in this paper, theoretical analysis and justification on how to prune the base classifiers on classification problem is presented, and a simple and practically feasible strategy for pruning redundant base classifiers on both classification and regression problems is proposed. Since the double optimization is performed on the basis of the single optimization, the ensemble of RVFL built by the proposed method outperforms that built by some single optimization methods. Experiment results on function approximation and classification problems verify that the proposed method could improve its convergence accuracy as well as reduce the complexity of the ensemble system. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, a new approach coupling adaptive particle swarm optimization (APSO) and a priori information for function approximation problem is proposed to obtain better generalization performance and faster convergence rate. It is well known that gradient-based learning algorithms such as backpropagation (BP) algorithm have good ability of local search, whereas PSO has good ability of global search. Therefore, in the new approach, first, APSO encoding the first-order derivative information of the approximated function is applied to train network to near global minima. Second, with the connection weights produced by APSO, the network is trained with a gradient-based algorithm. Due to combining APSO with local search algorithm and considering a priori information, the new approach has better generalization performance and convergence rate than traditional learning ones. Finally, simulation results are given to verify the efficiency and effectiveness of the proposed approach. [Copyright &y& Elsevier]