Statistical Ensemble Method (SEM): A New Meta-machine Learning Approach Based on Statistical Techniques

The goal of combining the outputs of multiple models is to form an improved meta-model with higher generalization capability than the best single model used in isolation. Most popular ensemble methods do specify neither the number of component models nor their complexity. However, these parameters strongly influence the generalization capability of the meta-model. In this paper we propose an ensemble method which generates a meta-model with optimal values for these parameters. The proposed method suggests using resampling techniques to generate multiple estimations of the generalization error and multiple comparison procedures to select the models that will be combined to form the meta-model. Experimental results show the performance of the model on regression and classification tasks using artificial and real databases.