Heuristic creation of deep rule ensemble through iterative expansion of feature space.

• We propose a deep rule ensemble creation approach. • It is driven by iterative expansion of the feature space. • It involves multiple ways of heuristic creation of diversity. • We compare the proposed approach with existing rule learning and ensemble methods. • The proposed approach achieves significant advances in classification accuracy. Rule learning approaches, which essentially aim to gerenate a decision tree or a set of "if-then" rules, have been popularly used in practice for automatically building rule-based models for prediction tasks, e.g., classification and regression. The key strength of rule-based models is their ability to interpret how an output is obtained given an input, in comparison with models trained by other machine learning approaches, e.g., neural networks. Moreover, ensemble learning approaches have been adopted as a popular way for advancing the performance of rule-based prediction through producing multiple rule-based models with diversity. Traditional approaches of ensemble learning are typically designed to train a single ensemble. In recent years, there have been some studies on creation of multiple ensembles towards increasing the diversity among rule-based models and the depth of ensemble learning. In this paper, we propose a feature expansion driven approach for automatic creation of deep rule ensembles, i.e., the dimensionality of the feature space is increased at each iteration by adding features newly created at the previous iteration. The proposed approach is compared with more recent approaches of rule learning and ensemble creation. The experimental results show that the proposed approach achieves improved performance on various data sets. [ABSTRACT FROM AUTHOR]