1. Evaluation of variable selection methods for random forests and omics data sets
- Author
-
Stephan Seifert, Silke Szymczak, and Frauke Degenhardt
- Subjects
Paper ,Clustering high-dimensional data ,Computer science ,0206 medical engineering ,high dimensional data ,Stability (learning theory) ,Breast Neoplasms ,Feature selection ,02 engineering and technology ,computer.software_genre ,Machine Learning ,Set (abstract data type) ,03 medical and health sciences ,feature selection ,relevant variables ,Biomarkers, Tumor ,Feature (machine learning) ,Humans ,Computer Simulation ,Molecular Biology ,030304 developmental biology ,Parametric statistics ,0303 health sciences ,Gene Expression Profiling ,Computational Biology ,DNA Methylation ,Random forest ,Variable (computer science) ,Female ,Data mining ,computer ,random forest ,Algorithms ,020602 bioinformatics ,Information Systems - Abstract
Machine learning methods and in particular random forests are promising approaches for prediction based on high dimensional omics data sets. They provide variable importance measures to rank predictors according to their predictive power. If building a prediction model is the main goal of a study, often a minimal set of variables with good prediction performance is selected. However, if the objective is the identification of involved variables to find active networks and pathways, approaches that aim to select all relevant variables should be preferred. We evaluated several variable selection procedures based on simulated data as well as publicly available experimental methylation and gene expression data. Our comparison included the Boruta algorithm, the Vita method, recurrent relative variable importance, a permutation approach and its parametric variant (Altmann) as well as recursive feature elimination (RFE). In our simulation studies, Boruta was the most powerful approach, followed closely by the Vita method. Both approaches demonstrated similar stability in variable selection, while Vita was the most robust approach under a pure null model without any predictor variables related to the outcome. In the analysis of the different experimental data sets, Vita demonstrated slightly better stability in variable selection and was less computationally intensive than Boruta. In conclusion, we recommend the Boruta and Vita approaches for the analysis of high-dimensional data sets. Vita is considerably faster than Boruta and thus more suitable for large data sets, but only Boruta can also be applied in low-dimensional settings.
- Published
- 2017