Back to Search
Start Over
The feature selection bias problem in relation to high-dimensional gene data.
- Source :
-
Artificial Intelligence in Medicine . Jan2016, Vol. 66, p63-71. 9p. - Publication Year :
- 2016
-
Abstract
- <bold>Objective: </bold>Feature selection is a technique widely used in data mining. The aim is to select the best subset of features relevant to the problem being considered. In this paper, we consider feature selection for the classification of gene datasets. Gene data is usually composed of just a few dozen objects described by thousands of features. For this kind of data, it is easy to find a model that fits the learning data. However, it is not easy to find one that will simultaneously evaluate new data equally well as learning data. This overfitting issue is well known as regards classification and regression, but it also applies to feature selection.<bold>Methods and Materials: </bold>We address this problem and investigate its importance in an empirical study of four feature selection methods applied to seven high-dimensional gene datasets. We chose datasets that are well studied in the literature-colon cancer, leukemia and breast cancer. All the datasets are characterized by a significant number of features and the presence of exactly two decision classes. The feature selection methods used are ReliefF, minimum redundancy maximum relevance, support vector machine-recursive feature elimination and relaxed linear separability.<bold>Results: </bold>Our main result reveals the existence of positive feature selection bias in all 28 experiments (7 datasets and 4 feature selection methods). Bias was calculated as the difference between validation and test accuracies and ranges from 2.6% to as much as 41.67%. The validation accuracy (biased accuracy) was calculated on the same dataset on which the feature selection was performed. The test accuracy was calculated for data that was not used for feature selection (by so called external cross-validation).<bold>Conclusions: </bold>This work provides evidence that using the same dataset for feature selection and learning is not appropriate. We recommend using cross-validation for feature selection in order to reduce selection bias. [ABSTRACT FROM AUTHOR]
- Subjects :
- *FEATURE selection
*GENETIC databases
*DATA mining
*REGRESSION analysis
*LEUKEMIA diagnosis
*ALGORITHMS
*COMPARATIVE studies
*DATABASES
*DECISION making
*GENES
*INFORMATION science
*RESEARCH methodology
*MEDICAL cooperation
*RESEARCH
*BIOINFORMATICS
*EVALUATION research
*RESEARCH bias
*OLIGONUCLEOTIDE arrays
*GENE expression profiling
RESEARCH evaluation
Subjects
Details
- Language :
- English
- ISSN :
- 09333657
- Volume :
- 66
- Database :
- Academic Search Index
- Journal :
- Artificial Intelligence in Medicine
- Publication Type :
- Academic Journal
- Accession number :
- 113007915
- Full Text :
- https://doi.org/10.1016/j.artmed.2015.11.001