Back to Search
Start Over
An experimental methodology to evaluate machine learning methods for fault diagnosis based on vibration signals
- Source :
- Expert Systems with Applications. 167:114022
- Publication Year :
- 2021
- Publisher :
- Elsevier BV, 2021.
-
Abstract
- This paper presents a systematic procedure to fairly compare experimental performance scores for machine learning methods for fault diagnosis based on vibration signals. In the vast majority of related scientific publications, the estimated accuracy and similar performance criteria are the sole quality parameter presented. However, the experimental design giving rise to these results is mostly biased, based on unacceptably simple validation methods and on recycling identical patterns in test data sets, previously used for training. Moreover, the methods in general overfit their hyperparameters, introducing additional overoptimistic results. In order to remedy this defect, we critically analyse the usual training-validation-test division and propose an algorithmic guideline in the form of a validation framework. This allows a well defined comparison of experimental results. In order to illustrate the ideas of the paper, the Case Western Reserve University Bearing Data benchmark is used as a case study. Four distinct classifiers are experimentally compared, under gradually more difficult generalization tasks using the proposed evaluation framework: K-Nearest-Neighbor, Support Vector Machine, Random Forest and One-Dimensional Convolutional Neural Network. An extensive literature review suggests that most vibration based research papers, particularly for the Case Western Reserve University Bearing Data, use similar patterns for training and testing, making their classification an easy task.
- Subjects :
- Hyperparameter
0209 industrial biotechnology
business.industry
Generalization
Computer science
General Engineering
02 engineering and technology
Overfitting
Machine learning
computer.software_genre
Convolutional neural network
Computer Science Applications
Random forest
Support vector machine
020901 industrial engineering & automation
Artificial Intelligence
0202 electrical engineering, electronic engineering, information engineering
Benchmark (computing)
020201 artificial intelligence & image processing
Artificial intelligence
business
computer
Test data
Subjects
Details
- ISSN :
- 09574174
- Volume :
- 167
- Database :
- OpenAIRE
- Journal :
- Expert Systems with Applications
- Accession number :
- edsair.doi...........03f68c82b9e6aa850d0acd6fe49e42a9