1. Simple is good: Investigation of history-state ensemble deep neural networks and their validation on rotating machinery fault diagnosis.
- Author
-
Wang, Yu and Vinogradov, Alexey
- Subjects
- *
ARTIFICIAL neural networks , *ROTATING machinery , *FAULT diagnosis , *DEEP learning - Abstract
• It is demonstrated that deep networks can generate multiple local optima during training process which can be combined to form a stronger model. • An efficient and easy-to-implement ensemble learning approach designed for deep networks is introduced. • The approach improves the performance of neural networks without additional training cost. • The approach can be directly applied to all kinds of neural networks without tuning the network architecture. • Comparison experiments with a range of ensemble strategies have shown that the simplest ensemble strategy performs best. The present work is motivated by the desire to find an efficient approach that can improve the performance of deep neural networks in a general sense. To this end, an easy-to-implement ensemble approach is proposed in this paper leveraging the 'local sub-optima' of deep networks, which is referred as to history-state ensemble (HSE) method. We demonstrated that neural networks can naturally generate multiple 'local sub-optima' with diversity during training process, and their combination can effectively improve the accuracy and stability of the single network. The merits of HSE are twofold: (1) It does not require additional training cost in order to acquire multiple base models, which is one of the main drawbacks limiting the generalization of ensemble techniques in deep learning. (2) It can be easily applied to any types of deep networks without tuning of network architectures. We proposed the simplest way to perform HSE and investigated more than 20 ensemble strategies for HSE as comparison. Experiments are conducted on six datasets and eight popular network architectures for the case of rotating machinery fault diagnosis. It is demonstrated that the stability and accuracy of neural networks can be generally improved through the simplest ensemble strategy proposed in this paper. [ABSTRACT FROM AUTHOR]
- Published
- 2023
- Full Text
- View/download PDF