Back to Search
Start Over
A novel deep auto-encoder considering energy and label constraints for categorization
- Source :
- Expert Systems with Applications. 176:114936
- Publication Year :
- 2021
- Publisher :
- Elsevier BV, 2021.
-
Abstract
- Deep auto-encoder (DAE) is one of the representative deep learning algorithms for feature extraction. However, it often shows relatively poor generalization performance to express data without considering the probability distribution of data. Additionally, it cannot be directly applied to classification, because label information is ignored in DAE to judge the given categories. To tackle these issues, in this paper, we propose an energy and label constrained DAE (ELDAE) by integrating energy and label constraints to improve the feature extraction ability of network for classification. Specifically, as the probability distribution for fitting data can be reflected by energy of a network, the energy constraint is designed in this study to improve the probability of ELDAE for fitting data, and make a better expression to data. Moreover, the label constraint is integrated in ELDAE using label information to describe categorization rule, contributing to enhancing the accuracy of classification. We first give the complexity analysis of ELDAE, which is crucial to the property of speed. To exhibit the performance of the proposed ELDAE, we perform comprehensive experiments on benchmark USPS and MNIST datasets, and parameter sensitivity analysis is then provided to investigate the effects of three key parameters including the balance coefficients of weight decay, energy constraint and label constraint. In addition, we compare ELDAE with six state-of-the-art algorithms including Auto-Encoder (AE), Sparse AE (SAE), Deep AE (DAE), Deep Belief Network (DBN), Noisy AE (NAE) and Semi-supervised AE (SSAE). The comparative experimental results demonstrate that ELDAE performs better than the other six competitors in terms of classification accuracy, and keeps the same order of magnitude in terms of training time and testing time.
- Subjects :
- 0209 industrial biotechnology
Computer science
business.industry
Deep learning
Feature extraction
General Engineering
Pattern recognition
02 engineering and technology
Autoencoder
Computer Science Applications
Constraint (information theory)
Deep belief network
020901 industrial engineering & automation
Artificial Intelligence
0202 electrical engineering, electronic engineering, information engineering
Benchmark (computing)
Probability distribution
020201 artificial intelligence & image processing
Artificial intelligence
business
MNIST database
Subjects
Details
- ISSN :
- 09574174
- Volume :
- 176
- Database :
- OpenAIRE
- Journal :
- Expert Systems with Applications
- Accession number :
- edsair.doi...........e483f16137ab68e6ed6a2e881429a87a