Back to Search
Start Over
A comprehensive survey on optimizing deep learning models by metaheuristics
- Source :
- Artificial Intelligence Review. 55:829-894
- Publication Year :
- 2021
- Publisher :
- Springer Science and Business Media LLC, 2021.
-
Abstract
- © 2021, The Author(s), under exclusive licence to Springer Nature B.V.Deep neural networks (DNNs), which are extensions of artificial neural networks, can learn higher levels of feature hierarchy established by lower level features by transforming the raw feature space to another complex feature space. Although deep networks are successful in a wide range of problems in different fields, there are some issues affecting their overall performance such as selecting appropriate values for model parameters, deciding the optimal architecture and feature representation and determining optimal weight and bias values. Recently, metaheuristic algorithms have been proposed to automate these tasks. This survey gives brief information about common basic DNN architectures including convolutional neural networks, unsupervised pre-trained models, recurrent neural networks and recursive neural networks. We formulate the optimization problems in DNN design such as architecture optimization, hyper-parameter optimization, training and feature representation level optimization. The encoding schemes used in metaheuristics to represent the network architectures are categorized. The evolutionary and selection operators, and also speed-up methods are summarized, and the main approaches to validate the results of networks designed by metaheuristics are provided. Moreover, we group the studies on the metaheuristics for deep neural networks based on the problem type considered and present the datasets mostly used in the studies for the readers. We discuss about the pros and cons of utilizing metaheuristics in deep learning field and give some future directions for connecting the metaheuristics and deep learning. To the best of our knowledge, this is the most comprehensive survey about metaheuristics used in deep learning field.
- Subjects :
- Linguistics and Language
Optimization problem
Artificial neural network
business.industry
Computer science
Feature vector
Deep learning
Computer Science::Neural and Evolutionary Computation
02 engineering and technology
Machine learning
computer.software_genre
Convolutional neural network
Language and Linguistics
Recurrent neural network
Artificial Intelligence
020204 information systems
0202 electrical engineering, electronic engineering, information engineering
Feature (machine learning)
020201 artificial intelligence & image processing
Artificial intelligence
business
Metaheuristic
computer
Subjects
Details
- ISSN :
- 15737462 and 02692821
- Volume :
- 55
- Database :
- OpenAIRE
- Journal :
- Artificial Intelligence Review
- Accession number :
- edsair.doi.dedup.....81db3bfdf37bcb4007a477ae0956e5ab