Back to Search
Start Over
Constructive learning of recurrent neural networks: limitations of recurrent cascade correlation and a simple solution
- Source :
- IEEE Transactions on Neural Networks. July, 1995, Vol. 6 Issue 4, p829, 8 p.
- Publication Year :
- 1995
-
Abstract
- It is often difficult to predict the optimal neural network size for a particular application. Constructive or destructive methods that add or subtract neurons, layers, connections, etc. might offer a solution to this problem. We prove that one method, recurrent cascade correlation, due to its topology, has fundamental limitations in representation and thus in its learning capabilities. It cannot represent with monotone (i.e., sigmoid) and hard-threshold activation functions certain finite state automata. We give a 'preliminary' approach on how to get around these limitations by devising a simple constructive training method that adds neurons during training while still preserving the powerful fully-recurrent structure. We illustrate this approach by simulations which learn many examples of regular grammars that the recurrent cascade correlation method is unable to learn.
Details
- ISSN :
- 10459227
- Volume :
- 6
- Issue :
- 4
- Database :
- Gale General OneFile
- Journal :
- IEEE Transactions on Neural Networks
- Publication Type :
- Academic Journal
- Accession number :
- edsgcl.17382422