Back to Search
Start Over
Learning hierarchically-structured concepts.
- Source :
-
Neural Networks . Nov2021, Vol. 143, p798-817. 20p. - Publication Year :
- 2021
-
Abstract
- We use a recently developed synchronous Spiking Neural Network (SNN) model to study the problem of learning hierarchically-structured concepts. We introduce an abstract data model that describes simple hierarchical concepts. We define a feed-forward layered SNN model, with learning modeled using Oja's local learning rule, a well known biologically-plausible rule for adjusting synapse weights. We define what it means for such a network to recognize hierarchical concepts; our notion of recognition is robust, in that it tolerates a bounded amount of noise. Then, we present a learning algorithm by which a layered network may learn to recognize hierarchical concepts according to our robust definition. We analyze correctness and performance rigorously; the amount of time required to learn each concept, after learning all of the sub-concepts, is approximately O 1 η k ℓmax log (k) + 1 ɛ + b log (k) , where k is the number of sub-concepts per concept, ℓmax is the maximum hierarchical depth, η is the learning rate, ɛ describes the amount of uncertainty allowed in robust recognition, and b describes the amount of weight decrease for "irrelevant" edges. An interesting feature of this algorithm is that it allows the network to learn sub-concepts in a highly interleaved manner. This algorithm assumes that the concepts are presented in a noise-free way; we also extend these results to accommodate noise in the learning process. Finally, we give a simple lower bound saying that, in order to recognize concepts with hierarchical depth two with noise-tolerance, a neural network should have at least two layers. The results in this paper represent first steps in the theoretical study of hierarchical concepts using SNNs. The cases studied here are basic, but they suggest many directions for extensions to more elaborate and realistic cases. [ABSTRACT FROM AUTHOR]
- Subjects :
- *ARTIFICIAL neural networks
*CONCEPT learning
*MACHINE learning
*ALGORITHMS
*SAWLOGS
Subjects
Details
- Language :
- English
- ISSN :
- 08936080
- Volume :
- 143
- Database :
- Academic Search Index
- Journal :
- Neural Networks
- Publication Type :
- Academic Journal
- Accession number :
- 152773923
- Full Text :
- https://doi.org/10.1016/j.neunet.2021.07.033