Search

The further study on the sensitivity analysis of Neocognitron is discussed in this paper. Fukushima's Neocognitron is capable of recognizing distorted patterns as well as tolerating positional shift. Supervised learning of the Neocognitron is fulfilled by training patterns layer by layer. However, many parameters, such as selectivity and receptive fields are set manually. Furthermore, in Fukushima's original Neocognitron, all the training patterns are designed empirically. In this paper, we use Genetic Algorithms (GAs) to tune the parameters of Neocognitron and search its reasonable training pattern sets. Four contributions are claimed: first, by analyzing the learning mechanism of Fukushima's original Neocognitron, the correlations amongst the training patterns are claimed to affect the performance of Neocognitron, tuning the Neocognitron's number of planes is equivalent to searching reasonable training patterns for its supervised learning; second, a GA-based supervised learning of the Neocognitron is carried out in this way, searching the parameters and training patterns by GAs but specifying the connection weights by training the Neocognitron; third, other than traditional GAs which are unsuitable for the large searching space of training patterns set, the cooperative coevolution is incorporated to play this role; fourth, an effective fitness function is given out when applying the above methodology into numeral recognition. The evolutionary computation in our initial experiments is implemented based on the original training pattern set, e.g. the individuals of the population are generated from Fukushima's original training patterns during initialization of GAs. The results prove that our correlation analysis is reasonable, and show that the performance of a Neocognitron is sensitive to its training patterns, selectivity and receptive fields, especially, the performance is not monotonically increasing with respect to the number of training patterns, and this GA-based supervised learning is able to improve Neocognitron's performance.

A hybrid cognitive system is proposed where a working neocognitron is synthesized with a set of production rules. The knowledge base of a neocognitron is constructed through incorporating production rules into its interlayer connections. Training for prototype patterns is not required. The semantic of interlayer connections is established. The resulting network can now be analyzed according to the rule structure and problematic portions can be corrected. Neocognitrons constructed using this hybrid approach have been tested on the same set of handwritten numerals initiated by Fukushima with scaling and skewing distortions, and with noise contamination. It is found that the performance is comparable to that of Fukushima's network obtained by supervised training.

Rule-embedded neocognitron (REN) is proposed where the knowledge base of a neocognitron is constructed through incorporating production rules into its interlayer connections. Prototype patterns training is not required. The semantic of interlayer connections is established. The resulting network can now be analyzed according to the rule structure and problematic portions can be corrected. We demonstrate the ease with which performance can be improved by applying REN on handwritten numeral recognition. The same set of handwritten numerals initiated by Fukushima is used to test this methodology. It is found that the performance is comparable with that of Fukushima's neocognitron with supervised training.