SOM-based information maximization to improve and interpret multi-layered neural networks: From information reduction to information augmentation approach to create new information

The present paper aims to show the necessity of information augmentation to cope with the natural decrease in information content in multi-layered neural networks. It is natural to make an effort to collect as much information as possible, because it is impossible to know which information is necessary or useful before learning. Thus, the present paper tries to force neural networks to form new network configurations to have as much information as possible, contrary to the conventional approach of information reduction, such as many types of regularization. For information augmentation, we use the self-organizing map (SOM), which can over-represent inputs and produce as many similar weights as possible. The method was applied to two data sets: the banknote authentication data set and the character recognition data set. In both experimental results, it was confirmed that redundant and excessive information generation in terms of the excessive number of connection weights was connected with improved generalization.