Search

The present paper aims to propose a new information-theoretic method to minimize and maximize selective information repeatedly. In particular, we try to solve the incomplete information control problem, where information cannot be fully controlled due to the existence of many contradictory factors inside. For this problem, the cost in terms of the sum of absolute connection weights is introduced for neural networks to increase and decrease information against contradictory forces in learning, such as error minimization. Thus, this method is called a "cost-forced" approach to control information. The method is contrary to the conventional regularization approach, where the cost has been used passively or negatively. The present method tries to use the cost positively, meaning that the cost can be augmented if necessary. The method was applied to an artificial and symmetric data set. In the symmetric data set, we tried to show that the symmetric property of the data set could be obtained by appropriately controlling information. In the second data set, that of residents in a nursing home, obtained by the complicated procedures of natural language processing, the experimental results confirmed that the present method could control selective information to extract non-linear relations as well as linear ones in increasing interpretation and generalization performance. [ABSTRACT FROM AUTHOR]

The present paper aims to propose a new information-theoretic method to minimize and maximize selective information repeatedly. In particular, we try to solve the incomplete information control problem, where information cannot be fully controlled due to the existence of many contradictory factors inside. For this problem, the cost in terms of the sum of absolute connection weights is introduced for neural networks to increase and decrease information against contradictory forces in learning, such as error minimization. Thus, this method is called a "cost-forced" approach to control information. The method is contrary to the conventional regularization approach, where the cost has been used passively or negatively. The present method tries to use the cost positively, meaning that the cost can be augmented if necessary. The method was applied to an artificial and symmetric data set. In the symmetric data set, we tried to show that the symmetric property of the data set could be obtained by appropriately controlling information. In the second data set, that of residents in a nursing home, obtained by the complicated procedures of natural language processing, the experimental results confirmed that the present method could control selective information to extract non-linear relations as well as linear ones in increasing interpretation and generalization performance. [ABSTRACT FROM AUTHOR]

The present paper aims to extend the potential learning method to overcome the problem of collective interpretation, which aims to interpret multi-layered neural networks by compressing them into the simplest ones. In the process of compression, positive, negative, and complicated weights have had unfavourable effects for interpretation. To deal with the problems of collective interpretation, the potential learning is extended only to use positive weights. In addition, to obtain more appropriate weights for interpretation, the number of candidate weights for higher potentialities is first increased as much as possible. Then, from among many candidates, more appropriate weights are selected as more important ones. This extended potentiality learning is expected to produce more stable and more simple representations for easy interpretation. The extended method was applied to three datasets, namely, an artificial dataset, a real eye-tracking dataset, and a student evaluation dataset. In all cases, it was observed that the selectivity of connection weights could be increased. Correspondingly, the majority of connection weights became positive, and the collective weights were quite similar to the regression coefficients of the logistic regression analysis. Finally, for the third dataset (student evaluations), the extended method could extract more explicit input-output relations, compared with the logistic regression analysis, while improving generalisation performance. [ABSTRACT FROM AUTHOR]