A self-organizing deep belief network based on information relevance strategy.

One of the major obstacles in using deep belief network (DBN) is the structure design. Numerous studies, both empirically and theoretically, show that choosing suitable structure can improve the performance of DBN. In this paper, a self-organizing DBN (S-DBN), based on the information relevance strategy (IRS), was proposed to design the structure of DBN. For this IRS, the maximal information coefficient was designed to measure the input and output information relevance of hidden neurons. Meanwhile, the mutual information was introduced to measure the information relevance among the hidden layers. Then, a novel self-organizing strategy was developed to grow and prune both the hidden neurons and layers during the training process. Moreover, a contrastive divergence algorithm was used to adjust the parameters of S-DBN. Finally, several benchmark problems were used to illustrate the effectiveness of S-DBN. The experimental results demonstrate that the proposed S-DBN owns better performance for classification problems and modeling nonlinear systems than some existing methods. [ABSTRACT FROM AUTHOR]