Removal of hidden neurons in multilayer perceptrons by orthogonal projection and weight crosswise propagation.

A new method of pruning away hidden neurons in neural networks is presented in this paper. The hidden neuron is removed by analyzing the orthogonal projection correlations among the outputs of other hidden neurons. The method guarantees the least loss of weight information in terms of orthogonal projection. The remaining weights and thresholds are updated based on the weight crosswise propagation. A practical technique for penalizing the superfluous hidden neurons is explored. Retraining is needed after pruning. Extensive experiments are conducted, and the results demonstrate that the method gives better initial points for retraining and retraining costs less epochs. [ABSTRACT FROM AUTHOR]