Search

During the last decade, a significant research progress has been drawn in both the theoretical aspects and the applications of Deep Learning Neural Networks. Besides their spectacular applications, optimal architectures of these neural networks may speed up the learning process and exhibit better generalization results. So far, many growing and pruning algorithms have been proposed by many researchers to deal with the optimization of standard Feedforward Neural Network architectures. However, applying both the growing and the pruning on the same net may lead a good model for a big data set and hence good selection results. This work is devoted to propose a new Growing and pruning Learning algorithm for Deep Neural Networks. This new algorithm is presented and applied on diverse medical data sets. It is shown that this algorithm outperforms various other artificial intelligent techniques in terms of accuracy and simplicity of the resulting architecture. [ABSTRACT FROM AUTHOR]

In this paper, we present a new pruning algorithm formed by combining the Statistical Stepwise Method (SSM) [1] with the Iterative Pruning (IP) [4] algorithms. This proposed algorithm (SSIP) is used to simultaneously remove unnecessary neurons or weight connections from a given feed-forward neural network (NN) in order to "optimize" its structure. Some modifications to the previous pruning algorithms published in [1] and [4] are also reported. Two versions of the combined SSIP are considered: In the first version, SSIP1, the modified IP is first applied to the given neural network in order to prune insignificant units, and then the modified SSM is applied to the pruned network to remove unnecessary links. In the second version, SSIP2, the above procedure is applied to each layer in turn, working from the input layer to the output layer. The performances of the algorithms are compared using two real world applications, brain disease detection and texture classification, and the superiority of the SSIP pruning algorithm is demonstrated. This new algorithm can eliminate approximately 59% of the links in the initial oversized network in order to improve performance by approximately +39% and +26% for the sensitivity of learning, and generalization, respectively. [ABSTRACT FROM AUTHOR]