Parallel neural networks for improved nonlinear principal component analysis

In this paper, a parallel neural network architecture is proposed to improve the performance of neural-network-based nonlinear principal component analysis. There exist two typical approaches for such analysis: simultaneous extraction of principal components using a single autoassociative neural network (also known as autoencoder), and sequential extraction using multiple neural networks in series. The proposed architecture can be obtained by systematically pruning the network connections of a fully connected autoassociative neural network, resulting in decoupled neural networks. As a result, more independent (i.e., less correlated) principal components can be obtained than the simultaneous extraction approach. The proposed architecture can be also viewed as a rearrangement of multiple neural networks for the sequential extraction in a parallel setting, and thus, the network training becomes more efficient. Simulation case studies are performed to illustrate the advantages of the proposed architecture, and it was shown that it is particularly beneficial for deep neural networks.