Mapping DCNN to a Three Layer Modular Architecture: A Systematic Way for Obtaining Wider and More Effective Network

We propose a modular Deep Convolutional Neural Network (DCNN) architecture which has the property of block-like design and re-usage of parameters by certain blocks. We leverage networks from the ResNet family as the backbone for our proposed architecture. The proposed network architecture is composed of three primary blocks, i) common block ii) hidden blocks and iii) output block. The common block mainly focuses on learning low-level features which are afterward fed to ρ hidden blocks at a time, where ρ is the branching factor. The hidden blocks are placed in a row-wise manner and replicated ρ times depending on the computational availability. The output block takes features as an input from the common block and all the hidden blocks during both training and testing phases. In addition, each of the hidden blocks together with the common block is a stand-alone neural network, which implies that we have ρ neural networks coupled in a single framework. During the training phase, in every single epoch, each of the stand-alone neural networks (i.e. common block + hidden block) is trained in Round Robin fashion followed by fine-tuning the whole network graph. Inference can be performed using either the whole network graph or just any sub-graph of it. To substantiate the effectiveness of our proposed neural architecture we performed experiment on CIFAR-10 and CIFAR-100 datasets.