FMSNet: A Multi-Stream CNN for Multi-Stereo Image Classification by Feature Map Sharing

Convolutional Neural Networks (CNNs) have achieved significant success in image classification and object detection. CNN models generally consist of a single-stream and process single image data at once. In addition, multi-stream (or multi-modal) models have recently begun to be proposed that allow the processing of more than one input at the same time. The data can be an image, video, voice, or any other sensor data. Multi-modality may help us extract some hidden features of the same object. Furthermore, several new studies examine sharing feature maps between different streams of the same CNN. However, systematic studies that can adequately demonstrate the contribution of multi-modality and feature map sharing features to performance have not yet been conducted. Processing power and lack of available datasets are among the important factors that negatively affect progress. In this study, the contributions of multi-modality and feature map sharing (FMS) to increase the performance in object recognition are examined in detail. For this purpose, a new dataset and a new multi-modal multi-feature map sharing CNN model, which we call FMSNet, are developed. The proposed model achieved a 3.06% higher accuracy rate than its non-FMS counterpart, DenseNet-201, exceeding most of the state-of-the-art single-stream CNN models.