DELTA: A deep dual-stream network for multi-label image classification.

Highlights • We propose an end-to-end deep model with dual streams, which is able to effectively extract and make use of the global image priors and the local image features. • We incorporate a spatial pyramid convolutional transfer layer into the deep model. • We utilize multi-instance pooling layer to effectively aggregate the information contained in the feature maps. • Extensive experiments demonstrate that DELTA outperforms state-of-the-art deep multi-label classification methods. Abstract Multi-label image classification problem is one of the most important and fundamental problems in computer vision. In an image with multiple labels, the objects usually locate at various positions with different scales and poses. Moreover, some labels are associated with the entire image instead of a small region. Therefore, both the global and local information are important for classification. To effectively extract and make full use of these information, in this paper, we present a novel deep Dual-stream nEtwork for the muLTi-lAbel image classification task, DELTA for short. As its name indicates, it is composed of two streams, i.e., the Multi-Instance network and the Global Priors network. The former is used to extract the multi-scale class-related local instances features by modeling the classification problem in a multi-instance learning framework. The latter is devised to capture the global priors from the input image as the global information. These two streams are fused by the final fusion layer. In this way, DELTA can extract and make full use of both the global and local information for classification. Extensive experiments on three benchmark datasets, i.e., PASCAL VOC 2007, PASCAL VOC 2012 and Microsoft COCO, demonstrate that DELTA significantly outperforms several state-of-the-art methods. Moreover, DELTA can automatically locate the key image patterns that trigger the labels. [ABSTRACT FROM AUTHOR]