Learning multi-layer coarse-to-fine representations for large-scale image classification.

Highlights • Inter-category visual and semantic correlations are exploited. • Large numbers of structural image classes are organized hierarchically. • Hierarchical multi-task SVMs are trained over the visual-semantic tree. • The visual-semantic tree and CNNs are integrated as another framework. • We perform our experiments on 10k image categories for algorithm evaluation. Abstract Recent studies on large-scale image classification mainly focus on categorizing images into 1000 object classes, and all these 1000 object classes are atomic and mutually exclusive in the semantic space. However, for a much larger set of image categories (such as the ImageNet 10k dataset), some of them may come from the high-level (non-leaf) nodes of the concept ontology and could contain some other lower-level categories semantically. The research that classifies images into large numbers of image categories with such inter-category subsumption correlations has received rare attention. In this paper, a Visual-Semantic Tree is learned to organize 10k image categories hierarchically in a coarse-to-fine fashion, where both the inter-category visual similarities and inter-category semantic correlations are seamlessly integrated for tree construction. Additionally, a deep learning method is developed by integrating the Visual-Semantic Tree with deep CNNs to learn more discriminative tree classifiers for large-scale image classification. Our experimental results have demonstrated that the proposed Visual-Semantic Tree can effectively organize large-scale structural image categories and significantly boost the classification accuracy rates for both atomic image categories and high-level image categories. [ABSTRACT FROM AUTHOR]