Hierarchical gate network for fine-grained visual recognition

The visual classification has achieved unprecedented progress in the last decade, and miscellaneous network architectures have emerged. However, these models yield inferior performance when deployed in fine-grained classification problems, as they are usually devised by enlarging the model capacity or facilitating the optimization, and few concentrate on the problem itself. In this paper, we argue that in most fine-grained classification problems, concepts are intrinsically hierarchically structured rather than evenly distributed, and thus classifying all concepts within a single layer simultaneously deteriorates the discrimination among different categories. Furthermore, the category hierarchy is usually not provided, which fails some existing methods where the human-defined hierarchy is required. In order to tackle these challenges, we propose a new architecture, referred to as Hierarchical Gate Network (HGNet), to exploit the interconnection among hierarchical categories. HGNet adopts an LSTM-like mechanism to transmit dependencies among classes of different levels in the hierarchy. In such a way, the context information in the hierarchical structure is utilized to boost the recognition performance. Experiments conducted on various benchmark datasets, including CUB-200-2011, Stanford Dogs, NABirds, Aircraft, iNaturalist, DeepFashion and DeepFashion2, demonstrate the superiority of the proposed method to the state-of-the-art algorithms.