1. Ultra Fine-Grained Image Semantic Embedding
- Author
-
Chun-Ta Lu, Sujith Ravi, Tom Duerig, Gao Yaxi, Da-Cheng Juan, Andrew Tomkins, Zhen Li, Yi-Ting Chen, Aleksei Timofeev, and Futang Peng
- Subjects
Computer science ,business.industry ,media_common.quotation_subject ,Pattern recognition ,02 engineering and technology ,010501 environmental sciences ,Semantics ,01 natural sciences ,Ranking (information retrieval) ,Image (mathematics) ,Perception ,0202 electrical engineering, electronic engineering, information engineering ,Graph (abstract data type) ,Embedding ,020201 artificial intelligence & image processing ,Artificial intelligence ,Ultra fine ,business ,Image retrieval ,0105 earth and related environmental sciences ,media_common - Abstract
"How to learn image embeddings that capture fine-grained semantics based on the instance of an image?" "Is it possible for such embeddings to further understand image semantics closer to humans' perception?" In this paper, we present, Graph-Regularized Image Semantic Embedding (Graph-RISE), a web-scale neural graph learning framework deployed at Google, which allows us to train image embeddings to discriminate an unprecedented O(40M) ultra-fine-grained semantic labels. The proposed Graph-RISE outperforms state-of-the-art image embedding algorithms on several evaluation tasks, including kNN search and triplet ranking: the accuracy is improved by approximately 2X on the ImageNet dataset and by more than 5X on the iNaturalist dataset. Qualitatively, image retrieval from one billion images based on the proposed Graph-RISE effectively captures semantics and, compared to the state-of-the-art, differentiates nuances at levels that are closer to human-perception.
- Published
- 2020
- Full Text
- View/download PDF