1. Knowledge-based generative adversarial networks for scene understanding in Cultural Heritage
- Author
-
Raissa Garozzo, Concetto Spampinato, Giuseppe Vecchio, and Cettina Santagati
- Subjects
010506 paleontology ,Archeology ,Ontology Concept ,Computer science ,Image classification ,Ontology (information science) ,Machine learning ,computer.software_genre ,01 natural sciences ,Domain (software engineering) ,Information retrieval ,0601 history and archaeology ,Digital documentation ,Digital Cultural Heritage ,Representation (mathematics) ,0105 earth and related environmental sciences ,Generative Adversarial Networks ,060102 archaeology ,Contextual image classification ,business.industry ,Deep learning ,06 humanities and the arts ,Cultural heritage ,Artificial intelligence ,Semantic Data Modeling ,business ,Ontology-driven deep learning ,computer ,Generative grammar - Abstract
This study shows the results of an interdisciplinary research aimed at devising artificial intelligence methods to support data understanding in Cultural Heritage. The objective is to create automated methods for automatically classifying and successively retrieving photographic data leveraging the current breed of AI methods based on Deep Learning paradigm. In this work, the lack of images to be used for the training of the AI system is addressed by testing an approach based on Generative Adversarial Networks (GANs) for automatically synthesizing unrealistic photos, which, in turn, can be used for training image classification and retrieval systems. Specifically, we propose a method to drive the generation of realistic classical order images using GAN approaches anchored to semantic ontology domain representation. More specifically, the proposed AI leverages the advantages of knowledge-based and data-driven methods and foresees a level image generation: the first one that synthesizes isolated objects corresponding to each ontology concept and the second one that, instead, combines the generated objects according to the spatial information provided by the ontology in order to generate realistic scenes. The resulting images can then be employed to train automated classifiers and more reliable retrieval methods by minimizing the efforts usually required by human operators to manually annotate data. In addition, the proposed generative strategy allows for a deeper understanding of cultural heritage visual data enabling the possibility to enhance design capabilities in simulation contexts.
- Published
- 2021