351. User evaluation of automatically generated keywords and toponyms for geo - referenced images
- Author
-
Frank O. Ostermann, Martin Tomko, Ross S. Purves, Faculty of Geo-Information Science and Earth Observation, Department of Geo-information Processing, UT-I-ITC-STAMP, University of Zurich, and Ostermann, Frank O
- Subjects
Information retrieval ,UFSP13-3 Language and Space ,Computer Networks and Communications ,Computer science ,1702 Artificial Intelligence ,Computer-assisted web interviewing ,1710 Information Systems ,ADLIB-ART-4768 ,1712 Software ,Human-Computer Interaction ,1709 Human-Computer Interaction ,Annotation ,10122 Institute of Geography ,Ranking ,Artificial Intelligence ,ITC-ISI-JOURNAL-ARTICLE ,1705 Computer Networks and Communications ,Information system ,910 Geography & travel ,Web resource ,Image retrieval ,Software ,Information Systems - Abstract
This article presents the results of a user evaluation of automatically generated concept keywords and place names (toponyms) for geo-referenced images. Automatically annotating images is becoming indispensable for effective information retrieval, since the number of geo-referenced images available online is growing, yet many images are insufficiently tagged or captioned to be efficiently searchable by standard information retrieval procedures. The Tripod project developed original methods for automatically annotating geo-referenced images by generating representations of the likely visible footprint of a geo-referenced image, and using this footprint to query spatial databases and web resources. These queries return raw lists of potential keywords and toponyms, which are subsequently filtered and ranked. This article reports on user experiments designed to evaluate the quality of the generated annotations. The experiments combined quantitative and qualitative approaches: To retrieve a large number of responses, participants rated the annotations in standardized online questionnaires that showed an image and its corresponding keywords. In addition, several focus groups provided rich qualitative information in open discussions. The results of the evaluation show that currently the annotation method performs better on rural images than on urban ones. Further, for each image at least one suitable keyword could be generated. The integration of heterogeneous data sources resulted in some images having a high level of noise in the form of obviously wrong or spurious keywords. The article discusses the evaluation itself and methods to improve the automatic generation of annotations.
- Published
- 2013