Back to Search
Start Over
Fluency-Guided Cross-Lingual Image Captioning
- Source :
- ACM Multimedia
- Publication Year :
- 2017
- Publisher :
- arXiv, 2017.
-
Abstract
- Image captioning has so far been explored mostly in English, as most available datasets are in this language. However, the application of image captioning should not be restricted by language. Only few studies have been conducted for image captioning in a cross-lingual setting. Different from these works that manually build a dataset for a target language, we aim to learn a cross-lingual captioning model fully from machine-translated sentences. To conquer the lack of fluency in the translated sentences, we propose in this paper a fluency-guided learning framework. The framework comprises a module to automatically estimate the fluency of the sentences and another module to utilize the estimated fluency scores to effectively train an image captioning model for the target language. As experiments on two bilingual (English-Chinese) datasets show, our approach improves both fluency and relevance of the generated captions in Chinese, but without using any manually written sentences from the target language.<br />Comment: 9 pages, 2 figures, accepted as ORAL by ACM Multimedia 2017
- Subjects :
- Closed captioning
FOS: Computer and information sciences
Cross lingual
Computer Science - Computation and Language
business.industry
Computer science
InformationSystems_INFORMATIONSTORAGEANDRETRIEVAL
ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION
02 engineering and technology
computer.software_genre
Image (mathematics)
03 medical and health sciences
Fluency
0302 clinical medicine
030221 ophthalmology & optometry
0202 electrical engineering, electronic engineering, information engineering
ComputingMethodologies_DOCUMENTANDTEXTPROCESSING
020201 artificial intelligence & image processing
Relevance (information retrieval)
Artificial intelligence
business
computer
Computation and Language (cs.CL)
Natural language processing
Subjects
Details
- Database :
- OpenAIRE
- Journal :
- ACM Multimedia
- Accession number :
- edsair.doi.dedup.....6a10c607c6542d53b3ed16afb1ef28d0
- Full Text :
- https://doi.org/10.48550/arxiv.1708.04390