Search

Recent captioning models are limited in their ability to describe concepts unseen in paired image–sentence pairs. This study presents a framework of multi-task learning for describing novel words not present in existing image-captioning datasets. The authors’ framework takes advantage of external sources-labelled images from image classification datasets, and semantic knowledge extracted from the annotated text. They propose minimising a joint objective which can learn from these diverse data sources and leverage distributional semantic embeddings. When in the inference step they change the BeamSearch step by considering both the caption model and language model enabling the model to generalise novel words outside of image-captioning datasets. They demonstrate that in the framework by adding an annotated text data which can help the image captioning model to describe images with the right corresponding novel words. Extensive experiments are conducted on both AI Challenger and Microsoft coco (MSCOCO) image captioning datasets of two different languages, demonstrating the ability of their framework to describe novel words such as scenes and objects.

The Wide Field Survey Telescope (WFST) is a dedicated photometric surveying facility being built jointly by University of Science and Technology of China (USTC) and the Purple Mountain Observatory (PMO). It is equipped with a 2.5-meter diameter primary mirror, an active optics system, and a mosaic CCD camera with 0.73 gigapixels on the primary focal plane for high-quality image capture over a 6.5-square-degree field of view. The installation of WFST near the summit of Saishiteng mountain in the Lenghu region is scheduled in summer of 2023, and the operation is planned to start three months later. WFST will scan the northern sky in four optical bands (u, g, rand i) at cadences from hourly/daily in the deep high-cadence survey (DHS) program, to semi-weekly in the wide field survey (WFS) program. During a photometric night, a nominal 30 s exposure in the WFS program will reach a depth of 22.27, 23.32, 22.84, and 22.31 (AB magnitudes) in these four bands, respectively, allowing for the detection of a tremendous amount of transients in the low-zuniverse and a systematic investigation of the variability of Galactic and extragalactic objects. In the DHS program, intranight 90 s exposures as deep as 23 (u) and 24 mag (g), in combination with target of opportunity follow-ups, will provide a unique opportunity to explore energetic transients in demand for high sensitivities, including the electromagnetic counterparts of gravitational wave events, supernovae within a few hours of their explosions, tidal disruption events and fast, luminous optical transients even beyond redshift of unity. In addition, the final 6-year co-added images, anticipated to reach g≃ 25.8 mag in WFS or 1.5 mags deeper in DHS, will be of fundamental importance to general Galactic and extragalactic science. The highly uniform legacy surveys of WFST will serve as an indispensable complement to those of the Vera C. Rubin Observatory’s Legacy Survey of Space and Time (LSST) that monitors the southern sky.