Search

Few-shot semantic segmentation models aim to segment images after learning from only a few annotated examples. A key challenge for them is how to avoid overfitting because limited training data is available. While prior works usually limited the overall model capacity to alleviate overfitting, this hampers segmentation accuracy. We demonstrate how to increase overall model capacity to achieve improved performance, by introducing objectness, which is class-agnostic and so not prone to overfitting, for complementary use with class-specific features. Extensive experiments demonstrate the versatility of our simple approach of introducing objectness for different base architectures that rely on different data loaders and training schedules (DENet, PFENet) as well as with different backbone models (ResNet-50, ResNet-101 and HRNetV2-W48). Given only one annotated example of an unseen category, experiments show that our method outperforms state-of-art methods with respect to mIoU by at least 4.7% and 1.5% on PASCAL-5i and COCO-20i respectively.

We introduce a new large-scale dataset that links the assessment of image quality issues to two practical vision tasks: image captioning and visual question answering. First, we identify for 39,181 images taken by people who are blind whether each is sufficient quality to recognize the content as well as what quality flaws are observed from six options. These labels serve as a critical foundation for us to make the following contributions: (1) a new problem and algorithms for deciding whether an image is insufficient quality to recognize the content and so not captionable, (2) a new problem and algorithms for deciding which of six quality flaws an image contains, (3) a new problem and algorithms for deciding whether a visual question is unanswerable due to unrecognizable content versus the content of interest being missing from the field of view, and (4) a novel application of more efficiently creating a large-scale image captioning dataset by automatically deciding whether an image is insufficient quality and so should not be captioned. We publicly-share our datasets and code to facilitate future extensions of this work: https://vizwiz.org.

While an important problem in the vision community is to design algorithms that can automatically caption images, few publicly-available datasets for algorithm development directly address the interests of real users. Observing that people who are blind have relied on (human-based) image captioning services to learn about images they take for nearly a decade, we introduce the first image captioning dataset to represent this real use case. This new dataset, which we call VizWiz-Captions, consists of over 39,000 images originating from people who are blind that are each paired with five captions. We analyze this dataset to (1) characterize the typical captions, (2) characterize the diversity of content found in the images, and (3) compare its content to that found in eight popular vision datasets. We also analyze modern image captioning algorithms to identify what makes this new dataset challenging for the vision community. We publicly-share the dataset with captioning challenge instructions at https://vizwiz.org

Many people search for foreground objects to use when editing images. While existing methods can retrieve candidates to aid in this, they are constrained to returning objects that belong to a pre-specified semantic class. We instead propose a novel problem of unconstrained foreground object (UFO) search and introduce a solution that supports efficient search by encoding the background image in the same latent space as the candidate foreground objects. A key contribution of our work is a cost-free, scalable approach for creating a large-scale training dataset with a variety of foreground objects of differing semantic categories per image location. Quantitative and human-perception experiments with two diverse datasets demonstrate the advantage of our UFO search solution over related baselines., Comment: To appear in ICCV 2019

Foreground object segmentation is a critical step for many image analysis tasks. While automated methods can produce high-quality results, their failures disappoint users in need of practical solutions. We propose a resource allocation framework for predicting how best to allocate a fixed budget of human annotation effort in order to collect higher quality segmentations for a given batch of images and automated methods. The framework is based on a prediction module that estimates the quality of given algorithm-drawn segmentations. We demonstrate the value of the framework for two novel tasks related to predicting how to distribute annotation efforts between algorithms and humans. Specifically, we develop two systems that automatically decide, for a batch of images, when to recruit humans versus computers to create 1) coarse segmentations required to initialize segmentation tools and 2) final, fine-grained segmentations. Experiments demonstrate the advantage of relying on a mix of human and computer efforts over relying on either resource alone for segmenting objects in images coming from three diverse modalities (visible, phase contrast microscopy, and fluorescence microscopy).

Metal absorption line systems in distant quasar spectra probe of the history of gas content in the universe. The MgII $\lambda \lambda$ 2796, 2803 doublet is one of the most important absorption lines since it is a proxy of the star formation rate and a tracer of the cold gas associated with high redshift galaxies. Machine learning algorithms have been used to detect absorption lines systems in large sky surveys, such as Principle Component Analysis (PCA), Gaussian Process (GP) and decision trees. A very powerful algorithm in the field of machine learning called deep neural networks, or '' deep learning'' is a new structure of neural network that automatically extracts semantic features from raw data and represents them at a high level. In this paper, we apply a deep convolutional neural network for absorption line detection. We use the previously published DR7 MgII catalog (Zhu et al. 2013) as the training and validation sample and the DR12 MgII catalog as the test set. Our deep learning algorithm is capable of detecting MgII absorption lines with an accuracy of $\sim$94% . It takes only $\sim 9$ seconds to analyze $\sim$ 50000 quasar spectra with our deep neural network, which is ten thousand times faster than traditional methods, while preserving high accuracy with little human interference. Our study shows that Mg II absorption line detection accuracy of a deep neutral network model strongly depends on the filter size in the filter layer of the neural network, and the best results are obtained when the filter size closely matches the absorption feature size., Comment: 14 pages, 9 figures, 3 tables, Accepted for publication in Monthly Notices of the Royal Astronomical Society

Studying the stellar mass, age, luminosity, star-formation rate, and impact parameter of quasar absorber host galaxies can aid in the understanding of galaxy formation and evolution as well as in testing their models. We derive the Spectral Energy Distribution (SED) and impact parameter limits of low redshift ($z_{abs} = 0.37 - 0.55$) Mg II absorbers and of higher redshift ($z_{abs} = 1.0 - 2.5$) 2175 \AA\ dust absorbers (2DAs). We use an imaging stacking technique to statistically boost the signal-to-noise ratio (SNR) to increase detection of the absorber host galaxies. The point spread function of the background quasar is modeled with Principal Component Analysis (PCA). This method efficiently reduces the uncertainty of traditional PSF modeling. Our SED for Mg II absorbers indicates that low redshift Mg II absorber host galaxies are likely star-forming galaxies transitioning into red quiescent galaxies, with a low star formation rate of 2.2 $M_\odot$ $yr^{-1}$. From the stacked images and simulations, we show that the average impact parameter of 2DAs is > 5 times smaller than that of Mg II absorbers, at < 7 kpc instead of Mg II absorbers' 48 kpc, indicating that 2DAs are likely associated with disk components of high redshift galaxies. This means that 2DAs are likely good probes to study precursors to the Milky Way., Comment: 19 pages, 14 figures, accepted by MNRAS. arXiv admin note: text overlap with arXiv:astro-ph/0609760 by other authors

Deep generative models have shown success in automatically synthesizing missing image regions using surrounding context. However, users cannot directly decide what content to synthesize with such approaches. We propose an end-to-end network for image inpainting that uses a different image to guide the synthesis of new content to fill the hole. A key challenge addressed by our approach is synthesizing new content in regions where the guidance image and the context of the original image are inconsistent. We conduct four studies that demonstrate our results yield more realistic image inpainting results over seven baselines.

We present the cold neutral content (H I and C I gas) of 13 quasar 2175 \AA$ $ dust absorbers (2DAs) at $z$ = 1.6 - 2.5 to investigate the correlation between the presence of the UV extinction bump with other physical characteristics. These 2DAs were initially selected from the Sloan Digital Sky Surveys I - III and followed up with the Keck-II telescope and the Multiple Mirror Telescope as detailed in our Paper I. We perform a correlation analysis between metallicity, redshift, depletion level, velocity width, and explore relationships between 2DAs and other absorption line systems. The 2DAs on average have higher metallicity, higher depletion levels, and larger velocity widths than Damped Lyman-$\alpha$ absorbers (DLAs) or subDLAs. The correlation between [Zn/H] and [Fe/Zn] or [Zn/H] and log$\Delta$V$_{90}$ can be used as alternative stellar mass estimators based on the well-established mass-metallicity relation. The estimated stellar masses of the 2DAs in this sample are in the range of $\sim$ 10$^9$ to $\sim$2 $\times$ 10$^{11}$ $M_{\odot}$ with a median value of $\sim$2 $\times$ 10$^{10}$ $M_{\odot}$. The relationship with other quasar absorption line systems can be described as (1) 2DAs are a subset of Mg II and Fe II absorbers, (2) 2DAs are preferentially metal-strong DLAs/subDLAs, (3) More importantly, all of the 2DAs show C I detections with logN(C I) $>$ 14.0 cm$^{-2}$, (4) 2DAs can be used as molecular gas tracers. Their host galaxies are likely to be chemically enriched, evolved, massive (more massive than typical DLA/subDLA galaxies), and presumably star-forming galaxies., Comment: 12 pages, 11 figures; accepted for publication in MNRAS

We present 13 new 2175 {\AA} dust absorbers at z_abs = 1.0 - 2.2 towards background quasars from the Sloan Digital Sky Survey. These absorbers are examined in detail using data from the Echelle Spectrograph and Imager (ESI) on the Keck II telescope. Many low-ionization lines including Fe II, Zn II, Mg II, Si II, Al II, Ni II, Mn II, Cr II, Ti II, and Ca II are present in the same absorber which gives rise to the 2175 {\AA} bump. The relative metal abundances (with respect to Zn) demonstrate that the depletion patterns of our 2175 {\AA} dust absorbers resemble that of the Milky Way clouds although some are disk-like and some are halo-like. The 2175 {\AA} dust absorbers have significantly higher depletion levels compared to literature Damped Lyman-{\alpha} absorbers (DLAs) and subDLAs. The dust depletion level indicator [Fe/Zn] tends to anti-correlate with bump strengths. The velocity profiles from the Keck/ESI spectra also provide kinematical information on the dust absorbers. The dust absorbers are found to have multiple velocity components with velocity widths extending from ~100 to ~ 600 km/s, which are larger than those of most DLAs and subDLAs. Assuming the velocity width is a reliable tracer of stellar mass, the host galaxies of 2175 {\AA} dust absorbers are expected to be more massive than DLA/subDLA hosts. Not all of the 2175 {\AA} dust absorbers are intervening systems towards background quasars. The absorbers towards quasars J1006+1538 and J1047+3423 are proximate systems that could be associated with the quasar itself or the host galaxy., Comment: 27 pages, 32 figures, 31 tables; Accepted for publication in MNRAS