Search

Automatic hand-drawn sketch recognition is an important task in computer vision. However, the vast majority of prior works focus on exploring the power of deep learning to achieve better accuracy on complete and clean sketch images, and thus fail to achieve satisfactory performance when applied to incomplete or destroyed sketch images. To address this problem, we first develop two datasets that contain different levels of scrawl and incomplete sketches. Then, we propose an angular-driven feedback restoration network (ADFRNet), which first detects the imperfect parts of a sketch and then refines them into high quality images, to boost the performance of sketch recognition. By introducing a novel “feedback restoration loop” to deliver information between the middle stages, the proposed model can improve the quality of generated sketch images while avoiding the extra memory cost associated with popular cascading generation schemes. In addition, we also employ a novel angular-based loss function to guide the refinement of sketch images and learn a powerful discriminator in the angular space. Extensive experiments conducted on the proposed imperfect sketch datasets demonstrate that the proposed model is able to efficiently improve the quality of sketch images and achieve superior performance over the current state-of-the-art methods. [ABSTRACT FROM AUTHOR]

As an unnoticed specialized task in image restoration, rain-by-snow weather removal aims to eliminate the complicated coexisting rain streaks and snow particles. In this work, we propose a simple attention-based sampling cross-hierarchy Transformer (SASCFormer). Initially, we explore the proximity of convolution network and Transformer in hierarchical architectures and experimentally find they perform approximately for intra-stage feature representation. On this basis, we utilize a Transformer-like convolution block (TCB) to replace the computation-heavy self-attention while preserving the attention characteristics for adapting to the input content. Meanwhile, we demonstrate that cross-stage sampling progression is critical for the performance improvement in rain-by-snow weather removal, and propose a global–local self-attention sampling mechanism (GLSASM) that samples the features while preserving both the global and local dependencies. Finally, we synthesize two novel rain-by-snow weather-degraded benchmarks, RSCityscapes and RS100K datasets. Extensive experiments verify that our proposed SASCFormer achieves the best trade-off between the performance and inference time. In particular, our approach advances existing methods by 1. 14 dB ∼ 4. 89 dB in peak signal-to-noise ratio. Related resources are available at https://github.com/chdwyb/Rain-by-snow. • We first focus on rain-by-snow weather-degraded image restoration. • We propose a simple attention-based sampling cross-hierarchy Transformer. • We demonstrate that cross-stage progression is crucial for performance improvement. • Two novel rain-by-snow datasets are synthesized for rain-by-snow weather removal. [ABSTRACT FROM AUTHOR]