Search

In many real world scenarios, it is difficult to capture the images in the visible light spectrum (VIS) due to bad lighting conditions. However, the images can be captured in such scenarios using Near-Infrared (NIR) and Thermal (THM) cameras. The NIR and THM images contain the limited details. Thus, there is a need to transform the images from THM/NIR to VIS for better understanding. However, it is non-trivial task due to the large domain discrepancies and lack of abundant datasets. Nowadays, Generative Adversarial Network (GAN) is able to transform the images from one domain to another domain. Most of the available GAN based methods use the combination of the adversarial and the pixel-wise losses (like L 1 or L 2 ) as the objective function for training. The quality of transformed images in case of THM/NIR to VIS transformation is still not up to the mark using such objective function. Thus, better objective functions are needed to improve the quality, fine details and realism of the transformed images. A new model for THM/NIR to VIS image transformation called Perceptual Cyclic-Synthesized Generative Adversarial Network (PCSGAN) is introduced to address these issues. The PCSGAN uses the combination of the perceptual (i.e., feature based) losses along with the pixel-wise and the adversarial losses. Both the quantitative and qualitative measures are used to judge the performance of the PCSGAN model over the WHU-IIP face and the RGB-NIR scene datasets. The proposed PCSGAN outperforms the state-of-the-art image transformation models, including Pix2pix, DualGAN, CycleGAN, PS2GAN, and PAN in terms of the SSIM, MSE, PSNR and LPIPS evaluation measures. The code is available at https://github.com/KishanKancharagunta/PCSGAN. [ABSTRACT FROM AUTHOR]

• Visually realistic photos can be generated with "divide and conquer" strategy. • Multi-scale structure can extract detailed features from current layer and affect the quality of features in the next layers. • Self-attention mechanism redistributes more attention on facial region. • The first and last layers of feature extractor contribute significantly to photos generation. In this study, we investigate the sketch-to-photo problem, which currently poses a significant challenge in the field of computer vision. A large number of GAN-based encoder-decoder methods have been proposed for image transformation, inspired by the pix2pix model; however, these methods do not produce satisfactory results for photo generation, due to the fact that (1) they miss detailed information of input images because of a single-scale convolution operator in the shallow encoder layers, and (2) they fail to learn long-range dependencies in the deep encoder layers. To better handle these challenges, we present an approach that follows a "divide and conquer" strategy. Our method combines the advantages of a multi-scale convolutional neural network and an attention mechanism and applies these two modules to different encoder layers. Additionally, by optimizing a well-designed loss function, the complex correlations between the sketch and the photo can be calculated. Experimental results show that our method is able to generate high-quality photos from sketch images, and qualitative and quantitative analysis demonstrates its effectiveness and superiority over state-of-the-art models. This work paves a path to replace the traditional encoder structure with the "divide and conquer" strategy to handle image transformation tasks. [ABSTRACT FROM AUTHOR]

Generating high fidelity identity-preserving faces with different facial attributes has a wide range of applications. Although a number of generative models have been developed to tackle this problem, there is still much room for further improvement. In particular, the current solutions usually ignore the perceptual information of images, which we argue that it benefits the output of a high-quality image while preserving the identity information, especially in facial attributes learning area. To this end, we propose to train GAN iteratively via regularizing the min-max process with an integrated loss, which includes not only the per-pixel loss but also the perceptual loss. In contrast to the existing methods only deal with either image generation or transformation, our proposed iterative architecture can achieve both of them. Experiments on the multi-label facial dataset CelebA demonstrate that the proposed model has excellent performance on recognizing multiple attributes, generating a high-quality image, and transforming image with controllable attributes. [ABSTRACT FROM AUTHOR]

We present a method based on a robust two-layer cascade reciprocal pipeline (TCPR) and the context-aware dissimilarity measure (CADM) for feature-based image registration. We first build the two-layer pipeline to find the optimal correspondence and to achieve the image transformation. In the first layer, the neighborhood structures of point sets are used to find and register reliable feature point sets. In the second layer, we create the point-to-point neighborhood structure to recover the remaining inliers for image registration. The CADM with scaling, translation and rotation invariance is then designed for dissimilarity evaluation. Extensive experiments regarding feature matching, image registration and retrieval on real and synthesized images demonstrate that our method can achieve a better accuracy–efficiency tradeoff compared with ten state-of-the-art methods. [ABSTRACT FROM AUTHOR]