Search

With the rapid and continuous development of AIGC, It is becoming increasingly difficult to distinguish between real and forged facial images, which calls for efficient forgery detection systems. Although many detection methods have noticed the importance of local artifacts, there has been a lack of in-depth discussion regarding the selection of locations and their effective utilization. Besides, the traditional image augmentation methods that are widely used have limited improvements for forgery detection tasks and require more specialized augmentation methods specifically designed for forgery detection tasks. In this paper, this study proposes Local Artifacts Amplification for Deepfakes Augmentation, which amplifies the local artifacts on the forged faces. Furthermore, this study incorporates prior knowledge about similar facial features into the model. This means that within the facial regions defined in this work, forged features exhibit similar patterns. By aggregating the results from all facial regions, the study can enhance the overall performance of the model. The evaluation experiments conducted in this research, achieving an AUC of 93.40% and an Acc of 87.03% in the challenging WildDeepfake dataset, demonstrate a promising improvement in accuracy compared to traditional image augmentation methods and achieve superior performance on intra-dataset evaluation. The cross-dataset evaluation also showed that the method presented in this study has strong generalization abilities., Competing Interests: Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 Elsevier Ltd. All rights reserved.)

Face sketch synthesis (FSS) plays an important role in facial entertainment, which includes face sketch morphing among two styles, multiview FSS and face sketch expression manipulation. For facial entertainment, most existing FSS methods generate sketches with over-smoothing effects, i.e., fine details are suppressed more or less. In this paper, we propose a face sketch generator based on the compositional model to handle this issue. It decomposes a face into different components instead of patches as before, and each component has several candidate templates. Multilevel B-spline approximation is utilized to delicately polish the chosen templates of all components. To fuse these components, Poisson blending is employed instead of the weighted average operator. The proposed compositional method crucially reduces the high frequency loss and improves the synthesis performance in comparison to the state-of-the-art methods. Experiments on face sketch morphing, expression manipulation, and multiview FSS, make further efforts to demonstrate the effectiveness of the proposed method.

Face sketch-photo synthesis plays an important role in law enforcement and digital entertainment. Most of the existing methods only use pixel intensities as the feature. Since face images can be described using features from multiple aspects, this paper presents a novel multiple representations-based face sketch-photo-synthesis method that adaptively combines multiple representations to represent an image patch. In particular, it combines multiple features from face images processed using multiple filters and deploys Markov networks to exploit the interacting relationships between the neighboring image patches. The proposed framework could be solved using an alternating optimization strategy and it normally converges in only five outer iterations in the experiments. Our experimental results on the Chinese University of Hong Kong (CUHK) face sketch database, celebrity photos, CUHK Face Sketch FERET Database, IIIT-D Viewed Sketch Database, and forensic sketches demonstrate the effectiveness of our method for face sketch-photo synthesis. In addition, cross-database and database-dependent style-synthesis evaluations demonstrate the generalizability of this novel method and suggest promising solutions for face identification in forensic science.

Face sketch-photo synthesis plays a critical role in many applications, such as law enforcement and digital entertainment. Recently, many face sketch-photo synthesis methods have been proposed under the framework of inductive learning, and these have obtained promising performance. However, these inductive learning-based face sketch-photo synthesis methods may result in high losses for test samples, because inductive learning minimizes the empirical loss for training samples. This paper presents a novel transductive face sketch-photo synthesis method that incorporates the given test samples into the learning process and optimizes the performance on these test samples. In particular, it defines a probabilistic model to optimize both the reconstruction fidelity of the input photo (sketch) and the synthesis fidelity of the target output sketch (photo), and efficiently optimizes this probabilistic model by alternating optimization. The proposed transductive method significantly reduces the expected high loss and improves the synthesis performance for test samples. Experimental results on the Chinese University of Hong Kong face sketch data set demonstrate the effectiveness of the proposed method by comparing it with representative inductive learning-based face sketch-photo synthesis methods.

The problem of cross-domain face recognition aims to identify facial images obtained across different domains, which attracts increasing attentions because of its wide applications on law-enforcement identification and camera surveillance. The problem is challenging due to the huge domain discrepancy. Despite great progress achieved in recent years, existing algorithms usually fail to fully exploit the semantic information for identifying cross-domain faces, which could be a strong clue for recognition. In this article, we propose an effective algorithm for cross-domain face recognition by exploiting semantic information integrated with deep convolutional neural networks (CNN). We first introduce a soft face parsing algorithm where the boundaries of facial components are measured as probabilistic values. By taking the original face image as the guidance to improve face parsing result, each pixel may belong partially to the facial component to avoid inaccurate segmentation around component boundaries. We then propose a hierarchical soft semantic representation framework for cross-domain face recognition. Both the soft semantic level and contour level deep features obtained via CNN are computed and combined together, which could fully exploit the identical semantic clue among cross-domain faces. We provide extensive experiments to demonstrate that the proposed soft semantic representation algorithm performs superior against state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Face sketch synthesis from a photo is of significant importance in digital entertainment. An intelligent face sketch synthesis system requires a strong robustness to lighting variations. Under uncontrolled lighting conditions in real-world settings, such a system will perform consistently well and have little restriction on the lighting conditions. However, previous face sketch synthesis methods tend to synthesize sketches under well-controlled lighting conditions. These methods are sensitive to lighting variations and produce unsatisfactory results when the lighting condition varies. In this paper, we propose a novel cascaded face sketch synthesis framework composed of a multiple feature generator and a cascaded low-rank representation. The multiple feature generator not only produces a generated sketch feature consistent with an artist’s drawing style but also extracts a photo feature that is robust to various illuminations. Both features ensure that given a photo patch, the optimal sketch candidates can be selected from the database. The cascaded low-rank representation enables a gradual reduction in the gap between the synthesized face sketch and the corresponding artist-drawn sketch. Experimental results illustrate that the proposed cascaded framework generates realistic sketches on par with the current methods on the Chinese University of Hong Kong face sketch database under well-controlled illuminations. Moreover, this framework exhibits greatly improved performance compared to these methods on the extended Chinese University of Hong Kong face sketch database and Chinese celebrity face photos from the web under different illuminations. We argue that this framework paves a novel way for the implementation of computer-aided optical systems that are of essential importance in both face sketch synthesis and optical imaging. [ABSTRACT FROM AUTHOR]

Face sketch synthesis is a crucial technique in digital entertainment. However, the existing face sketch synthesis approaches usually generate face sketches with coarse structures. The fine details on some facial components fail to be generated. In this paper, inspired by the artists during drawing face sketches, we propose a bionic face sketch generator. It includes three parts: 1) a coarse part; 2) a fine part; and 3) a finer part. The coarse part builds the facial structure of a sketch by a generative adversarial network in the U-Net. In the middle part, the noise produced by the coarse part is erased and the fine details on the important face components are generated via a probabilistic graphic model. To compensate for the fine sketch with distinctive edge and area of shadows and lights, we learn a mapping relationship at the high-frequency band by a convolutional neural network in the finer part. The experimental results show that the proposed bionic face sketch generator can synthesize the face sketch with more delicate and striking details, satisfy the requirement of users in the digital entertainment, and provide the students with the coarse, fine, and finer face sketch copies when learning sketches. Compared with the state-of-the-art methods, the proposed approach achieves better results in both visual effects and quantitative metrics. [ABSTRACT FROM AUTHOR]

This paper considers the problem of face sketch synthesis in the wild, which transforms a face photo into a face sketch. Face sketch synthesis is widely applied in law enforcement as well as digital entertainment fields. However, the existing methods either focus on hand-crafted techniques where prior human experience is relied on or adopt deep learning techniques as an end-to-end framework, where facial details cannot be well represented. In this paper, we propose a novel approach for face sketch synthesis in the wild via a deep patch representation-based probabilistic graphical model (DeepPGM). A Siamese network is constructed to extract deep patch representation from a raw facial patch, where the representative detail information for robust face sketch synthesis can be exploited. The generated deep patch representation and facial image patches are then optimally combined through a probabilistic graphical model. The proposed DeepPGM approach not only outperforms the state-of-the-art on public face sketch datasets but also can cope with forensic photos in the wild conditions, including varying lightings, poses, occlusions, skin colors, and ethnic origins. The superiority of the proposed method is demonstrated by extensive experiments on two public face sketch datasets and real-world forensic photos in the wild. [ABSTRACT FROM AUTHOR]

Exemplar-based face sketch synthesis has long been impeded by the difficulty of accurate neighbor selection. Given a test patch extracted from the test photograph, the $K$ -nearest neighbor ($K$ -NN) matching algorithm is generally performed by existing methods to find $K$ -nearest photograph patches in the training data set, which contains some pairs of face sketches and photographs. Then, the training sketch patches corresponding to the selected nearest photograph patches are taken as the candidate to synthesize the target sketch patch. In the aforementioned neighbor selection process, training sketch patches is not taken into consideration in the process of $K$ -nearest neighbor selection. In this paper, we proposed a simple yet effective neighbor selection algorithm, namely, anchored neighborhood index (ANI), to boost the synthesis performance by taking training sketch patches into the consideration. In addition, the proposed ANI can be conducted offline and, thus, it does not increase the computational complexity. Extensive experiments on public available database demonstrate that the proposed algorithm achieves superior performance compared with the state-of-the-art methods in terms of both objective image quality scores and face recognition accuracy. [ABSTRACT FROM AUTHOR]

Face sketch synthesis refers to the technique generating a sketch from an input photo. Existing methods are data-driven, which synthesize a sketch by linearly combining K candidate sketch patches which are purposely selected from the training data. However, these methods have large computation cost due to neighbor selection process that perform neighbor searching on a large scale of training image patches. Instead of the aforementioned commonly used data-driven strategy, we propose to learn some models from training photos to training sketches which could speed the synthesis process a lot while preserving comparable or even better synthesis performance. Specially, we learn some ridge regressors from training photo patch intensities to training sketch patch intensities. An initial estimation is obtained from these regressors. Simultaneously, a high-frequency image is hallucinated from some ridge regressors which are learned from the high-frequency information of training photos and sketches. The high-frequency image is superimposed to the initial estimation to compensate the filtered details due to the dense average in the initial estimation process. Extensive experiments on public face sketch database illustrate the effectiveness of proposed model-driven strategy. [ABSTRACT FROM AUTHOR]

Heterogeneous face recognition (HFR) refers to matching face images acquired from different sources (i.e., different sensors or different wavelengths) for identification. HFR plays an important role in both biometrics research and industry. In spite of promising progresses achieved in recent years, HFR is still a challenging problem due to the difficulty to represent two heterogeneous images in a homogeneous manner. Existing HFR methods either represent an image ignoring the spatial information, or rely on a transformation procedure which complicates the recognition task. Considering these problems, we propose a novel graphical representation based HFR method (G-HFR) in this paper. Markov networks are employed to represent heterogeneous image patches separately, which takes the spatial compatibility between neighboring image patches into consideration. A coupled representation similarity metric (CRSM) is designed to measure the similarity between obtained graphical representations. Extensive experiments conducted on multiple HFR scenarios (viewed sketch, forensic sketch, near infrared image, and thermal infrared image) show that the proposed method outperforms state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

Face sketch synthesis is crucial in many practical applications, such as digital entertainment and law enforcement. Previous methods relying on many photo–sketch pairs have made great progress. State-of-the-art face sketch synthesis algorithms adopt Bayesian inference (BI) (e.g., Markov random fields) to select local sketch patches around corresponding position from a set of training data. However, these methods have two limitations: 1) they depend on many training photo–sketch pairs and 2) they cannot tackle nonfacial factors (e.g., hairpins, glasses, backgrounds, and image size) if these factors are excluded in training data. In this paper, we propose a novel face sketch synthesis method that is capable of handling nonfacial factors only using a single photo–sketch pair from coarse to fine. Our method proposes a cascaded image synthesis (CIS) strategy and integrates sparse representation-based greedy search (SRGS) and BI for face sketch synthesis. We first apply SRGS to select candidate sketch patches from the whole training photo–sketch pairs sampled from the only photo–sketch pair. We then employ BI to estimate an initial sketch. Afterward, the input photo and the estimated initial sketch are taken as an additional photo–sketch pair for training. Finally, we adopt CIS with the given two photo–sketch pairs to further improve the quality of the initial sketch. The experimental results on several databases demonstrate that our algorithm outperforms state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Face sketch–photo synthesis technique has attracted growing attention in many computer vision applications, such as law enforcement and digital entertainment. Existing methods either simply perform the face sketch–photo synthesis on the holistic image or divide the face image into regular rectangular patches ignoring the inherent structure of the face image. In view of such situations, this paper presents a novel superpixel-based face sketch–photo synthesis method by estimating the face structures through image segmentation. In our proposed method, face images are first segmented into superpixels, which are then dilated to enhance the compatibility of neighboring superpixels. Each input face image induces a specific graphical structure modeled by Markov networks. We employ a two-stage synthesis process to learn the face structures through Markov networks constructed from two scales of dilation, respectively. Experiments on several public databases demonstrate that our proposed face sketch–photo synthesis method achieves superior performance compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• To the best of our knowledge, we first employ full-scale identity supervision to synthesis model, which provides a comprehensive constraint. • Our CTIA divides the face synthesis into cross-domain translation and intra-domain adaptation, which is effective to generate realistic images. • We first incorporate pseudo face images to augment the training set, and significantly improve the robustness for generative model. Face photo-sketch synthesis refers transforming a face image between photo domain and sketch domain. It plays a crucial role in law enforcement and digital entertainment. A great deal of efforts have been devoted on face photo-sketch synthesis. However, limited by the weak identity supervision, existing methods mostly yield indistinct details or great deformation, resulting in poor perceptual appearance or low recognition accuracy. In the past several years, face identification achieved great progress, which represents the face images much more precisely than before. Considering the face image translation is also a type of face image re-representation, we attempt to introduce face recognition models to improve the synthesis performance. First, we applied existing synthesis models to augment the training set. Then, we proposed a full-scale identity supervision method to reduce redundant information introduced by these pseudo samples and take the valid information to enhance the intra-class variations. The proposed framework consists of two sub-networks: cross-domain translation (CT) network and intra-domain adaptation (IA) network. The CT network translates the input image from source domain to latent image of target domain, which overcomes the great gap between two domains with less structural deformation. The IA network adapts the perceptual appearance of latent image to target image by adversarial learning. Experimental results on CUHK Face Sketch Database and CUHK Face Sketch FERET Database demonstrate the proposed method preserved best perceptual appearance and more distinct details with less deformation. [ABSTRACT FROM AUTHOR]

Sketch–photo synthesis plays an important role in sketch-based face photo retrieval and photo-based face sketch retrieval systems. In this paper, we propose an automatic sketch–photo synthesis and retrieval algorithm based on sparse representation. The proposed sketch–photo synthesis method works at patch level and is composed of two steps: sparse neighbor selection (SNS) for an initial estimate of the pseudoimage (pseudosketch or pseudophoto) and sparse-representation-based enhancement (SRE) for further improving the quality of the synthesized image. SNS can find closely related neighbors adaptively and then generate an initial estimate for the pseudoimage. In SRE, a coupled sparse representation model is first constructed to learn the mapping between sketch patches and photo patches, and a patch-derivative-based sparse representation method is subsequently applied to enhance the quality of the synthesized photos and sketches. Finally, four retrieval modes, namely, sketch-based, photo-based, pseudosketch-based, and pseudophoto-based retrieval are proposed, and a retrieval algorithm is developed by using sparse representation. Extensive experimental results illustrate the effectiveness of the proposed face sketch–photo synthesis and retrieval algorithms. [ABSTRACT FROM AUTHOR]

Face photo–sketch synthesis plays an increasingly important role in the field of law enforcement and entertainment. However, most of the existing methods only focus on how to cross the modality differences between the photo domain and the sketch domain, while neglecting the image quality enhancement of the generated faces. In this paper, we propose a novel intra-domain enhancement (IDE) method for the face photo–sketch synthesis task. Our method is composed of two steps to cope with the inter-domain generation gap and the intra-domain quality defect respectively, i.e. inter-domain face generation and intra-domain face enhancement. In the inter-domain face generation stage, we apply a probabilistic graphical model which is capable of synthesizing the coarse face sketch from photos with low quality. In the intra-domain face enhancement stage, a Multi-Layer feature Aggregation (MLA) based Generative Adversarial Networks (GAN) is designed to enhance facial details in the sketches and further refine the synthesized image quality. Our first phase helps bridge the modality gap between photos and sketches and ensures the shape consistency between them. The second phase aims to improve the detail-quality and resolution of the synthesized sketches. Extensive experimental results on public face sketch databases outperform the state-of-the-art face sketch synthesis methods. [ABSTRACT FROM AUTHOR]