Search

The exponential surge in specialized image editing software has intensified visual forgery, with splicing attacks emerging as a popular forgery technique. In this context, Siamese neural networks are a remarkable tool in pattern identification for detecting image manipulations. This paper introduces a deep learning approach for splicing detection based on a Siamese neural network tailored to identifying manipulated image regions. The Siamese neural network learns unique features of specific image areas and detects tampered regions through feature comparison. This architecture employs two identical branches with shared weights and image features to compare image blocks and identify tampered areas. Subsequently, a K-means algorithm is applied to identify similar centroids and determine the precise localization of duplicated regions in the image. The experimental results encompass various splicing attacks to assess effectiveness, demonstrating a high accuracy of 98.6% and a precision of 97.5% for splicing manipulation detection. This study presents an advanced splicing image forgery detection and localization algorithm, showcasing its efficacy through comprehensive experiments.

With the flourishing development of generative models, image manipulation is receiving increasing attention. Rather than text modality, several elegant designs have delved into leveraging audio to manipulate images. However, existing methodologies mainly focus on image generation conditional on semantic alignment, ignoring the vivid affective information depicted in the audio. We propose an Emotion-aware StyleGAN Manipulator (EmoStyle), a framework where affective information from audio can be explicitly extracted and further utilized during image manipulation. Specifically, we first leverage the multi-modality model ImageBind for initial cross-modal retrieval between images and music, and select the music-related image for further manipulation. Simultaneously, by extracting sentiment polarity from the lyrics of the audio, we generate an emotionally rich auxiliary music branch to accentuate the affective information. We then leverage pre-trained encoders to encode audio and the audio-related image into the same embedding space. With the aligned embeddings, we manipulate the image via a direct latent optimization method. We conduct objective and subjective evaluations on the generated images, and our results show that our framework is capable of generating images with specified human emotions conveyed in the audio.

With the increasing visual realism of computer-graphics (CG) images generated by advanced rendering engines, the distinction between CG images and natural images (NIs) has become an important research problem in the image forensics community. Previous research works mainly focused on the conventional supervised learning framework, which usually requires a good quantity of labeled data for training. To our knowledge, we study, for the first time in the literature, the utility of the self-supervised learning mechanism for the forensic classification of CG images and NIs. The idea is to make use of a large number of readily available unlabeled data, along with a self-supervised training procedure on a well-designed pretext task for which labels can be generated in an automatic and convenient way without human manual labeling effort. Differing from existing self-supervised methods, based on pretext tasks targeted at image understanding, or based on contrastive learning, we propose carrying out self-supervised training on a forensics-oriented pretext task of classifying authentic images and their modified versions after applying various manipulations. Experiments and comparisons showed the effectiveness of our method for solving the CG forensics problem under different evaluation scenarios. Our proposed method outperformed existing self-supervised methods in all experiments. It could sometimes achieve comparable, or better, performance. compared with a state-of-the-art fully supervised method under difficult evaluation scenarios with data scarcity and a challenging forensic problem. Our study demonstrates the utility and potential of the self-supervised learning mechanism for image forensics applications.

Due to the high flexibility and conformity to people’s usage habits, text description has been widely used in image synthesis research recently and has achieved many encouraging results. However, the text can only determine the basic content of the generated image and cannot determine the specific shape of the synthesized object, which leads to poor practicability. More importantly, the current text-to-image synthesis research cannot use new text descriptions to further modify the synthesis result. To solve these problems, this paper proposes a text-guided customizable image synthesis and manipulation method. The proposed method synthesizes the corresponding image based on the text and contour information at first. It then modifies the synthesized content based on the new text to obtain a satisfactory result. The text and contour information in the proposed method determine the specific content and object shape of the desired composite image, respectively. Aside from that, the input text, contour, and subsequent new text for content modification can be manually input, which significantly improves the artificial controllability in the image synthesis process, making the entire method superior to other methods in flexibility and practicability. Experimental results on the Caltech-UCSD Birds-200-2011 (CUB) and Microsoft Common Objects in Context (MS COCO) datasets demonstrate our proposed method’s feasibility and versatility.

Digital image forensics has become necessary as an emerging technology. Images can be adulterated effortlessly using image tools. The latest techniques are available to detect whether an image is adulterated by a particular operator. Most of the existing techniques are suitable for high resolution and manipulated images by a single operator. In a real scenario, multiple operators are applied to manipulate the image many times. In this paper, a robust moderate-sized convolutional neural network is proposed to identify manipulation operators and also the operator’s sequence for two operators in particular. The proposed bottleneck approach is used to make the network deeper and reduce the computational cost. Only one pooling layer, called a global averaging pooling layer, is utilized to retain the maximum flow of information and to avoid the overfitting issue between the layers. The proposed network is also robust against low resolution and JPEG compressed images. Even though the detection of the operator is challenging due to the limited availability of statistical information in low resolution and JPEG compressed images, the proposed model can also detect an operator with different parameters and compression quality factors that are not considered in training.

Image forgery is an active topic in digital image tampering that is performed by moving a region from one image into another image, combining two images to form one image, or retouching an image. Moreover, recent developments of generative adversarial networks (GANs) that are used to generate human facial images have made it more challenging for even humans to detect the tampered one. The spread of those images on the internet can cause severe ethical, moral, and legal issues if the manipulated images are misused. As a result, much research has been conducted to detect facial image manipulation based on applying machine learning algorithms on tampered face datasets in the last few years. This paper introduces a deep learning-based framework that can identify manipulated facial images and GAN-generated images. It is comprised of multiple convolutional layers, which can efficiently extract features using multi-level abstraction from tampered regions. In addition, a data-based approach, cost-sensitive learning-based approach (class weight), and ensemble-based approach (eXtreme Gradient Boosting) is applied to the proposed model to deal with the imbalanced data problem (IDP). The superiority of the proposed model that deals with an IDP is verified using a tampered face dataset and a GAN-generated face dataset under various scenarios. Experimental results proved that the proposed framework outperformed existing expert systems, which has been used for identifying manipulated facial images and GAN-generated images in terms of computational complexity, area under the curve (AUC), and robustness. As a result, the proposed framework inspires the development of research on image forgery identification and enables the potential to integrate these models into practical applications, which require tampered facial image detection.

The visual search performance of radiologists is systematically challenged due to the use of flat-panel displays when viewing multi-slice images. Three-dimensional visualization of information is expected to facilitate visual search for radiologists. However, as the differences in user performance, depending on radiologist experience, have not yet been assessed for real three-dimensional images, it is difficult to estimate the usability of new visualization systems for radiologists with different amounts of professional search experience. In this study, we aimed to determine how three-dimensional visual search is affected by radiologist experience and the physical properties of stimuli. Radiologists, residents, and medical students accomplished visual search tasks with varying set sizes and target-distractor similarities. Search items were presented on multiple display planes and image manipulation was conducted using a computer keyboard. We compared the search outcome and strategy among three groups of participants. All three groups submitted a similar number of correct responses depending on target-distractor similarity. However, in contrast to radiologists and residents, it took a considerably longer time for students to find the targets. The major differences were evident in search strategy when comparing the image manipulations of radiologists with those of residents and students. Our results demonstrated that experienced radiologists’ search performance benefited from seeing the entire three-dimensional view of an image, whereas residents and students focused instead on reducing the amount of information and searched in a selective manner. These findings should be taken into account when planning the implementation of new workstations with three-dimensional visualization in radiology.