Search

The modern society generates vast amounts of digital content, whose credibility plays a pivotal role in shaping public opinion and decision-making processes. The rapid development of social networks and generative technologies, such as deepfakes, significantly increases the risk of disinformation through image manipulation. This article aims to review methods for verifying images' integrity, particularly through deep learning techniques, addressing both passive and active approaches. Their effectiveness in various scenarios has been analyzed, highlighting their advantages and limitations. This study reviews the scientific literature and research findings, focusing on techniques that detect image manipulations and localize areas of tampering, utilizing both statistical properties of images and embedded hidden watermarks. Passive methods, based on analyzing the image itself, are versatile and can be applied across a broad range of cases; however, their effectiveness depends on the complexity of the modifications and the characteristics of the image. Active methods, which involve embedding additional information into the image, offer precise detection and localization of changes but require complete control over creating and distributing visual materials. Both approaches have their applications depending on the context and available resources. In the future, a key challenge remains the development of methods resistant to advanced manipulations generated by diffusion models and further leveraging innovations in deep learning to protect the integrity of visual content. [ABSTRACT FROM AUTHOR]

Text-to-image conversion has garnered significant research attention, with contemporary methods leveraging the latent space analysis of StyleGAN. However, issues with latent code decoupling, interpretability, and controllability often remain, leading to misaligned image attributes. To address these challenges, we propose a refined approach that segments StyleGAN’s latent code using the Visual Language Model (CLIP). Our method aligns the latent code segments with text embeddings via an image-text alignment module and modulates them through a text injection module. Additionally, we incorporate semantic segmentation loss and mouth loss to constrain operations that affect irrelevant attributes. Compared to previous CLIP-driven techniques, our approach significantly enhances decoupling, interpretability, and controllability. Experiments on the CelebA-HQ and FFHQ datasets validate our model’s efficacy through both qualitative and quantitative comparisons. Our model effectively handles a wide range of style variations, achieving an FID score of 21.15 for facial attributes and an ID metric of 0.88 for hair attributes. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNN) are widely used by researchers due to their extensive advantages over various applications. However, images are highly susceptible to malicious attacks using perturbations that are unrecognized even under human intervention. This causes significant security perils and challenges to CNN-related applications. In this article, an efficient adversarial training model against malevolent attacks is demonstrated. This model is highly robust to black-box malicious examples, it is processed with different malicious samples. Initially, malicious training models like fast gradient descent (FGS), recursive-FGSM (I-FGS), Deep-Fool, and Carlini and Wagner (CW) techniques are utilized that generate adversarial input by means of the CNN acknowledged to the attacker. In the experimentation process, the MNIST dataset comprising 60K and 10K training and testing grey-scale images are utilized. In the experimental section, the adversarial training model reduces the attack accuracy rate (ASR) by an average of 29.2% for different malicious inputs, when preserving the accuracy of 98.9% concerning actual images in the MNIST database. The simulation outcomes show the preeminence of the model against adversarial attacks. [ABSTRACT FROM AUTHOR]

This paper presents mosaic-library, a Python software package designed for underwater mosaicking applications to manipulate and process seabed video data. The library aims to simplify and enhance the process of creating mosaics from underwater videos, allowing for improved exploration and analysis of seabed environments for marine science applications (such as bottom feature classification and biodiversity monitoring). The library offers various functionalities, including reading input videos, colour and contrast balancing, image resizing, image registration using feature detection and description, transformation estimation, homography transformations, visual-inertial alignment, and mosaic generation. Moreover, version 2 of the library contains an extensible set of classes to allow advanced users to develop their own mosaicking applications, with support for CUDA acceleration on NVIDIA hardware. The library is currently used for several applications to map the seabottom and in support of fish stock assessment procedures and biodiversity analyses. The software’s capabilities are demonstrated with examples showcasing the various features.

Synthetic images ar---e created using computer graphics modeling and artificial intelligence techniques, referred to as deepfakes. They modify human features by using generative models and deep learning algorithms, posing risks violations of social media regulations and spread false information. To address these concerns, the study proposed an improved generative adversarial network (GAN) model which improves accuracy while differentiating between real and fake images focusing on data augmentation and label smoothing strategies for GAN training. The study utilizes a dataset containing human faces and employs DCGAN (deep convolutional generative adversarial network) as the base model. In comparison with the traditional GANs, the proposed GAN outperform in terms of frequently used metrics i.e., Fréchet Inception Distance (FID) and accuracy. The model effectiveness is demonstrated through evaluation on the Flickr-Faces Nvidia dataset and Fakefaces d--ataset, achieving an FID score of 55.67, an accuracy of 98.82%, and an F1-score of 0.99 in detection. This study optimizes the model parameters to achieve optimal parameter settings. This study fine-tune the model parameters to reach optimal settings, thereby reducing risks in synthetic image generation. The article introduces an effective framework for both image manipulation and detection. [ABSTRACT FROM AUTHOR]

The modern society generates vast amounts of digital content, whose credibility plays a pivotal role in shaping public opinion and decision-making processes. The rapid development of social networks and generative technologies, such as deepfakes, significantly increases the risk of disinformation through image manipulation. This article aims to review methods for verifying images’ integrity, particularly through deep learning techniques, addressing both passive and active approaches. Their effectiveness in various scenarios has been analyzed, highlighting their advantages and limitations. This study reviews the scientific literature and research findings, focusing on techniques that detect image manipulations and localize areas of tampering, utilizing both statistical properties of images and embedded hidden watermarks. Passive methods, based on analyzing the image itself, are versatile and can be applied across a broad range of cases; however, their effectiveness depends on the complexity of the modifications and the characteristics of the image. Active methods, which involve embedding additional information into the image, offer precise detection and localization of changes but require complete control over creating and distributing visual materials. Both approaches have their applications depending on the context and available resources. In the future, a key challenge remains the development of methods resistant to advanced manipulations generated by diffusion models and further leveraging innovations in deep learning to protect the integrity of visual content.

With the flourishing development of generative models, image manipulation is receiving increasing attention. Rather than text modality, several elegant designs have delved into leveraging audio to manipulate images. However, existing methodologies mainly focus on image generation conditional on semantic alignment, ignoring the vivid affective information depicted in the audio. We propose an Emotion-aware StyleGAN Manipulator (EmoStyle), a framework where affective information from audio can be explicitly extracted and further utilized during image manipulation. Specifically, we first leverage the multi-modality model ImageBind for initial cross-modal retrieval between images and music, and select the music-related image for further manipulation. Simultaneously, by extracting sentiment polarity from the lyrics of the audio, we generate an emotionally rich auxiliary music branch to accentuate the affective information. We then leverage pre-trained encoders to encode audio and the audio-related image into the same embedding space. With the aligned embeddings, we manipulate the image via a direct latent optimization method. We conduct objective and subjective evaluations on the generated images, and our results show that our framework is capable of generating images with specified human emotions conveyed in the audio. [ABSTRACT FROM AUTHOR]

Synthetic images ar­­­e created using computer graphics modeling and artificial intelligence techniques, referred to as deepfakes. They modify human features by using generative models and deep learning algorithms, posing risks violations of social media regulations and spread false information. To address these concerns, the study proposed an improved generative adversarial network (GAN) model which improves accuracy while differentiating between real and fake images focusing on data augmentation and label smoothing strategies for GAN training. The study utilizes a dataset containing human faces and employs DCGAN (deep convolutional generative adversarial network) as the base model. In comparison with the traditional GANs, the proposed GAN outperform in terms of frequently used metrics i.e., Fréchet Inception Distance (FID) and accuracy. The model effectiveness is demonstrated through evaluation on the Flickr-Faces Nvidia dataset and Fakefaces d­­ataset, achieving an FID score of 55.67, an accuracy of 98.82%, and an F1-score of 0.99 in detection. This study optimizes the model parameters to achieve optimal parameter settings. This study fine-tune the model parameters to reach optimal settings, thereby reducing risks in synthetic image generation. The article introduces an effective framework for both image manipulation and detection.

Generative Adversarial Networks (GANs) have revolutionized the field of deep learning by enabling the production of high-quality synthetic data. However, the effectiveness of GANs largely depends on the size and quality of training data. In many real-world applications, collecting large amounts of high-quality training data is time-consuming, and expensive. Accordingly, in recent years, GAN models that use limited data have begun to be developed. In this study, we propose a GAN model that can learn from a single training image. Our model is based on the principle of multiple GANs operating sequentially at different scales, where each GAN learns the features of the training image and transfers them to the next GAN, ultimately generating examples with different realistic structures at the final scale. In our model, we utilized a selfattention and new scaling method to increase the realism and quality of the generated images. The experimental results show that our model performs image generation successfully. In addition, we demonstrated the robustness of our model by testing it in different image manipulation applications. As a result, our model can successfully produce realistic, high-quality, diverse images from a single training image, providing short training time and good training stability. [ABSTRACT FROM AUTHOR]

We present a novel framework for simulating time-varying decay effects for natural images. Conventional methods assume the input image includes enough decay information and uses the color or texture information of the decayed regions to transfer its effect to the non-decayed regions. Unlike these approaches, our framework generates diverse patterns of decay effects by leveraging a decay effects generator network without referencing the decay features of the input image, which allows us to handle more general images with non-decayed objects. Our decay generator network is formed by a style-based generative adversarial network with an arbitrary-sized stationary texture generation mechanism that allows us to synthesize various sizes of decay textures. This arbitrary-sized stationary texture generation is necessary to synthesize photo-realistic decay effects since the appropriate resolutions of the decay textures depend on those of the target objects. We construct a novel decay texture image dataset that contains various types of decay texture images, such as mossy and rust, to train the decay generator network. We show that our framework is able to synthesize diverse decay effects on various non-decayed objects without using additional decayed object images.

Image rotation makes fake images more realistic in geometric perspective, and the performance of existing JPEG image rotation angle estimation algorithms is easily disturbed by block artifacts and the image block size. It remains an imminent and challenging work for the rotation angle interval [1°,15°]. This study presents an effective algorithm to estimate the rotation angle of JPEG images based on block artifact spectral analysis. Firstly, the edges of the image are extracted and removed using a variant colony algorithm to highlight block effects. Secondly, the effect of image texture is further mitigated by cross-differentiation. Thirdly, extraneous peaks are removed in the Fourier transform domain by setting reasonable thresholds and using a Gaussian high-pass filter to reduce interference. Finally, the amplitude component of the Fourier spectrum is projected into the polar coordinates, and the polar angle corresponding to the peak in the polar coordinates is the estimation of rotation angle we need. The experimental results show that the average absolute error of the method is lower than that of the existing methods for the detection of small-size JPEG images whose rotation angle lies within the [1°,15°] interval. Besides, when the compression quality factor is gradually reduced, the performance of the proposed method is still better than the existing methods, and the robustness of the proposed method to JPEG compression is better. [ABSTRACT FROM AUTHOR]

The exponential surge in specialized image editing software has intensified visual forgery, with splicing attacks emerging as a popular forgery technique. In this context, Siamese neural networks are a remarkable tool in pattern identification for detecting image manipulations. This paper introduces a deep learning approach for splicing detection based on a Siamese neural network tailored to identifying manipulated image regions. The Siamese neural network learns unique features of specific image areas and detects tampered regions through feature comparison. This architecture employs two identical branches with shared weights and image features to compare image blocks and identify tampered areas. Subsequently, a K-means algorithm is applied to identify similar centroids and determine the precise localization of duplicated regions in the image. The experimental results encompass various splicing attacks to assess effectiveness, demonstrating a high accuracy of 98.6% and a precision of 97.5% for splicing manipulation detection. This study presents an advanced splicing image forgery detection and localization algorithm, showcasing its efficacy through comprehensive experiments.

With the flourishing development of generative models, image manipulation is receiving increasing attention. Rather than text modality, several elegant designs have delved into leveraging audio to manipulate images. However, existing methodologies mainly focus on image generation conditional on semantic alignment, ignoring the vivid affective information depicted in the audio. We propose an Emotion-aware StyleGAN Manipulator (EmoStyle), a framework where affective information from audio can be explicitly extracted and further utilized during image manipulation. Specifically, we first leverage the multi-modality model ImageBind for initial cross-modal retrieval between images and music, and select the music-related image for further manipulation. Simultaneously, by extracting sentiment polarity from the lyrics of the audio, we generate an emotionally rich auxiliary music branch to accentuate the affective information. We then leverage pre-trained encoders to encode audio and the audio-related image into the same embedding space. With the aligned embeddings, we manipulate the image via a direct latent optimization method. We conduct objective and subjective evaluations on the generated images, and our results show that our framework is capable of generating images with specified human emotions conveyed in the audio.

The retouching and altering of portrait photographs used to require extreme professionalism from the artists, thus it was a privilege for photographers. However, thanks to modern image editing techniques, the process has become completely automated, to the point where it is no longer uncommon to find artificial intelligence-based software to perform portrait editing. The present research uses a self-report questionnaire to investigate the functioning of an image editing software that uses artificial intelligence to anatomically transform portrait photos beautiful at one push of a button. Participants are asked to give their opinions on pairs of photographs, one of which is an original, unedited picture, meanwhile the other is its idealised version created by the artificial intelligence-based software. The results of the research showed that the viewer's perception of automatic image retouching was not influenced by the gender or age of the model, nor by the age of the recipients. However, the participants’ first judgement of the models’ beauty influenced their attitude towards both the model and the photograph. In most cases, the photo version that participants considered more beautiful was the one they would have preferred to see both in their own social media news feed and on the cover of a magazine, furthermore, more people would have preferred to meet the model based on the photo they thought to be beautiful. The research also reveals that viewers tend to associate negative emotions with image manipulation. Although the enhancement of photographs is a phenomenon that media users regularly encounter, not much research have been conducted on the ideal of beauty in relation to AI-technology. The current study aims to contribute to filling the aforementioned research gap.

Images can be maliciously manipulated to hide content or duplicate certain objects. Detecting an elaborate copy-move forgery is very challenging for both humans and machines, and current methods cannot detect copy-move images with the precision required, especially for pixel-level tampered images, which is a challenge for the current existing methods. In this paper we present our own dataset (CMF58K) - the first pixel-level copy-move dataset, which consists of 580,000 images covering copy-move tampered objects in various life scenes with more than 32 object classes. Furthermore, we propose a network for detecting and locating copy-move forgeries: Efficient and Robust Identification Network (ERINet). It mainly includes four main modules: the efficient feature pyramid network (EFPN), the residual receptive field block (RRFB), the hierarchical decoding identification (HDI), and the cascaded group-reversal attention (GRA) blocks. Considering the inevitable external factors of rotation, scaling, blurring, compression and noise can hide traces of tampering and increase the difficulty of detection, we applied MaxBlurPool to our network and obtained a strong robustness. ERINet outperforms various state-of-the-art manipulation detection baselines on four image manipulation datasets. The inference speed is ∼ 49 fps on a single GPU without I/O time on the test dataset. [ABSTRACT FROM AUTHOR]

Abstract Background Safe and accurate execution of surgeries to date mainly rely on preoperative plans generated based on preoperative imaging. Frequent intraoperative interaction with such patient images during the intervention is needed, which is currently a cumbersome process given that such images are generally displayed on peripheral two-dimensional (2D) monitors and controlled through interface devices that are outside the sterile filed. This study proposes a new medical image control concept based on a Brain Computer Interface (BCI) that allows for hands-free and direct image manipulation without relying on gesture recognition methods or voice commands. Method A software environment was designed for displaying three-dimensional (3D) patient images onto external monitors, with the functionality of hands-free image manipulation based on the user’s brain signals detected by the BCI device (i.e., visually evoked signals). In a user study, ten orthopedic surgeons completed a series of standardized image manipulation tasks to navigate and locate predefined 3D points in a Computer Tomography (CT) image using the developed interface. Accuracy was assessed as the mean error between the predefined locations (ground truth) and the navigated locations by the surgeons. All surgeons rated the performance and potential intraoperative usability in a standardized survey using a five-point Likert scale (1 = strongly disagree to 5 = strongly agree). Results When using the developed interface, the mean image control error was 15.51 mm (SD: 9.57). The user's acceptance was rated with a Likert score of 4.07 (SD: 0.96) while the overall impressions of the interface was rated as 3.77 (SD: 1.02) by the users. We observed a significant correlation between the users' overall impression and the calibration score they achieved. Conclusions The use of the developed BCI, that allowed for a purely brain-guided medical image control, yielded promising results, and showed its potential for future intraoperative applications. The major limitation to overcome was noted as the interaction delay.

Fake or manipulated images propagated through the Web and social media have the capacity to deceive, emotionally distress, and influence public opinions and actions. Yet few studies have examined how individuals evaluate the authenticity of images that accompany online stories. This article details a 6-batch large-scale online experiment using Amazon Mechanical Turk that probes how people evaluate image credibility across online platforms. In each batch, participants were randomly assigned to 1 of 28 news-source mockups featuring a forged image, and they evaluated the credibility of the images based on several features. We found that participants’ Internet skills, photo-editing experience, and social media use were significant predictors of image credibility evaluation, while most social and heuristic cues of online credibility (e.g. source trustworthiness, bandwagon, intermediary trustworthiness) had no significant impact. Viewers’ attitude toward a depicted issue also positively influenced their credibility evaluation.

The growing ease with which digital images can be convincingly manipulated and widely distributed on the Internet makes viewers increasingly susceptible to visual misinformation and deception. In situations where ill-intentioned individuals seek to deliberately mislead and influence viewers through fake online images, the harmful consequences could be substantial. We describe an exploratory study of how individuals react, respond to, and evaluate the authenticity of images that accompany online stories in Internet-enabled communications channels. Our preliminary findings support the assertion that people perform poorly at detecting skillful image manipulation, and that they often fail to question the authenticity of images even when primed regarding image forgery through discussion. We found that viewers make credibility evaluation based mainly on non-image cues rather than the content depicted. Moreover, our study revealed that in cases where context leads to suspicion, viewers apply post-hoc analysis to support their suspicions regarding the authenticity of the image.

The growing ease with which digital images can be convincingly manipulated and widely distributed on the Internet makes viewers increasingly susceptible to visual misinformation and deception. In situations where ill-intentioned individuals seek to deliberately mislead and influence viewers through fake online images, the harmful consequences could be substantial. We describe an exploratory study of how individuals react, respond to, and evaluate the authenticity of images that accompany online stories in Internet-enabled communications channels. Our preliminary findings support the assertion that people perform poorly at detecting skillful image manipulation, and that they often fail to question the authenticity of images even when primed regarding image forgery through discussion. We found that viewers make credibility evaluation based mainly on non-image cues rather than the content depicted. Moreover, our study revealed that in cases where context leads to suspicion, viewers apply post-hoc analysis to support their suspicions regarding the authenticity of the image.

Artificial Intelligence (AI) is a phenomenon that has become embedded in human life, and this symbiotic relationship between technology and humanity is here to stay. One such use of AI is deepfakes. The use of AI for deepfakes is arguably one of the most controversial topics because it raises ethical issues. Deepfakes are images or recordings that have been convincingly altered and manipulated to misrepresent someone as doing or saying something that they did not actually do or say. These manipulations thrive in the political arena and recently in the pornography industry, in which women's faces are masked onto other bodies to create video illusions that cause non-consensual sexual-image abuse and other harms. It is no surprise that the malicious use of deepfake technology has prompted regulatory legislation like the United States National Defense Authorization Act (NDAA), and the recent ratification of amendments to the Digital Services Act (DSA) on criminalizing malicious deepfakes. Scholars, advocates, and victims continue to call for more specific and stricter laws to regulate deepfakes and assign penalties for non-adherence. This paper presents a timely analysis of deepfake pornography as a type of image-based sexual abuse, and of the position of the law on malicious use of deepfake technology. Data protection concerns under the General Data Protection Regulation, and policy recommendations and measures for redress, control, and eradication are also addressed. [ABSTRACT FROM AUTHOR]

With the increasing visual realism of computer-graphics (CG) images generated by advanced rendering engines, the distinction between CG images and natural images (NIs) has become an important research problem in the image forensics community. Previous research works mainly focused on the conventional supervised learning framework, which usually requires a good quantity of labeled data for training. To our knowledge, we study, for the first time in the literature, the utility of the self-supervised learning mechanism for the forensic classification of CG images and NIs. The idea is to make use of a large number of readily available unlabeled data, along with a self-supervised training procedure on a well-designed pretext task for which labels can be generated in an automatic and convenient way without human manual labeling effort. Differing from existing self-supervised methods, based on pretext tasks targeted at image understanding, or based on contrastive learning, we propose carrying out self-supervised training on a forensics-oriented pretext task of classifying authentic images and their modified versions after applying various manipulations. Experiments and comparisons showed the effectiveness of our method for solving the CG forensics problem under different evaluation scenarios. Our proposed method outperformed existing self-supervised methods in all experiments. It could sometimes achieve comparable, or better, performance. compared with a state-of-the-art fully supervised method under difficult evaluation scenarios with data scarcity and a challenging forensic problem. Our study demonstrates the utility and potential of the self-supervised learning mechanism for image forensics applications. [ABSTRACT FROM AUTHOR]

Image forensics is an investigation of digital images to identify manipulations that have been done on them. Nowadays, due to the availability of different low-cost devices for capturing images, digital images are gaining quite a bit of popularity. It occurs frequently that these images are manipulated by mistake or on purpose, resulting in inaccurate information being presented by the image. There is a need to develop techniques to identify forgeries present in digital images used by the media, in court trials, and for maintaining visual records, since digital images are commonly used as evidence through the media, in court, and for maintaining record keeping. A detailed review of various image forgery detection techniques is presented in this article including comparisons between the various methods, pros and cons, and results obtained during the experimentation. [ABSTRACT FROM AUTHOR]

Image manipulation of the human face is a trending topic of image forgery, which is done by transforming or altering face regions using a set of techniques to accomplish desired outputs. Manipulated face images are spreading on the internet due to the rise of social media, causing various societal threats. It is challenging to detect the manipulated face images effectively because (i) there has been a limited number of manipulated face datasets because most datasets contained images generated by GAN models; (ii) previous studies have mainly extracted handcrafted features and fed them into machine learning algorithms to perform manipulated face detection, which was complicated, error-prone, and laborious; and (iii) previous models failed to prove why their model achieved good performances. In order to address these issues, this study introduces a large face manipulation dataset containing vast variations of manipulated images created and manually validated using various manipulation techniques. The dataset is then used to train a fine-tuned RegNet model to detect manipulated face images robustly and efficiently. Finally, a manipulated region analysis technique is implemented to provide some in-depth insights into the manipulated regions. The experimental results revealed that the RegNet model showed the highest classification accuracy of 89% on the proposed dataset compared to standard deep learning models. [ABSTRACT FROM AUTHOR]

The ability to edit an image using intuitive commands and primitives is a desired feature for any image editing software. In this paper, we combine recent results in medial axes with the well-established morphological tree representations to develop an interactive image editing tool that provides global and local image manipulation using high-level primitives. We propose a new way to render interactive morphological trees using icicle plots and introduce different ways of manipulating spline-based medial axis transforms for grayscale and colored image editing. Different applications of the tool, such as watermark removal, image deformation, dataset augmentation for machine learning, artistic illumination manipulation, image rearrangement, and clothing design, are described and showcased on examples. [Display omitted] • A tool using spline-based medial axis transforms for image editing. • A new way to render interactive morphological trees using icicle plots. • Watermark removal and image deformation applications. • Dataset augmentation and image rearrangement applications. [ABSTRACT FROM AUTHOR]

Due to the high flexibility and conformity to people's usage habits, text description has been widely used in image synthesis research recently and has achieved many encouraging results. However, the text can only determine the basic content of the generated image and cannot determine the specific shape of the synthesized object, which leads to poor practicability. More importantly, the current text-to-image synthesis research cannot use new text descriptions to further modify the synthesis result. To solve these problems, this paper proposes a text-guided customizable image synthesis and manipulation method. The proposed method synthesizes the corresponding image based on the text and contour information at first. It then modifies the synthesized content based on the new text to obtain a satisfactory result. The text and contour information in the proposed method determine the specific content and object shape of the desired composite image, respectively. Aside from that, the input text, contour, and subsequent new text for content modification can be manually input, which significantly improves the artificial controllability in the image synthesis process, making the entire method superior to other methods in flexibility and practicability. Experimental results on the Caltech-UCSD Birds-200-2011 (CUB) and Microsoft Common Objects in Context (MS COCO) datasets demonstrate our proposed method's feasibility and versatility. [ABSTRACT FROM AUTHOR]

Copyright of Etkileşim: Academic Journal of Uskudar University Faculty of Communication is the property of Etkilesim and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

There is a long history of tourists substantiating their visits to a destination through the purchase of portraits that show them against a backdrop of the local setting. While its initial expression in the form of paintings was confined to the social elite who could afford to commission and sit for an artist, the advent of photography democratized the process, enabling the aspiring middle classes to partake in the custom. While some tourists took their own photographs, the majority relied on local photographers who offered their services in studio and open-air settings. Smaller-sized images, such as Cartes de Visite (2.5″ × 4″) and Cabinet Cards (4.5″ × 6.5″), could be enclosed with letters to family and social circles, thus providing proof of visits while the voyage was still in progress. The development of picture postcards as a postal item in the 1890s, coupled with the manufacture of precut photographic paper with preprinted address fields, revolutionized tourist portraiture. Photographers could set-up outside tourist attractions, where tourists could have their portrait taken with formulaic framing against a canonized background. Efficient production flows meant that tourists could pick up their printed portraits, ready for mailing within an hour. Using examples of San Marc’s Basilica in Venice (Italy), as well as Ostrich Farms in California and Florida (U.S.A.), this paper contextualizes the production and consumption of such commercial tourist portraits as objects of social validation. It discusses their ability to situate the visitor in locales iconic of the destination, substantiating their presence and validating their experience. Given the speed of production (within an hour) and their ability to be immediately mailed through the global postal network, such images were the precursor of the modern-day ‘selfies’ posted on social media.

A postcard ‘craze’ engulfed the developed world and colonial world during the end of the nineteenth and the beginning of the twentieth century, when over 2.3 million picture postcards were mailed in 1904 alone. Formally published picture postcards can provide a rich source of information for heritage studies as they depict landscape scenery, towns, individual buildings and public plantings such as parks. The evidentiary value of late nineteenth and early twentieth century postcards depends on the veracity of the depicted image. While based on photographs, processes of postcard production allowed the publisher to modify the original imagery to improve the messaging entailed in the image. Modes of image manipulation, such as retouching, can sufficiently alter the content of the image to create limitations to using published postcard imagery as a tool for historic landscape and building analysis. This is the first paper to systematically discuss the process of postcard production and the manipulation of images depicted on the view size of picture postcards. It demonstrates that where evidentiary emphasis is placed on postcard images, it is imperative that a systematic search for variants is carried out.

In this paper, we present a novel Hybrid Generative Adversarial Network (HGAN) inversion framework that enables facial images to be rapidly inverted while preserving identity and personality characteristics. Accurate inversion of facial images requires high precision in computer vision and is critical to the success of future facial manipulations (age progression, regression, accessory, and hair stylization). However, existing methods often fail to preserve the personality characteristics of the real image, negatively affecting the accuracy of manipulations. In this context, our key contribution lies in using a transformer-based strategy to initiate the generator, which effectively models spatial relationships for detailed image processing. This approach is innovative because it leverages transformer structures to enhance image inversion tasks. Additionally, we introduce a novel loss function to enhance convergence speed and reliability, ensuring high accuracy in identity and personality trait preservation. Experimental results show that our method achieves a reconstruction accuracy of 93% and improves inversion time by 86%. This advancement could significantly impact facial manipulation technologies, laying the foundation for a technological breakthrough with potential applications in secure digital authentication systems and personal data protection. Our method may have a significant impact on privacy and security in future studies, contributing to the development of secure digital authentication systems and enhancing the protection of personal data. Therefore, our work is crucial for advancing the field of facial image manipulation and ensuring the privacy and security of personal data. • Hybrid GAN Inversion Method: Our newly hybrid GAN inversion method enables fast and accurate inversion of face images. • Fusion of Encoder and Optimization-Based Methods: Method combines the advantages of encoder and optimization inversion methods. • High Speed and Accuracy Inversion: Our model operates at high speed and achieves accuracy rates of up to 93%. • Our optimization strategies save time with fewer iterations, achieving excellent inversion results. [ABSTRACT FROM AUTHOR]

The field of scientific image integrity presents a challenging research bottleneck given the lack of available datasets to design and evaluate forensic techniques. The sensitivity of data also creates a legal hurdle that restricts the use of real-world cases to build any accessible forensic benchmark. In light of this, there is no comprehensive understanding on the limitations and capabilities of automatic image analysis tools for scientific images, which might create a false sense of data integrity. To mitigate this issue, we present an extendable open-source algorithm library that reproduces the most common image forgery operations reported by the research integrity community: duplication, retouching, and cleaning. We create a large scientific forgery image benchmark (39,423 images) with enriched ground truth using this library and realistic scientific images. All figures within the benchmark are synthetically doctored using images collected from creative commons sources. While collecting the source images, we ensured that the they did not present any suspicious integrity problems. Because of the high number of retracted papers due to image duplication, this work evaluates the state-of-the-art copy-move detection methods in the proposed dataset, using a new metric that asserts consistent match detection between the source and the copied region. All evaluated methods had a low performance in this dataset, indicating that scientific images might need a specialized copy-move detector. The dataset and source code are available at . [ABSTRACT FROM AUTHOR]

Background: Safe and accurate execution of surgeries to date mainly rely on preoperative plans generated based on preoperative imaging. Frequent intraoperative interaction with such patient images during the intervention is needed, which is currently a cumbersome process given that such images are generally displayed on peripheral two-dimensional (2D) monitors and controlled through interface devices that are outside the sterile filed. This study proposes a new medical image control concept based on a Brain Computer Interface (BCI) that allows for hands-free and direct image manipulation without relying on gesture recognition methods or voice commands.Method: A software environment was designed for displaying three-dimensional (3D) patient images onto external monitors, with the functionality of hands-free image manipulation based on the user's brain signals detected by the BCI device (i.e., visually evoked signals). In a user study, ten orthopedic surgeons completed a series of standardized image manipulation tasks to navigate and locate predefined 3D points in a Computer Tomography (CT) image using the developed interface. Accuracy was assessed as the mean error between the predefined locations (ground truth) and the navigated locations by the surgeons. All surgeons rated the performance and potential intraoperative usability in a standardized survey using a five-point Likert scale (1 = strongly disagree to 5 = strongly agree).Results: When using the developed interface, the mean image control error was 15.51 mm (SD: 9.57). The user's acceptance was rated with a Likert score of 4.07 (SD: 0.96) while the overall impressions of the interface was rated as 3.77 (SD: 1.02) by the users. We observed a significant correlation between the users' overall impression and the calibration score they achieved.Conclusions: The use of the developed BCI, that allowed for a purely brain-guided medical image control, yielded promising results, and showed its potential for future intraoperative applications. The major limitation to overcome was noted as the interaction delay. [ABSTRACT FROM AUTHOR]

Digital image forensics has become necessary as an emerging technology. Images can be adulterated effortlessly using image tools. The latest techniques are available to detect whether an image is adulterated by a particular operator. Most of the existing techniques are suitable for high resolution and manipulated images by a single operator. In a real scenario, multiple operators are applied to manipulate the image many times. In this paper, a robust moderate-sized convolutional neural network is proposed to identify manipulation operators and also the operator's sequence for two operators in particular. The proposed bottleneck approach is used to make the network deeper and reduce the computational cost. Only one pooling layer, called a global averaging pooling layer, is utilized to retain the maximum flow of information and to avoid the overfitting issue between the layers. The proposed network is also robust against low resolution and JPEG compressed images. Even though the detection of the operator is challenging due to the limited availability of statistical information in low resolution and JPEG compressed images, the proposed model can also detect an operator with different parameters and compression quality factors that are not considered in training. [ABSTRACT FROM AUTHOR]

A new style transfer-based image manipulation framework combining generative networks and style transfer networks is presented in this paper. Unlike conventional style transfer tasks, we tackle a new task, text-guided image manipulation. We realize style transfer-based image manipulation that does not require any reference style images and generate a style image from the user’s input sentence. In our method, since an initial reference input sentence for a content image can automatically be given by an image-to-text model, the user only needs to update the reference sentence. This scheme can help users when they do not have any images representing the desired style. Although this text-guided image manipulation is a new challenging task, quantitative and qualitative comparisons showed the superiority of our method.

The importance and relevance of digital-image forensics has attracted researchers to establish different techniques for creating and detecting forgeries. The core category in passive image forgery is copy–move image forgery that affects the originality of image by applying a different transformation. In this paper, a frequency-domain image-manipulation method is presented. The method exploits the localized nature of discrete wavelet transform (DWT) to attain the region of the host image to be manipulated. Both patch and host image are subjected to DWT at the same level l to obtain 3 l + 1 sub-bands, and each sub-band of the patch is pasted to the identified region in the corresponding sub-band of the host image. Resulting manipulated host sub-bands are then subjected to inverse DWT to obtain the final manipulated host image. The proposed method shows good resistance against detection by two frequency-domain forgery detection methods from the literature. The purpose of this research work is to create a forgery and highlight the need to produce forgery detection methods that are robust against malicious copy–move forgery. [ABSTRACT FROM AUTHOR]

With the increasing visual realism of computer-graphics (CG) images generated by advanced rendering engines, the distinction between CG images and natural images (NIs) has become an important research problem in the image forensics community. Previous research works mainly focused on the conventional supervised learning framework, which usually requires a good quantity of labeled data for training. To our knowledge, we study, for the first time in the literature, the utility of the self-supervised learning mechanism for the forensic classification of CG images and NIs. The idea is to make use of a large number of readily available unlabeled data, along with a self-supervised training procedure on a well-designed pretext task for which labels can be generated in an automatic and convenient way without human manual labeling effort. Differing from existing self-supervised methods, based on pretext tasks targeted at image understanding, or based on contrastive learning, we propose carrying out self-supervised training on a forensics-oriented pretext task of classifying authentic images and their modified versions after applying various manipulations. Experiments and comparisons showed the effectiveness of our method for solving the CG forensics problem under different evaluation scenarios. Our proposed method outperformed existing self-supervised methods in all experiments. It could sometimes achieve comparable, or better, performance. compared with a state-of-the-art fully supervised method under difficult evaluation scenarios with data scarcity and a challenging forensic problem. Our study demonstrates the utility and potential of the self-supervised learning mechanism for image forensics applications.

Text-guided image editing (TIE) seeks to manipulate images by the guidance of language. However, the existing TIE methods always overlook the target-irrelevant pixels and the editing may make the background discolored, distorted, or partially disappear. To overcome this problem, we propose a novel TIE method named FocusGAN, which edits the text-relevant pixels precisely as well as keeps the background invariant. Specifically, we build a network of two stages. In each stage, we first construct a channel-wise subject focusing attention to make the generator focus on the sub-region that best matches each word. Then, the word-level discriminator provides fine-grained feedback by correlating words with image areas, so that the generator can manipulate specific visual attributes without affecting the background. Last, we propose a background-keeping cyclic loss to further improve the invariance of the background and to encourage the edit of the subject that matches the given text. Experiments on CUB and Oxford datasets demonstrate that our approach can effectively keep the background invariant in manipulating images using natural language descriptions. [ABSTRACT FROM AUTHOR]

There is a long history of tourists substantiating their visits to a destination through the purchase of portraits that show them against a backdrop of the local setting. While its initial expression in the form of paintings was confined to the social elite who could afford to commission and sit for an artist, the advent of photography democratized the process, enabling the aspiring middle classes to partake in the custom. While some tourists took their own photographs, the majority relied on local photographers who offered their services in studio and open-air settings. Smaller-sized images, such as Cartes de Visite (2.5″ × 4″) and Cabinet Cards (4.5″ × 6.5″), could be enclosed with letters to family and social circles, thus providing proof of visits while the voyage was still in progress. The development of picture postcards as a postal item in the 1890s, coupled with the manufacture of precut photographic paper with preprinted address fields, revolutionized tourist portraiture. Photographers could set-up outside tourist attractions, where tourists could have their portrait taken with formulaic framing against a canonized background. Efficient production flows meant that tourists could pick up their printed portraits, ready for mailing within an hour. Using examples of San Marc's Basilica in Venice (Italy), as well as Ostrich Farms in California and Florida (U.S.A.), this paper contextualizes the production and consumption of such commercial tourist portraits as objects of social validation. It discusses their ability to situate the visitor in locales iconic of the destination, substantiating their presence and validating their experience. Given the speed of production (within an hour) and their ability to be immediately mailed through the global postal network, such images were the precursor of the modern-day 'selfies' posted on social media. [ABSTRACT FROM AUTHOR]

We describe a method for detecting physical inconsistencies in lighting from the shading and shadows in an image. This method imposes a multitude of shading- and shadow-based constraints on the projected location of a distant point light source. The consistency of a collection of such constraints is posed as a linear programming problem. A feasible solution indicates that the combination of shading and shadows is physically consistent, while a failure to find a solution provides evidence of photo tampering.

Due to the high flexibility and conformity to people’s usage habits, text description has been widely used in image synthesis research recently and has achieved many encouraging results. However, the text can only determine the basic content of the generated image and cannot determine the specific shape of the synthesized object, which leads to poor practicability. More importantly, the current text-to-image synthesis research cannot use new text descriptions to further modify the synthesis result. To solve these problems, this paper proposes a text-guided customizable image synthesis and manipulation method. The proposed method synthesizes the corresponding image based on the text and contour information at first. It then modifies the synthesized content based on the new text to obtain a satisfactory result. The text and contour information in the proposed method determine the specific content and object shape of the desired composite image, respectively. Aside from that, the input text, contour, and subsequent new text for content modification can be manually input, which significantly improves the artificial controllability in the image synthesis process, making the entire method superior to other methods in flexibility and practicability. Experimental results on the Caltech-UCSD Birds-200-2011 (CUB) and Microsoft Common Objects in Context (MS COCO) datasets demonstrate our proposed method’s feasibility and versatility.

Eyeglasses removal is challenging in removing different kinds of eyeglasses, e.g., rimless glasses, full-rim glasses, and sunglasses, and recovering appropriate eyes. Due to the significant visual variants, the conventional methods lack scalability. Most existing works focus on the frontal face images in the controlled environment, such as the laboratory, and need to design specific systems for different eyeglass types. To address the limitation, we propose a unified eyeglass removal model called the eyeglasses removal generative adversarial network (ERGAN), which could handle different types of glasses in the wild. The proposed method does not depend on the dense annotation of eyeglasses location but benefits from the large-scale face images with weak annotations. Specifically, we study the two relevant tasks simultaneously, that is, removing eyeglasses and wearing eyeglasses. Given two face images with and without eyeglasses, the proposed model learns to swap the eye area in two faces. The generation mechanism focuses on the eye area and invades the difficulty of generating a new face. In the experiment, we show the proposed method achieves a competitive removal quality in terms of realism and diversity. Furthermore, we evaluate ERGAN on several subsequent tasks, such as face verification and facial expression recognition. The experiment shows that our method could serve as a preprocessing method for these tasks. [ABSTRACT FROM AUTHOR]

A postcard 'craze' engulfed the developed world and colonial world during the end of the nineteenth and the beginning of the twentieth century, when over 2.3 million picture postcards were mailed in 1904 alone. Formally published picture postcards can provide a rich source of information for heritage studies as they depict landscape scenery, towns, individual buildings and public plantings such as parks. The evidentiary value of late nineteenth and early twentieth century postcards depends on the veracity of the depicted image. While based on photographs, processes of postcard production allowed the publisher to modify the original imagery to improve the messaging entailed in the image. Modes of image manipulation, such as retouching, can sufficiently alter the content of the image to create limitations to using published postcard imagery as a tool for historic landscape and building analysis. This is the first paper to systematically discuss the process of postcard production and the manipulation of images depicted on the view size of picture postcards. It demonstrates that where evidentiary emphasis is placed on postcard images, it is imperative that a systematic search for variants is carried out. [ABSTRACT FROM AUTHOR]

Transferring human motion and appearance between videos of human actors remains one of the key challenges in Computer Vision. Despite the advances from recent image-to-image translation approaches, there are several transferring contexts where most end-to-end learning-based retargeting methods still perform poorly. Transferring human appearance from one actor to another is only ensured when a strict setup has been complied, which is generally built considering their training regime's specificities. In this work, we propose a shape-aware approach based on a hybrid image-based rendering technique that exhibits competitive visual retargeting quality compared to state-of-the-art neural rendering approaches. The formulation leverages the user body shape into the retargeting while considering physical constraints of the motion in 3D and the 2D image domain. We also present a new video retargeting benchmark dataset composed of different videos with annotated human motions to evaluate the task of synthesizing people's videos, which can be used as a common base to improve tracking the progress in the field. The dataset and its evaluation protocols are designed to evaluate retargeting methods in more general and challenging conditions. Our method is validated in several experiments, comprising publicly available videos of actors with different shapes, motion types, and camera setups. The dataset and retargeting code are publicly available to the community at: https://www.verlab.dcc.ufmg.br/retargeting-motion. [ABSTRACT FROM AUTHOR]

Digital image forensics has become necessary as an emerging technology. Images can be adulterated effortlessly using image tools. The latest techniques are available to detect whether an image is adulterated by a particular operator. Most of the existing techniques are suitable for high resolution and manipulated images by a single operator. In a real scenario, multiple operators are applied to manipulate the image many times. In this paper, a robust moderate-sized convolutional neural network is proposed to identify manipulation operators and also the operator’s sequence for two operators in particular. The proposed bottleneck approach is used to make the network deeper and reduce the computational cost. Only one pooling layer, called a global averaging pooling layer, is utilized to retain the maximum flow of information and to avoid the overfitting issue between the layers. The proposed network is also robust against low resolution and JPEG compressed images. Even though the detection of the operator is challenging due to the limited availability of statistical information in low resolution and JPEG compressed images, the proposed model can also detect an operator with different parameters and compression quality factors that are not considered in training.

We describe a geometric technique to detect physically inconsistent arrangements of shadows in an image. This technique combines multiple constraints from cast and attached shadows to constrain the projected location of a point light source. The consistency of the shadows is posed as a linear programming problem. A feasible solution indicates that the collection of shadows is physically plausible, while a failure to find a solution provides evidence of photo tampering. © 2013 ACM.

Despite the great success of Generative Adversarial Networks (GANs) in synthesizing images, there lacks enough understanding of how photo-realistic images are generated from the layer-wise stochastic latent codes introduced in recent GANs. In this work, we show that highly-structured semantic hierarchy emerges in the deep generative representations from the state-of-the-art GANs like StyleGAN and BigGAN, trained for scene synthesis. By probing the per-layer representation with a broad set of semantics at different abstraction levels, we manage to quantify the causality between the layer-wise activations and the semantics occurring in the output image. Such a quantification identifies the human-understandable variation factors that can be further used to steer the generation process, such as changing the lighting condition and varying the viewpoint of the scene. Extensive qualitative and quantitative results suggest that the generative representations learned by the GANs with layer-wise latent codes are specialized to synthesize various concepts in a hierarchical manner: the early layers tend to determine the spatial layout, the middle layers control the categorical objects, and the later layers render the scene attributes as well as the color scheme. Identifying such a set of steerable variation factors facilitates high-fidelity scene editing based on well-learned GAN models without any retraining (code and demo video are available at https://genforce.github.io/higan). [ABSTRACT FROM AUTHOR]

The advent of sophisticated photo editing software has made it increasingly easier to manipulate digital images. Often visual inspection cannot definitively distinguish the resulting forgeries from authentic photographs. In response, forensic techniques have emerged to detect geometric or statistical inconsistencies that result from specific forms of photo manipulation. In this article we describe a new forensic technique that focuses on geometric inconsistencies that arise when fake reflections are inserted into a photograph or when a photograph containing reflections is manipulated. This analysis employs basic rules of reflective geometry and linear perspective projection, makes minimal assumptions about the scene geometry, and only requires the user to identify corresponding points on an object and its reflection. The analysis is also insensitive to common image editing operations such as resampling, color manipulations, and lossy compression. We demonstrate this technique with both visually plausible forgeries of our own creation and commercially produced forgeries. © 2012 ACM.

It has been suggested that luminance edges in retinal images are potential cues for glossiness perception, particularly when the perception relies on low-luminance specular regions. However, a previous study has shown only statistical correlations between luminance edges and perceived glossiness, not their causal relations. Additionally, although specular components should be embedded at various spatial frequencies depending on the micro-roughness on the object surface, it is not well understood what spatial frequencies are essential for glossiness perception on objects with different micro-roughness. To address these issues, we examined the impact of a sub-band contrast enhancement on the perceived glossiness in the two conditions of stimuli: the Full condition where the stimulus had natural specular components and the Dark condition where it had specular components only in dark regions. Object images with various degrees of surface roughness were generated as stimuli, and their contrast was increased in various spatial-frequency sub-bands. The results indicate that the enhancement of the sub-band contrast can significantly increase perceived glossiness as expected. Furthermore, the effectiveness of each spatial frequency band depends on the surface roughness in the Full condition. However, effective spatial frequencies are constant at a middle spatial frequency regardless of the stimulus surface roughness in the Dark condition. These results suggest that, for glossiness perception, our visual system depends on specular-related information embedded in high spatial frequency components but may change the dependency on spatial frequency based on the surface luminance to be judged.