Search

Advances in editing tools and compression technologies have made it possible to easily manipulate videos without leaving any visual traces and then compress them using video codecs. Among the various forging operations, fine-grained manipulations such as filtering and noise addition accompany various forgery scenarios. Detecting low-level features left in videos by such manipulations in order to spot forgeries is a challenging task. Furthermore, when fine-grained manipulations are applied to videos, the presence of compression artifacts left by codecs are also added, making it more difficult to classify the manipulations to the videos. To overcome these obstacles, we propose a dual-path network (DPN) for identifying fine-grained manipulations in H.264 videos. The DPN consists of two single-path networks: one for learning low-level features caused by manipulations and the other utilizing a discrete cosine transform (DCT) histogram for capturing block DCT-based compression artifacts. The fusion network incorporates features learned in each stream, enabling comprehensive forensic clue learning. Experimental results indicate that the proposed DPN achieves superior performance compared to comparable baselines in terms of multi-class classification. Furthermore, our work can localize the manipulated areas through temporal- and spatial-localization.

Image forensics comprises the analyses and classifications of manipulations that have been applied to images. The ability to classify various manipulations that have been employed in the process of forgery is essential. Techniques to identify multiple manipulations applied to uncompressed images have been reported thus far, but the forensic approach for JPEG images compressed with various qualities has not been proposed. In this paper, we propose the manipulation classification network (MCNet) to exploit multi-domain features of the spatial, frequency, and compression domains. The proposed MCNet learns several forensic features for each domain through a multi-stream structure and distinguishes manipulations by comprehensively analyzing the fused features. Our work jointly considers visual artifacts caused by image manipulations and compression artifacts due to JPEG compression; therefore, rich forensic features can be explored and learned in the training phase. To enable forgery analysis in the real-world environment, data were generated based on twenty types of manipulation algorithms and various compression parameters. To demonstrate the effectiveness of the proposed MCNet, extensive experiments were conducted using state-of-the-art baselines. Compared to these baselines, our proposed method outperforms in terms of multi-class manipulation classification. In addition, we experimentally proved that the fine-tuned model based on the multi-class manipulation task was effective for different forensic tasks such as DeepFake detection or integrity authentication of JPEG images.

Depth-image-based rendering (DIBR), where arbitrary views are synthesized from a center image and depth image, has received much attention in the three-dimensional (3D) research field. With advances in depth-acquisition techniques and the proliferation of 3D glasses and 3D display devices, there is a growing demand for schemes to protect the copyrights of DIBR 3D images. Digital watermarking is a typical protection technology and designing a watermarking method for DIBR 3D images is a challenging task because the synchronization of watermarks can easily be broken in the process of generating synthetic images. To address this issue, we propose a non-subsampled contourlet transform (NSCT)-based blind watermarking for DIBR 3D images. To ensure the proposed method has properties of robustness against the DIBR process, we conduct an analysis of robustness for NSCT subbands against DIBR attacks. Based on the analysis results, we select subbands that are robust against DIBR attacks and embed watermark in the low coefficients of NSCT subbands using quantization-based embedding. While ensuring robustness, the proposed method also improves the imperceptibility of watermarks by adjusting their embedding strength according to computed perceptual masking values. Through experiments, we show that the proposed method is robust against both desynchronization attacks of the DIBR process and common attacks including signal processing operations and geometric distortions. The high imperceptibility of our method is also verified by several evaluation metrics in a subjective and objective manner.

Detection of double JPEG compression is essential in the field of digital image forensics. Although double JPEG compression detection methods have greatly improved with the development of convolutional neural networks (CNNs), they rely on handcrafted features such as discrete cosine transform (DCT) histograms. In this Letter, the authors propose an end-to-end trainable 3D CNN in the DCT domain for double JPEG compression detection. Moreover, they also propose a new type of module, called feature rescaling , to insert the quantisation table into the network suitably. The experiments show that the proposed method outperforms state-of-the-art methods.

Steganalysis refers to the study of identifying hidden messages in images inserted by steganography. Although detection performance is greatly improved when adopting convolutional neural networks (CNNs), they require sophisticated tricks, such as preprocessing for suppression of image content, using absolute and truncated activation functions, and utilizing domain knowledge. These tricks help networks train stably and mitigate the convergence problem of early stages in training, but they also restrict the flexibility of CNNs, which limits their performance. In this paper, we propose a local-source enhanced residual network (LSER) with end-to-end learning. The LSER is simple in its architecture but has two distinct characteristics from previous methods. First, the LSER uses residual blocks without any normalization. We find batch normalization is an unnecessary module in our framework. Second, a local-source skip connection is added to bypass features of different levels, which allows more abundant feature representation. Moreover, the LSER exhibits state-of-the-art results compared with the existing work in both spatial and JPEG domain steganalysis. Furthermore, a simple self-ensemble method further improves its performance without any side information.

In recent years, some researchers have been interested in whether robustness and blindness can be simultaneously secured in a watermarking based on machine learning. However, achieving robustness against various attacks at once is still difficult for watermarking techniques. To address the problem, in this paper, we propose a learning framework for robust and blind watermarking based on reinforcement learning. We repeat three stages: watermark embedding, attack simulation, and weight updating. Specifically, we present image watermarking networks called WMNet using convolutional neural networks (CNNs). Two methods to embed a watermark are proposed and these two methods are based on backpropagation and autoencoder, respectively. We can optimize the robustness while carefully considering the invisibility of the watermarking system. The experimental results show that the trained WMNet captures more robust features than the current watermarking schemes, which use the frequency domain. The trade-off between the robustness and the invisibility of each technique was measured. Also, we adopt a visual masking with which we can achieve the appropriate balance between robustness and invisibility of the watermark. Our reinforcement-learning-based technique has better robustness than the existing techniques for both attacks seen in learning and unseen attacks. Due to the generalization ability of WMNet, moreover, it shows high robustness against multiple attacks and various levels of attacks which are not considered in training stage.

Detecting and localizing image splicing has become essential to fight against malicious forgery. A major challenge to localize spliced areas is to discriminate between authentic and tampered regions with intrinsic properties such as compression artifacts. We propose CAT-Net, an end-to-end fully convolutional neural network including RGB and DCT streams, to learn forensic features of compression artifacts on RGB and DCT domains jointly. Each stream considers multiple resolutions to deal with spliced object’s various shapes and sizes. The DCT stream is pretrained on double JPEG detection to utilize JPEG artifacts. The proposed method outperforms state-of-the-art neural networks for localizing spliced regions in JPEG or non-JPEG images.

Seam carving is a representative content-aware image retargeting approach to adjust the size of an image while preserving its visually prominent content. To maintain visually important content, seam-carving algorithms first calculate the connected path of pixels, referred to as the seam, according to a defined cost function and then adjust the size of an image by removing and duplicating repeatedly calculated seams. Seam carving is actively exploited to overcome diversity in the resolution of images between applications and devices; hence, detecting the distortion caused by seam carving has become important in image forensics. In this paper, we propose a convolutional neural network (CNN)-based approach to classifying seam-carving-based image retargeting for reduction and expansion. To attain the ability to learn low-level features, we designed a CNN architecture comprising five types of network blocks specialized for capturing subtle signals. An ensemble module is further adopted to both enhance performance and comprehensively analyze the features in the local areas of the given image. To validate the effectiveness of our work, extensive experiments based on various CNN-based baselines were conducted. Compared to the baselines, our work exhibits state-of-the-art performance in terms of three-class classification (original, seam inserted, and seam removed). In addition, our model with the ensemble module is robust for various unseen cases. The experimental results also demonstrate that our method can be applied to localize both seam-removed and seam-inserted areas.

Convolutional neural networks for image steganalysis demonstrate better performances with employing concepts from high-level vision tasks. The major employed concept is to use data augmentation to avoid overfitting due to limited data. To augment data without damaging the message embedding, only rotating multiples of 90 ° or horizontally flipping are used in steganalysis, which generates eight fixed results from one sample. To overcome this limitation, the authors propose BitMix, a data augmentation method for spatial image steganalysis. BitMix mixes a cover and stego image pair by swapping the random patch and generates an embedding adaptive label with the ratio of the number of pixels modified in the swapped patch to those in the cover–stego pair. The authors explore optimal hyperparameters, the ratio of applying BitMix in the mini-batch, and the size of the bounding box for swapping patch. The results reveal that using BitMix improves the performance of spatial image steganalysis and better than other data augmentation methods.

Watermarking inserts invisible data into content to protect copyright. The embedded information provides proof of authorship and facilitates the tracking of illegal distribution, etc. Current robust watermarking techniques have been proposed to preserve inserted copyright information from various attacks, such as content modification and watermark removal attacks. However, since the watermark is inserted in the form of noise, there is an inevitable effect of reducing content visual quality. In general, most robust watermarking techniques tend to have a greater effect on quality, and content creators and users are often reluctant to insert watermarks. Thus, there is a demand for a watermark that maintains maximum image quality, even if the watermark performance is slightly inferior. Therefore, we propose a watermarking technique that maximizes invisibility while maintaining sufficient robustness and data capacity to be applied in real situations. The proposed method minimizes watermarking energy by adopting curvelet domain multi-directional decomposition to maximize invisibility and maximizes robustness against signal processing attacks with a watermarking pattern suitable for curvelet transformation. The method is also robust against geometric attacks by employing the watermark detection method utilizing curvelet characteristics. The proposed method showed very good results of a 57.65 dB peak signal-to-noise ratio in fidelity tests, and the mean opinion score showed that images treated with the proposed method were hardly distinguishable from the originals. The proposed technique also showed good robustness against signal processing and geometric attacks compared with existing techniques.

Depth image based rendering (DIBR) is a promising technique for extending viewpoints with a monoscopic center image and its associated per-pixel depth map. With its numerous advantages including low-cost bandwidth, 2D-to-3D compatibility and adjustment of depth condition, DIBR has received much attention in the 3D research community. In the case of a DIBR-based broadcasting system, a malicious adversary can illegally distribute both a center view and synthesized virtual views as 2D and 3D content, respectively. To deal with the issue of copyright protection for DIBR 3D Images, we propose a scale invariant feature transform (SIFT) features based blind watermarking algorithm. To design the proposed method robust against synchronization attacks from DIBR operation, we exploited the parameters of the SIFT features: the location, scale and orientation. Because the DIBR operation is a type of translation transform, the proposed method uses high similarity between the SIFT parameters extracted from a synthesized virtual view and center view images. To enhance the capacity and security, we propose an orientation of keypoints based watermark pattern selection method. In addition, we use the spread spectrum technique for watermark embedding and perceptual masking taking into consideration the imperceptibility. Finally, the effectiveness of the presented method was experimentally verified by comparing with other previous schemes. The experimental results show that the proposed method is robust against synchronization attacks from DIBR operation. Furthermore, the proposed method is robust against signal distortions and typical attacks from geometric distortions such as translation and cropping.

Double compression (DC) involves compressing a video twice with the video codec. DC usually occurs when a compressed video is manipulated, because most videos are saved in the compression format after manipulating. Given this principle, if we can detect DC traces in videos, we can determine whether a video has been manipulated. Prior works have proposed methods for detecting DC of H.264 videos; however, they have limitations in that they can detect DC only for a limited dataset and set of compression parameters. To overcome these limitations, we propose a two-stream neural network that incorporates two components that analyze intra-coded frames and predictive frames. We explain why the proposed two-stream neural network can detect DC traces in videos more accurately than the prior method, and we demonstrate that the proposed network can detect DC videos that have various contents and parameters through extensive experiments.

Content-aware image resizing is the process of adjusting the size of an image while preserving its important content. Image resizing is used to overcome diversity in resolutions between modules, such as display devices and applications, and can thus be deliberately exploited to distort or remove original content; therefore, detecting such tampering has become an important topic in forensics. This paper proposes a deep neural network architecture to capture subtle local artifacts caused by seam-based image resizing. Unlike past approaches that only classified two classes, our approach is the first attempt to solve a given forensic task with three-class classification: original, seam insertion, and seam carving. The experimental results show that our work performs better than the handcrafted feature-based method and networks designed for different forensic tasks.

The proposed method is a watermarking method for high-quality images that achieves high invisibility and good robustness using a curvelet domain. Recently, there has been a demand for a watermarking technique that does not impair image quality, as interest in high-quality images has increased. To meet this demand, the authors adopted a method of minimising the watermark-embedding energy by using the multi-directional decomposition technique, curvelet transformation. However, if a watermark is inserted into the curvelet domain with the watermarking technique of the conventional domains, it is distorted during the forward and inverse curvelet transform. To solve this problem, they designed a watermarking method by considering the characteristics of the curvelet filter. By minimising the watermark distortion caused by the curvelet transform, high robustness is achieved with small watermarking energy. The proposed method shows very good invisibility and is difficult to distinguish the original. In terms of robustness, the correlation value increased by >80% and BER decreased by >20% compared with previous methods.