Search

The visual security index (VSI) is a quantized indicator for objective visual security evaluation of selectively encrypted images. One challenging problem in current research is that the performance of VSIs is highly sensitive to the extracted features and the method of similarity measurement, and it is hard to choose appropriate handcrafted features from encrypted images, as well as to find an effective similarity measurement. In this paper, we make the first attempt to present a novel convolutional neural network-based visual security index (CNNVSI). Our proposed CNNVSI is purely data-driven and trained end-to-end. We propose three specialized designs to make the approach work for encrypted low-quality images without any handcrafted features or prior knowledge about the human vision system (HVS). First, we present a patch labeling algorithm to assign each encrypted patch a visual security score. Second, we design a multiscale attention residual network (MARNet) for feature learning. Last, we propose to fuse the learned features from plain images, encrypted images and their discrepancy images. Extensive and systematic experiments are conducted on five publicly available image databases to analyze the performance of our proposed CNNVSI, and the experimental results and their analysis demonstrate that our proposed CNNVSI significantly outperforms the existing state-of-the-art methods in terms of accuracy and stability. [ABSTRACT FROM AUTHOR]

Perceptual/selective encryption has been gaining widespread attention as an emerging technology for image privacy protection. However, few studies focus on the visual security evaluation of perceptually encrypted images, which has a significant impact on measuring the effectiveness and practicality of these encryption methods. In this paper, we propose an image importance-based visual security index (IIBVSI) by leveraging spatial contrast and texture features. Based on the characteristics of perceptually encrypted images, we present an averaged high-order gradient magnitude map to describe the spatial contrast feature and introduce a combined local amplitude map of multiple log-Gabor filters to represent the texture feature. Specifically, the multiresolution representation of an image is first created by downsampling to simulate the hierarchical property of the human visual system. Next, for each scale of image resolution, the spatial contrast and the texture feature maps are extracted from both plain and encrypted images. Similarity measurements are then conducted on these feature maps to generate the contrast and the texture similarity maps. An image importance-based pooling strategy is subsequently proposed to combine these measurements and generate a visual security score. The final IIBVSI score is computed by averaging the visual security scores of all scales of image resolution. Extensive experiments are conducted on several publicly available databases, and the results demonstrate the superiority and robustness of our proposed IIBVSI compared with existing state-of-the-art work in the low and moderate image quality ranges. [ABSTRACT FROM AUTHOR]

In this paper, we propose a deep neural network-based no-reference (NR) video quality assessment (VQA) method with spatiotemporal feature fusion and hierarchical information integration to evaluate the perceptual quality of videos. First, a feature extraction model is proposed by using 2D and 3D convolutional layers to gradually extract spatiotemporal features from raw video clips. Second, we design a hierarchical branching network to fuse multiframe features, and the feature vectors at each hierarchical level are comprehensively considered during the process of network optimization. Finally, these two modules and quality regression are synthesized into an end-to-end architecture. Experimental results obtained on benchmark VQA databases demonstrate the superiority of our method over other state-of-the-art algorithms. The source code is available online. [ABSTRACT FROM AUTHOR]

In this paper, we focus on learning semantic representations for large-scale highly abstract sketches that were produced by the practical sketch-based application rather than the excessively well dawn sketches obtained by crowd-sourcing. We propose a dual-branch CNN-RNN network architecture to represent sketches, which simultaneously encodes both the static and temporal patterns of sketch strokes. Based on this architecture, we further explore learning the sketch-oriented semantic representations in two practical settings, i.e., hashing retrieval and zero-shot recognition on million-scale highly abstract sketches produced by practical online interactions. Specifically, we use our dual-branch architecture as a universal representation framework to design two sketch-specific deep models: (i) We propose a deep hashing model for sketch retrieval, where a novel hashing loss is specifically designed to further accommodate both the abstract and messy traits of sketches. (ii) We propose a deep embedding model for sketch zero-shot recognition, via collecting a large-scale edge-map dataset and proposing to extract a set of semantic vectors from edge-maps as the semantic knowledge for sketch zero-shot domain alignment. Both deep models are evaluated by comprehensive experiments on million-scale abstract sketches produced by a global online game QuickDraw and outperform state-of-the-art competitors. [ABSTRACT FROM AUTHOR]

Image annotation aims to annotate a given image with a variable number of class labels corresponding to diverse visual concepts. In this paper, we address two main issues in large-scale image annotation: 1) how to learn a rich feature representation suitable for predicting a diverse set of visual concepts ranging from object, scene to abstract concept and 2) how to annotate an image with the optimal number of class labels. To address the first issue, we propose a novel multi-scale deep model for extracting rich and discriminative features capable of representing a wide range of visual concepts. Specifically, a novel two-branch deep neural network architecture is proposed, which comprises a very deep main network branch and a companion feature fusion network branch designed for fusing the multi-scale features computed from the main branch. The deep model is also made multi-modal by taking noisy user-provided tags as model input to complement the image input. For tackling the second issue, we introduce a label quantity prediction auxiliary task to the main label prediction task to explicitly estimate the optimal label number for a given image. Extensive experiments are carried out on two large-scale image annotation benchmark datasets, and the results show that our method significantly outperforms the state of the art. [ABSTRACT FROM AUTHOR]