Search

Objects at different spatial positions in an image exhibit different scales. Adaptive receptive fields are expected to capture suitable ranges of context for accurate pixel level semantic prediction. Recently, atrous convolution with different dilation rates has been used to generate features of multi-scales through several branches which are then fused for prediction. However, there is a lack of explicit interaction among the branches of different scales to adaptively make full use of the contexts. In this paper, we propose a Content-Adaptive Scale Interaction Network (CASINet) to exploit the multi-scale features for scene parsing. We build CASINet based on the classic Atrous Spatial Pyramid Pooling (ASPP) module, followed by a proposed contextual scale interaction (CSI) module, and a scale adaptation (SA) module. Specifically, in the CSI module, for each spatial position of some scale, instead of being limited by a fixed set of convolutional filters that are shared across different spatial positions for feature learning, we promote the adaptivity of the convolutional filters to spatial positions. We achieve this by the context interaction among the features of different scales. The SA module explicitly and softly selects the suitable scale for each spatial position and each channel. Ablation studies demonstrate the effectiveness of the proposed modules. We achieve state-of-the-art performance on three scene parsing benchmarks Cityscapes, ADE20K and LIP.

Source camera identification is still a hard task in forensics community, especially for the case of the query images with small size. In this paper, we propose a solution to identify the source camera of the small-size images: content-adaptive fusion residual networks. According to the differences of the image contents, firstly, the images are divided into three subsets: saturation, smoothness and others. Then, we train three fusion residual networks for saturated images, smooth images, and others, separately, through transform learning. The fusion residual networks is formed with three paralleled residual networks and the difference of three residual networks lies in the convolutional kernel size of preprocessing layer. The features learned from the last residual blocks of three residual networks are fused and fed into softmax classifier. In particular, the residual networks is designed to learn better feature representation from the input data. The convolutional operation is added in preprocessing stage and three residual blocks are used. The experiment results show that the proposed method has satisfactory performances at three levels of source camera identification: brand level, model level, and device level.

This paper proposes a content-adaptive sharpening algorithm using two-dimensional (2D) FIR filters trained by pre-emphasis for various image pairs. In the learning stage, all low-quality (LQ) and high-quality (HQ) image pairs are first pre-emphasized, i.e., properly sharpened. Then selective 2D FIR filter coefficients for high-frequency synthesis are trained using the pre-emphasized LQ–HQ image pairs, and then are stored in a dictionary that resembles an LUT (look-up table). In the inference stage, each input image is pre-emphasized in the same manner as in the learning stage. The best-matched 2D filter for each LQ patch is then found in the dictionary, and an HQ patch corresponding to the input LQ patch is synthesized using the resultant 2D FIR filter. The experiment results show that the proposed algorithm visually outperforms existing ones and that the mean of absolute errors (MAEs) and MSSSIM (multi-scale structure similarity) of the proposed algorithm are about 10% to 60% lower and about 0.002–0.053 higher, respectively than those of the existing algorithms.

In this paper, a content-adaptive deblocking method is proposed to improve the visual quality of block-based DCT compressed videos. We find that the edge information obtained through the global orientation energy edge detection (OEED) on an initially deblocked image provides a robust partition of local directional features (LDFs). Based on this partition, for the directional featured region, we design corresponding filter orientation and thresholds to best preserve image details while reducing blocking artifacts; for the consecutive non-featured region, we impose extra smoothing to suppress the visually severe blocking artifacts. Experimental results demonstrate that our method effectively improves the visual quality while well maintaining the objective fidelity of block-based DCT compressed videos, compared with the H.264/AVC deblocking method and other existing directional deblocking methods.

In this paper, a method for Lagrange multiplier selection is proposed in the context of rate-distortion optimisation for wavelet-based scalable video coding targeting quality scalability. Despite the prevalence of the conventional method for Lagrange multiplier selection in hybrid video coding, the underlying formulation is not applicable to wavelet-based scalable video coding. To address the inherent challenges, a thorough analysis of the rate-distortion models for transform video coding is provided with regard to low and middle-to-high bit-rates, respectively. Based on the analysis, the models are consolidated according to experimental observations and the consolidated rate-distortion models serve as the basis for the derivation of the Lagrange multiplier. Considering the influence of the open-loop prediction structure on the rate-distortion performance, the Lagrange multiplier is initially derived for a single-targeted bit-rate. Moreover, the method for Lagrange multiplier selection in scalable video coding aiming at multiple-targeted bit-rates is proposed in a general sense of bit-rate range, varying from low to high bit-rates, building on the initially derived Lagrange multiplier for a single-targeted bit-rate. The proposed Lagrange multiplier is content adaptive and well suited for wavelet-based scalable video coding where quantisation steps are unavailable. Detailed performance evaluation of the proposed method for wavelet-based scalable video coding is provided with regard to a given targeted bit-rate and multiple-targeted bit-rates, respectively. The experimental results have demonstrated the effectiveness of the proposed Lagrange multiplier for rate-distortion optimisation considering quality scalability in wavelet-based scalable video coding.