Search

Sparse representation has been shown to be highly correlated with the visual perception of natural images, which can be characterized by a linear combination of neuronal responses in the visual cortex. Divisive normalization transform (DNT) has been proven to be an effective method in reducing statistical and perceptual dependencies for nonlinear properties in primary visual cortex. In this paper, we develop a divisively normalized sparse coding scheme, aiming to further bridge the gap between sparse representation and human visual perception. We show that such a scheme is perceptually meaningful for representing visual signals, with which the pixel-domain image representation and processing tasks can be feasibly and efficiently achieved in the divisively normalized sparse-domain. Specifically, we develop a sparse-domain similarity (SDS) index for perceptual quality evaluation, where the DNT is employed for transforming image signals into a perceptually uniform space. Furthermore, the proposed SDS index is employed to optimize the sparse coding process when representing natural images. The experimental results indicate that the SDS can provide accurate and consistent predictions of perceived image quality, and the performance of sparse coding can be significantly improved in terms of both objective and subjective quality evaluations.

A new general synthesis of pharmaceutically important azolo[1,5- a ]pyrimidines starting from widely available 3(5)-aminoazoles, aldehydes, and triethylamine is developed. The key is to enable the vinylation reaction that allows the in situ generation of elusive acyclic enamines and the subsequent annulation reaction to occur. This direct and practical strategy is capable of constructing a range of 5,6-unsubstituted pyrazolo[1,5- a ]pyrimidines and [1,2,4]triazolo[1,5- a ]pyrimidines. More importantly, this protocol provides a concise synthetic route to prepare the clinically used zaleplon.

Recent deep network-based compressive sensing (CS) methods have achieved great success. However, most of them regard different sampling matrices as different independent tasks and need to train a specific model for each target sampling matrix. Such practices give rise to inefficiency in computing and suffer from poor generalization ability. In this paper, we propose a novel COntrollable Arbitrary-Sampling neTwork, dubbed COAST, to solve CS problems of arbitrary-sampling matrices (including unseen sampling matrices) with one single model. Under the optimization-inspired deep unfolding framework, our COAST exhibits good interpretability. In COAST, a random projection augmentation (RPA) strategy is proposed to promote the training diversity in the sampling space to enable arbitrary sampling, and a controllable proximal mapping module (CPMM) and a plug-and-play deblocking (PnP-D) strategy are further developed to dynamically modulate the network features and effectively eliminate the blocking artifacts, respectively. Extensive experiments on widely used benchmark datasets demonstrate that our proposed COAST is not only able to handle arbitrary sampling matrices with one single model but also to achieve state-of-the-art performance with fast speed.

The flexible partitioning structure such as quad-tree plus binary-tree (QTBT) plays important roles in the next generation video coding standard. This paper investigates new coding unit (CU) partitioning methods to further improve the compression efficiency. In particular, we propose the extended quad-tree partitioning and asymmetric ternary-tree partitioning to further develop the principle behind the existing partitioning structure. First, a central extended quad-tree (CENTRAL-EQT) partitioning is introduced, which extends the traditional QT partitioning with central pattern and generates four elaborately designed sub-CUs with different sizes. Second, we propose a parallel extended quad-tree (PARALLEL-EQT) partitioning that allows the CU to split along a single direction, leading to four identical size sub-blocks. Third, we present an asymmetric ternary-tree (ATT) method, which splits the CU asymmetrically into three sub-blocks. The proposed new partitioning methods allow the interleaving with binary-tree partitioning for enhanced adaptability, and they can be jointly enabled to capture different characteristics of the local content. Simulation results on the JEM7-QTBT-Only platform show that averagely 4.92%, 4.81% and 5.08% BD-Rate savings are achieved for the luma component by the proposed methods under random access (RA), low-delay P (LDP) and low-delay B (LDB) configurations, respectively, with around 725% encoding time increase and negligible decoding time increase. Furthermore, experimental results also reveal that the proposed partitioning schemes are effective when cooperating with the upcoming Versatile Video Coding standard.

Recently, convolutional neural network (CNN) has attracted tremendous attention and has achieved great success in many image processing tasks. In this paper, we focus on CNN technology combined with image restoration to facilitate video coding performance and propose the content-aware CNN based in-loop filtering for high-efficiency video coding (HEVC). In particular, we quantitatively analyze the structure of the proposed CNN model from multiple dimensions to make the model interpretable and optimal for CNN-based loop filtering. More specifically, each coding tree unit (CTU) is treated as an independent region for processing, such that the proposed content-aware multimodel filtering mechanism is realized by the restoration of different regions with different CNN models under the guidance of the discriminative network. To adapt the image content, the discriminative neural network is learned to analyze the content characteristics of each region for the adaptive selection of the deep learning model. The CTU level control is also enabled in the sense of rate-distortion optimization. To learn the CNN model, an iterative training method is proposed by simultaneously labeling filter categories at the CTU level and fine-tuning the CNN model parameters. The CNN based in-loop filter is implemented after sample adaptive offset in HEVC, and extensive experiments show that the proposed approach significantly improves the coding performance and achieves up to 10.0% bit-rate reduction. On average, 4.1%, 6.0%, 4.7%, and 6.0% bit-rate reduction can be obtained under all intra, low delay, low delay P, and random access configurations, respectively.

Motion compensation has been widely employed for removing temporal redundancies in typical hybrid video coding framework. The popular video compression standards, such as H.264/AVC and HEVC, adopt the block based partitioning model to describe the motion filed due to its high compression efficiency and relatively low computational complexity. However, block based motion compensation may not align with the actual object motion boundaries, potentially limiting the compression efficiency. In view of this, we propose a three-zone segmentation based motion compensation scheme to improve the description accuracy of motion field as well as the coding efficiency. In particular, the segmentation information is implied in the reference frame instead of being explicitly signalled. Based on the segmentation information, three motion compensation zones can be identified, including one edge, one foreground, and one background zone. The foreground zone is motion compensated by the signalled motion vector of the block, and the background zone is motion compensated by the motion information implicitly derived from the local motion field. Regarding the edge zone, it is viewed as an overlapped area and the weighted compensation strategy is adopted. The proposed algorithm is implemented into the reference software VTM-1.0 of Versatile Video Coding (VVC), and simulation results show that the algorithm can achieve 1.14% and 1.06% bitrate savings for random access and lowdelay configurations, respectively.

Image quality assessment (IQA) has attracted more and more attention due to the urgent demand in image services. The perceptual-based image compression is one of the most prominent applications that require IQA metrics to be highly correlated with human vision. To explore IQA algorithms that are more consistent with human vision, several calibrated databases have been constructed. However, the distorted images in the existing databases are usually generated by corrupting the pristine images with various distortions in coarse levels, such that the IQA algorithms validated on them may be inefficient to optimize the perceptual-based image compression with fine-grained quality differences. In this paper, we construct a large-scale image database which can be used for fine-grained quality assessment of compressed images. In the proposed database, reference images are compressed at constant bitrate levels by JPEG encoders with different optimization methods. To distinguish subtle differences, the pair-wise comparison method is utilized to rank them in subjective experiments. We select 100 reference images for the proposed database, and each image is compressed into three target bitrates by four different JPEG optimization methods, such that 1200 distorted images are generated in total. Sixteen well-known IQA algorithms are evaluated and analyzed on the proposed database. With the devised fine-grained IQA database, we expect to further promote image quality assessment by shifting it from a coarse-grained stage to a fine-grained stage. The database is available at: https://sites.google.com/site/zhangxinf07/fg-iqa.

Transform and quantization account for a considerable amount of computation time in video encoding process. However, there are a large number of discrete cosine transform coefficients which are finally quantized into zeros. In essence, blocks with all zero quantized coefficients do not transmit any information, but still occupy substantial unnecessary computational resources. As such, detecting all-zero block (AZB) before transform and quantization has been recognized to be an efficient approach to speed up the encoding process. Instead of considering the hard-decision quantization (HDQ) only, in this paper, we incorporate the properties of soft-decision quantization into the AZB detection. In particular, we categorize the AZB blocks into genuine AZBs (G-AZB) and pseudo AZBs (P-AZBs) to distinguish their origins. For G-AZBs directly generated from HDQ, the sum of absolute transformed difference-based approach is adopted for early termination. Regarding the classification of P-AZBs which are generated in the sense of rate-distortion optimization, the rate-distortion models established based on transform coefficients together with the adaptive searching of the maximum transform coefficient are jointly employed for the discrimination. Experimental results show that our algorithm can achieve up to 24.16% transform and quantization time-savings with less than 0.06% RD performance loss. The total encoder time saving is about 5.18% on average with the maximum value up to 9.12%. Moreover, the detection accuracy of larger TU sizes, such as and can reach to 95% on average.

An AC-driven powder electroluminescent (EL) device has been achieved by constructing a CuO nanowire-Zn 2 GeO 4 :Mn phosphor heterogeneous junction. The CuO nanowires enhance the local electric field, resulting in electroluminescence of an oxide-based phosphor in EL devices owing to field injection at the nanowire tips. The CuO nanowire array was synthesized by an in situ thermal oxidation method at 400 °C in air and employed as an electric field enhancement layer in the EL device. The heterogeneous structures were created through drop coating of a phosphor suspension on the CuO nanowire array. The initial EL device tests show good luminescent performance with very promising brightness maintenance for over 360 h, with a loss of luminescent intensity of under 1 % at over 10 cd m -2 luminance. The fabrication method offers the prospect of simple, low-cost, large-scale EL devices with the potential to solve the limited operational lifetime of sulfide-based AC powder EL devices., (© 2018 Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim.)

Sparse representation leads to an efficient way to approximately recover a signal by the linear composition of a few bases from a learnt dictionary based on which various successful applications have been achieved. However, in the scenario of data compression, its efficiency and popularity are hindered. It is because of the fact that encoding sparsely distributed coefficients may consume more bits for representing the index of nonzero coefficients. Therefore, introducing an accurate rate constraint in sparse coding and dictionary learning becomes meaningful, which has not been fully exploited in the context of sparse representation. According to the Shannon entropy inequality, the variance of Gaussian distributed data bound its entropy, indicating the actual bitrate can be well estimated by its variance. Hence, a globally variance-constrained sparse representation (GVCSR) model is proposed in this paper, where a variance-constrained rate term is introduced to the optimization process. Specifically, we employ the alternating direction method of multipliers (ADMMs) to solve the non-convex optimization problem for sparse coding and dictionary learning, both of them have shown the state-of-the-art rate-distortion performance for image representation. Furthermore, we investigate the potential of applying the GVCSR algorithm in the practical image set compression, where the optimized dictionary is trained to efficiently represent the images captured in similar scenarios by implicitly utilizing inter-image correlations. Experimental results have demonstrated superior rate-distortion performance against the state-of-the-art methods.

We propose a novel just noticeable difference (JND) model for a screen content image (SCI). The distinct properties of the SCI result in different behaviors of the human visual system when viewing the textual content, which motivate us to employ a local parametric edge model with an adaptive representation of the edge profile in JND modeling. In particular, we decompose each edge profile into its luminance, contrast, and structure, and then evaluate the visibility threshold in different ways. The edge luminance adaptation, contrast masking, and structural distortion sensitivity are studied in subjective experiments, and the final JND model is established based on the edge profile reconstruction with tolerable variations. Extensive experiments are conducted to verify the proposed JND model, which confirm that it is accurate in predicting the JND profile, and outperforms the state-of-the-art schemes in terms of the distortion masking ability. Furthermore, we explore the applicability of the proposed JND model in the scenario of perceptually lossless SCI compression, and experimental results show that the proposed scheme can outperform the conventional JND guided compression schemes by providing better visual quality at the same coding bits.

We propose a perceptual video coding framework based on the divisive normalization scheme, which is found to be an effective approach to model the perceptual sensitivity of biological vision, but has not been fully exploited in the context of video coding. At the macroblock (MB) level, we derive the normalization factors based on the structural similarity (SSIM) index as an attempt to transform the discrete cosine transform domain frame residuals to a perceptually uniform space. We further develop an MB level perceptual mode selection scheme and a frame level global quantization matrix optimization method. Extensive simulations and subjective tests verify that, compared with the H.264/AVC video coding standard, the proposed method can achieve significant gain in terms of rate-SSIM performance and provide better visual quality.

In this paper, a motion-aligned auto-regressive (MAAR) model is proposed for frame rate up conversion, where each pixel is interpolated as the average of the results generated by one forward MAAR (Fw-MAAR) model and one backward MAAR (Bw-MAAR) model. In the Fw-MAAR model, each pixel in the to-be-interpolated frame is generated as a linear weighted summation of the pixels within a motion-aligned square neighborhood in the previous frame. To derive more accurate interpolation weights, the aligned actual pixels in the following frame are also estimated as a linear weighted summation of the newly interpolated pixels in the to-be-interpolated frame by the same weights. Consequently, the backward-aligned actual pixels in the following frame can be estimated as a weighted summation of the corresponding pixels within an enlarged square neighborhood in the previous frame. The Bw-MAAR is performed likewise except that it is operated in the reverse direction. A damping Newton algorithm is then proposed to compute the adaptive interpolation weights for the Fw-MAAR and Bw-MAAR models. Extensive experiments demonstrate that the proposed MAAR model is able to achieve superior performance than the traditional frame interpolation methods such as MCI, OBMC, and AOBMC, and it is even better than STAR model for the most test sequences with moderate or large motions.