Search

This paper proposes a deep learning method for intra prediction. Different from traditional methods utilizing some fixed rules, we propose using a fully connected network to learn an end-to-end mapping from neighboring reconstructed pixels to the current block. In the proposed method, the network is fed by multiple reference lines. Compared with traditional single line-based methods, more contextual information of the current block is utilized. For this reason, the proposed network has the potential to generate better prediction. In addition, the proposed network has good generalization ability on different bitrate settings. The model trained from a specified bitrate setting also works well on other bitrate settings. Experimental results demonstrate the effectiveness of the proposed method. When compared with high efficiency video coding reference software HM-16.9, our network can achieve an average of 3.4% bitrate saving. In particular, the average result of 4K sequences is 4.5% bitrate saving, where the maximum one is 7.4%.

Screen content coding plays an important role in many applications. Conventional reference picture management (RPM) strategies developed for natural content may not work well for screen content. This is because many regions in screen content remain static for a long time, causing a lot of repetitive contents to stay in the decoded picture buffer. The repetitive contents are not conducive to inter prediction, but still occupy valuable memory. This paper proposes a diversity-based RPM scheme for screen content coding. The concept of diversity is introduced for the reference picture set (RPS) to help formulate the RPM problem. By maximizing the diversity of RPS, more potentially better predictions are provided. Better compression performance can then be achieved. Meanwhile, the proposed scheme is nonnormative and compatible with existing video coding standards, such as High Efficiency Video Coding. The experimental results show that, for low delay screen content coding, the bit saving of the proposed scheme is 4.9% on average and up to 13.7%, without increasing encoding time.

Screen content coding plays an important role in many applications. To meet the growing demands of screen content coding, the emerging AV1 video codec incorporates several coding tools, which are specially designed for screen content utilizing its distinctive characteristics. Among these tools, the intra block copy utilizes the characteristic that repeating patterns frequently occur in screen content. This paper presents the technology of intra block copy in AV1. In particular, to efficiently search the predictor in the reconstructed regions of the current picture, AV1 uses the hash matching method at the encoder side. For the generation of hash table, a bottom-to-up manner is adopted to reduce the redundant computation and then decrease the encoding time. In addition, several constraints are involved to facilitate hardware design. Experimental results demonstrate that the intra block copy in AV1 can bring 27.1% bitrate saving for screen content. When compared with the non hash-based intra block copy, the hash-based method achieves 12.2% bitrate saving.

Traditional intra prediction methods exploit some fixed rules to generate prediction, which might not be adaptive enough to handle complicated contents. In this paper, we investigate applying deep neural network to improve the state-of-the-art intra prediction. Considering the characteristics of block-based video coding framework, we propose a fully connected network for intra prediction where all layers except non-linear ones are fully connected. In the proposed network, the inputs are multiple reference lines of the current block and the output is the prediction for the block. When compared with the traditional intra prediction method, the richer context of current block is exploited. For this reason, the proposed network is capable of providing more accurate prediction. Experimental results demonstrate the effectiveness of proposed network. When integrated into the HEVC reference software, the proposed method can achieve up to 3.3% bitrate saving and an average of 1.6% bitrate saving for 4K sequences.

Traditional intra prediction schemes usually only use the nearest adjacent reference lineto generate the prediction. Although the nearest reference line generally has the strongeststatistical correlation with current block, the farther non-adjacent reference lines can stillprovide potentialb etter prediction in some cases. Thus, in this paper, not only the nearestreference line but also the farther reference lines are utilized to help intra prediction. Whenusing the farther reference lines, an additional residue compensation procedure is introducedto further refine the prediction. In particular, this paper designs three solutions to meetdifferent complexity requirements. They are multiple line-based intra prediction (MLIP),fast search for multiple line-based intra prediction (FS-MLIP), and dual line-based intraprediction (DLIP). Experimentalresul ts verify the effectiveness of the proposed methods. When compared with HM-16.9, the proposed MLIP achieves 2.4% bit saving on averagewith the encoding time increasing about 362%. The FS-MLIP achieves 2.0% bit saving onaverage with the encoding time increasing about 114%. The DLIP achieves 0.9% bit savingon average with the encoding time increasing about only 15%.

Perspective motion is commonly represented in video content that is captured and compressed for various applications including cloud gaming, vehicle and aerial monitoring, etc. Existing approaches based on an eight-parameter homography motion model cannot deal with this efficiently, either due to low prediction accuracy or excessive bit rate overhead. In this paper, we consider the camera motion model and scene structure in such video content and propose a joint global and local homography motion coding approach for video with perspective motion. The camera motion is estimated by a computer vision approach, and camera intrinsic and extrinsic parameters are globally coded at the frame level. The scene is modeled as piece-wise planes, and three plane parameters are coded at the block level. Fast gradient-based approaches are employed to search for the plane parameters for each block region. In this way, improved prediction accuracy and low bit costs are achieved. Experimental results based on the HEVC test model show that up to 9.1% bit rate savings can be achieved (with equal PSNR quality) on test video content with perspective motion. Test sequences for the example applications showed a bit rate savings ranging from 3.7 to 9.1%.

Screen content refers to computer generated content like text, graphics, and animations. In such video, many regions may remain static for a long period after a sudden change. Traditional hierarchical Quantization Parameter (QP) setting may not be able to handle these regions efficiently because the encoder probably needs to refine the quality of these static regions multiple times. It will cost more bits while the quality of the static regions may reach the expected degree which the flat QP setting is able to achieve. This paper proposes using different QP settings for different regions in a picture. Region classification algorithms are developed to determine whether a flat or hierarchical QP setting is used. Experimental results demonstrate that the proposed scheme can achieve an average bitrate reduction of 3.1%, and up to 8.1% bitrate reduction for IBBB coding. The proposed method improves coding efficiency without increasing encoding complexity.

Emerging screen content coding brings great challenges to rate control due to significantly different characteristics of screen content from conventional video, e.g. large motion, frequent scene changes. This paper proposes an efficient rate control scheme for screen content coding by considering the characteristics of screen content. We first classify pictures of a video sequence into different groups by comparing the current picture with its neighbours. Then for each group, we apply different strategies to bit allocation and parameter updating process based on the characteristics of each group. The proposed algorithm can control the bitrate accurately without introducing any additional encoding delay. Experiment results show that the proposed scheme can significantly improve PSNR with a more accurate bitrate compared with the existing rate control scheme.

This paper presents a comparison-based test of the objective compression performance of the High Efficiency Video Coding (HEVC) standard, its format range extensions (RExt), and its draft screen content coding extensions (SCC). The current dominant standard, H.264/MPEG-4 AVC, is used as an anchor reference in the comparison. The conditions used for the comparison tests were designed to reflect relevant application scenarios and to enable a fair comparison to the maximum extent feasible – i.e., using comparable quantization settings, reference frame buffering, intra refresh periods, rate-distortion optimization decision processing, etc. It is noted that such PSNR-based objective comparisons generally provide more conservative estimates of HEVC benefit than are found in subjective studies. The experimental results show that, when compared with H.264/MPEG-4 AVC, HEVC version 1 provides a bit rate savings for equal PSNR of about 23% for all-intra coding, 34% for random access coding, and 38% for low-delay coding. This is consistent with prior studies and the general characterization that HEVC can provide about a bit rate savings of about 50% for equal subjective quality for most applications. The HEVC format range extensions provide a similar bit rate savings of about 13–25% for all-intra coding, 28–33% for random access coding, and 32–38% for low-delay coding at different bit rate ranges. For lossy coding of screen content, the HEVC screen content coding extensions achieve a bit rate savings of about 66%, 63%, and 61% for all-intra coding, random access coding, and low-delay coding, respectively. For lossless coding, the corresponding bit rate savings are about 40%, 33%, and 32%, respectively.

This paper presents an efficient video coding algorithm: Three-dimensional embedded subband coding with optimized truncation (3-D ESCOT), in which coefficients in different subbands are independently coded using fractional bit-plane coding and candidate truncation points are formed at the end of each fractional bit-plane. A rate-distortion optimized truncation scheme is used to multiplex all subband bitstreams together into a layered one. A novel motion threading technique is proposed to form threads along the motion trajectories in a scene. For efficient coding of motion threads, memory-constrained temporal wavelet transforms are applied along entire motion threads. Block-based motion threading is implemented in conjunction with 3-D ESCOT in a real video coder. Extension of 3-D ESCOT to object-based coding is also addressed. Experiments demonstrate that 3-D ESCOT outperforms MPEG-4 for most test sequences at the same bit rate.

This paper presents the results of an informal subjective quality comparison between the current state of the emerging High Efficiency Video Coding (HEVC) draft standard and the well-established H.264 / MPEG-4 AVC High Profile (HP) for low-delay applications. The tests consisted of two basic encoding comparisons. First, we compare the Main profile low-delay configuration of the HEVC reference software (HM) against a similarly configured H.264 / MPEG-4 AVC HP reference encoder (JM). Additionally, to complement these results, the widely-recognized production-quality H.264 / MPEG-4 AVC encoder known as x264 is compared with a production-quality HEVC implementation from eBrisk Video. The encoding configurations are designed to reflect relevant application scenarios and to enable a fair comparison to the maximum extent feasible. When viewing HM and JM encoded video side-by-side in which the JM was configured to use approximately twice the bit rate of the HM encoded video, viewers indicated that they preferred the HM encoded video in approximately 74% of trials. Similarly, when comparing the eBrisk HEVC and x264 H.264 / MPEG-4 AVC production encoders in which x264 was configured to use approximately twice the bit rate of the eBrisk encoded video, viewers indicated they preferred the eBrisk HEVC encoded video in approximately 62% of trials. The selection of which encoding was displayed on which side for the side-by-side viewing was established in a randomized manner, and the subjective viewing experiments were administered in a double-blind fashion. The results reported in this paper generally confirm that the HEVC design (as represented by HM version 7.1 and separately by a production-quality HEVC implementation) exhibits a substantial improvement in compression capability beyond that of H.264 / MPEG-4 AVC (as represented by a similarly-configured JM version 18.3 and x264 version core 122 r2184, respectively) for low-delay video applications, with HEVC exhibiting roughly twice or more of the overall compression capability of H.264 / MPEG-4 AVC.

In this paper, we present several strategies for transcoding from H.264/AVC bit streams to High Efficiency Video Coding (HEVC) bit streams. Because HEVC and AVC share the similar coding architecture, we try to exploit the information in AVC bit streams as much as possible. For inter picture, we utilize the power spectrum based rate-distortion optimization (PS-RDO) model as well as the input residual, modes and motion vectors to estimate the best coding unit (CU) split quad tree, the best prediction unit (PU) mode and the best motion vector of each PU partition For intra picture, we propose to reduce the CU and PU partition candidates. The proposed strategies can significantly reduce the transcoding complexity in terms of reduced processing for RDO evaluations, motion estimation, motion compensation as well as fractional pixel interpolation operations. Experiment results show that the proposed transcoding methods can achieve a good tradeoff between coding efficiency and transcoding complexity.

We propose two bit-allocation schemes for the progressive fine granularity scalable video coding with the temporal-SNR scalability (PFGST in short). One is a simple method which gives the reference images high priority. The other is a method based on the rate-distortion relationship of each frame. Experimental results show that both the methods achieve significant gain if compared with the uniform rate allocation scheme. At high bit rates, we can achieve 0.5dB gain and even more, at middle bit rates the gain can be up to 7dB depending on the sequences coded and coding parameters.

We developed a very low bit rate watercolor-like video form, which is constructed with a bi-level Y luminance component and a number of UV chrominance combinations. The bi-level Y-component outlines the features of an image and the UV-combinations describe the basic color information of the image. The UV combinations are dynamically chosen during encoding. The resultant video looks better than bi-level video and its bit rate is still much lower than that of full-color video. Watercolor video can be used in video broadcast and communication in very low bandwidth networks.