Search

This paper presents a new objective quality metric that was adapted to the complex characteristics of immersive video (IV) which is prone to errors caused by processing and compression of multiple input views and virtual view synthesis. The proposed metric, IV-PSNR, contains two techniques that allow for the evaluation of quality loss for typical immersive video distortions: corresponding pixel shift and global component difference. The performed experiments compared the proposal with 31 state-of-the-art quality metrics, showing their performance in the assessment of quality in immersive video coding and processing, and in other applications, using commonly used image quality assessment databases– TID2013 and CVIQ. As presented, IV-PSNR outperforms other metrics in immersive video applications and still can be efficiently used in the evaluation of different images and videos. Moreover, basing the metric on the calculation of PSNR allowed the computational complexity to remain low. Publicly available, efficient implementation of IV-PSNR software was provided by the authors of this paper and is used by ISO/IEC MPEG for evaluation and research on the upcoming MPEG Immersive video (MIV) coding standard. [ABSTRACT FROM AUTHOR]

In the three-dimensional video system, the depth image-based rendering is a key technique for generating synthesized views, which provides audiences with depth perception and interactivity. However, the inaccuracy of depth information leads to geometrical rendering position errors, and the compression distortion of texture and depth videos degrades the quality of the synthesized views. Although existing quality enhancement methods can eliminate the distortions in the synthesized views, their huge computational complexity hinders their applications in real-time multimedia systems. To this end, a residual distillation enhanced network (RDEN)-guided lightweight synthesized view quality enhancement (SVQE) method is proposed to minimize holes and compression distortions in the synthesized views while reducing the model complexity. First, a rethinking on the deep-learning-based SVQE methods is performed. Then, a feature distillation attention block is proposed to effectively reduce the distortions in the synthesized views and make the model fulfill more real-time tasks, which is a lightweight and flexible feature extraction block using an information distillation mechanism and a lightweight multi-scale spatial attention mechanism. Third, a residual feature fusion block is proposed to improve the enhancement performance by using the feature fusion mechanism, which efficiently improves the feature extraction capability without introducing any additional parameters. Experimental results prove that the proposed RDEN efficiently improves the SVQE performance while consuming few computational complexities compared with the state-of-the-art SVQE methods. [ABSTRACT FROM AUTHOR]

The depth video contains large smooth contents with sharp edges. Since the deep learning-based color video orientated intra prediction methods pay no attention to the characteristics of depth video, they are unsuitable for optimizing the coding efficiency of depth video. In this paper, a multiple resolution prediction method with deep up-sampling is proposed to promote the coding efficiency of depth video. To efficiently encode the depth blocks of different complexity, the depth block is selectively encoded at different resolutions, including $\times 1$ , $\times 1$ /2, and $\times 1$ /4 resolutions. If the block is encoded with a low-resolution (LR), the resolution of reconstructed LR depth block is recovered by an up-sampling network. To constrain the quality of both reconstructed high-resolution depth block and its synthesized view, a view synthesis distortion guidance mechanism is proposed for the up-sampling network. In addition, a distillation-based lightweight up-sampling network is proposed to reduce the computational complexity. Experimental results demonstrate that the proposed multiple resolution prediction method obtains an average of 10.84% BD-rate saving in comparison with 3D-HEVC. [ABSTRACT FROM AUTHOR]

The discrete cosine transform (DCT) is a relevant tool in signal processing applications, mainly known for its good decorrelation properties. Current image and video coding standards—such as JPEG and HEVC—adopt the DCT as a fundamental building block for compression. Recent works have introduced low-complexity approximations for the DCT, which become paramount in applications demanding real-time computation and low-power consumption. The design of DCT approximations involves a trade-off between computational complexity and performance. This paper introduces a new multiparametric transform class encompassing the round-off DCT (RDCT) and the modified RDCT (MRDCT), two relevant multiplierless 8-point approximate DCTs. The associated fast algorithm is provided. Four novel orthogonal low-complexity 8-point DCT approximations are obtained by solving a multicriteria optimization problem. The optimal 8-point transforms are scaled to lengths 16 and 32 while keeping the arithmetic complexity low. The proposed methods are assessed by proximity and coding measures with respect to the exact DCT. Image and video coding experiments and hardware realization are performed. The novel transforms perform close to or outperform the current state-of-the-art DCT approximations. [ABSTRACT FROM AUTHOR]

Light Field (LF) video technology is a step towards offering a better immersive experience through on-demand refocusing and perspective viewing. However, the significant increase in captured data makes the need for efficient compression of paramount importance. In this paper, we proposed two prediction structures and coding orders that efficiently compress LF video content using the existing HEVC standard. This is achieved by utilizing horizontal and vertical correlation among the views for better inter-view prediction. To assess the performance of the schemes, ten publicly available and widely used LF video sequences were used. Our first method is highly suitable for applications demanding high compression efficiency. It outperforms the best pseudo-sequence-based compression technique to date by up to 17% in bitrate reduction while being scalable in the number of views. The second method is proficient for low computational and random-access complexity to any arbitrary view in the light field video. It offers 10% faster decoding and 20% lower random-access complexity compared to the best existing technique. [ABSTRACT FROM AUTHOR]

In video-based dynamic point cloud compression (V-PCC), 3D point clouds are projected into patches, and then the patches are padded into 2D images suitable for the video compression framework. However, the patch projection-based method produces a large number of empty pixels; the far and near components are projected to generate different 2D images (video frames), respectively. As a result, the generated video is with high resolutions and double frame rates, so the V-PCC has huge computational complexity. This paper proposes an occupancy map guided fast V-PCC method. Firstly, the relationship between the prediction coding and block complexity is studied based on a local linear image gradient model. Secondly, according to the V-PCC strategies of patch projection and block generation, we investigate the differences of rate-distortion characteristics between different types of blocks, and the temporal correlations between the far and near layers. Finally, by taking advantage of the fact that occupancy maps can explicitly indicate the block types, we propose an occupancy map guided fast coding method, in which coding is performed on the different types of blocks. Experiments have tested typical dynamic point clouds, and shown that the proposed method achieves an average 43.66% time-saving at the cost of only 0.27% and 0.16% Bjontegaard Delta (BD) rate increment under the geometry Point-to-Point (D1) error and attribute Luma Peak-Signal-Noise-Ratio (PSNR), respectively. [ABSTRACT FROM AUTHOR]

Flexible optimization on video coding computational complexity is imperative to adapt to diverse terminals in video streaming, especially with high definition videos and increasingly comprehensive encoders. Until now, few efforts have been contributed to the encoder of multiview videos, which complicates its encoding structure with duplex frame display. To address this issue, we propose a flexible complexity optimization framework in this paper. In the first place, the proposed algorithm flexibly reduces the encoder complexity at different external-defined complexity constraints; furthermore, it is also committed to compression efficiency optimization at different complexity levels. The framework is achieved by a hybrid approach: complexity allocation, which allocates the external-defined complexity constraint to all views, hierarchical layers and frames based on linear programming; and complexity regulation, which dynamically adjusts local candidate partitions to fulfil the targeted local complexity constraint, with a probability-driven Alternate Partition Cost (APC) minimization. The overall algorithm is implemented on the popular multiview encoder, Multiview High Efficiency Video Coding (MV-HEVC), with promising Rate-Distortion (RD) performances at different complexity constraints, which are also superior to two recent complexity optimization algorithms. [ABSTRACT FROM AUTHOR]

Recent learned image coding has emerged with superior efficiency to conventional methods. It, however, is criticized for its complexity-exhaustive deep neural network (DNN) architectures, especially on resource-constrained mobile platforms. Thus we devise a two-stage approach: first, a range pre-processing is applied to constrain the dynamic range of feature map activation by leveraging its sparsity nature with densely clustered distribution, then a layer-wise range-adaptive quantization for convolutional parameter (e.g., weight, bias), and simple yet efficient linear scaling and range-dependent normalization for activation are executed, leading to a fully fixed-point inference architecture. All arithmetic operations and associated data tensors are processed using low-bit-width fixed-point numbers, yielding significant reductions of the computational complexity, memory space, and the elimination of platform-dependent inconsistency induced by floating-point operations. We first exemplify such fixed-point inference in a DNN-based image decoder, showing the comparable coding efficiency with its native floating-point model, against the same anchor using the High-Efficiency Video Coding (HEVC)-based intra image coder. We also extend proposed approach to the super-resolution network for learned resolution scaling-based video streaming, and VGG network-based classification tasks, both of which present negligible performance loss. These evidence the generalization of our approach for efficient DNN processing of various tasks. All materials are made publicly accessible at http://njuvision.github.io/fixed-point/. [ABSTRACT FROM AUTHOR]

This paper presents an overview of the decoder-side motion vector refinement (DMVR) algorithm in the Versatile Video Coding (VVC) standard. The proposed DMVR algorithm aims to increase the prediction accuracy of the blocks coded in merge mode using the bilateral matching-based refinement method. Compared with previous decoder-side motion vector derivation approaches, the proposed method significantly increases the coding efficiency without signaling additional side information. Furthermore, the hardware implementation considerations of the DMVR design are particularly focused in this study. This paper details and analyzes the novel features of DMVR contributing to the increase in coding efficiency and the reduction in computational complexity and implementation difficulty. Experimental results based on the VVC test model version 8.0 demonstrate that average Bjøntegaard Delta rate savings of 0.80 % and 2.81 % are achieved for the “tool-off” and “tool-on” test configurations, respectively. Moreover, 4 % additional decoding time and negligible additional external memory bandwidth requirements of DMVR based on the common test conditions for VVC are reported. [ABSTRACT FROM AUTHOR]

Motion-compensated prediction adopts fractional-pixel interpolation to obtain the best motion vector. Traditional fixed interpolation filters cannot handle various content and structures well, and existing convolutional neural network based methods cannot fully exploit the distortion characteristics for fractional interpolation. Therefore, this paper proposes a distortion-aware multi-task learning framework (DA-MLF) to perform fractional interpolation. First, a multi-task training framework is proposed to provide the distortion characteristics as complementary information for improving the performance of subsequent interpolation. Then, a uniform interpolation sub-network is proposed to accomplish fractional interpolation, which utilizes the feature fusion module to fuse abundant local features, and the distortion awareness module to capture the multi-scale information of compression artifacts. Furthermore, DA-MLF is integrated into High Efficiency Video Coding (HEVC) test model, and multiple experiments are performed to evaluate the effectiveness of our method. On HEVC testing sequences, DA-MLF achieves 5.0%, 4.0% and 1.7% BD-rate reduction on average compared to the HEVC baseline, under low-delay P, low-delay B and random-access configurations, respectively. The experimental results validate that our framework not only achieves the best interpolation performance but also has the lowest computational complexity compared with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Video encoding draws high research interest, due to the enormous demand for video traffic and real-time encoding for transmission. In video encoding standards such as HEVC (High-Efficiency Video Coding), the final step of the encoding stage is the CABAC (Context-Adaptive Binary Arithmetic Coding). The coding efficiency of the CABAC comes at the cost of increased computational complexity, especially for parallelization purposes, being the BAE (Binary Arithmetic Encoder) the critical part of CABAC. Thus, an important goal is to balance the real-time throughput requirements and the power/energy consumption in the design of BAE dedicated hardware. This work introduces a novel configurable high-throughput BAE design, named ET-BAE, utilizing a combination of a new modified Multiple-Bypass Bins Scheme (MBBS) and a power-saving approach into a single ASIC design with two-mode configuration. Synthesis and power-analysis results show that the configurable BAE design, the first of its kind with this feature, is more energy-efficient and less area consuming than utilizing non-configurable versions. The ET-BAE is able to accomplish the same real-time requirements of its competitors. [ABSTRACT FROM AUTHOR]

Deep learning provides a great potential for in-loop filtering to improve both coding efficiency and subjective quality in video coding. State-of-the-art work focuses on network structure design and employs a single powerful network to solve all problems. In contrast, this paper proposes a deep learning based systematic approach that includes an effective Convolutional Neural Network (CNN) structure, a hierarchical training strategy, and a video codec oriented switchable mechanism. First, we propose a novel CNN structure, i.e., Squeeze-and-Excitation Filtering CNN (SEFCNN), as an optional in-loop filter. To capture the non-linear interaction between channels, the SEFCNN is comprised of two subnets, i.e., Feature EXtracting (FEX) subnet and Feature ENhancing (FEN) subnet. Then, we develop a hierarchical model training strategy to adapt the two subnets to different coding scenarios. For high-rate videos with small artifacts, we train a single global model using the FEX for all types of frames, whereas for low-rate videos with large artifacts, different models are trained using both FEX and FEN for different types of frames. Finally, we propose an adaptive enhancing mechanism which is switchable between the CNN-based and the conventional methods. We selectively apply the CNN model to some frames or some regions in a frame. Experimental results show that the proposed scheme outperforms state-of-the-art work in coding efficiency, while the computational complexity is acceptable after GPU acceleration. [ABSTRACT FROM AUTHOR]

Screen content coding (SCC) is an extension of high efficiency video coding (HEVC), and it is developed to improve the coding efficiency of screen content videos by adopting two new coding modes: Intra Block Copy (IBC) and Palette (PLT). However, the flexible quadtree-based coding tree unit (CTU) partitioning structure and various mode candidates make the fast algorithms of the SCC extremely challenging. To efficiently reduce the computational complexity of SCC, we propose a deep learning-based fast prediction network DeepSCC that contains two parts: DeepSCC-I and DeepSCC-II. Before feeding to DeepSCC, incoming coding units (CUs) are divided into two categories: dynamic CTUs and stationary CTUs. For dynamic CTUs having different content as their collocated CTUs, DeepSCC-I takes raw sample values as the input to make fast predictions. For stationary CTUs having the same content as their collocated CTUs, DeepSCC-II additionally utilizes the optimal mode maps of the stationary CTU to further reduce the computational complexity. Compared with the HEVC-SCC reference software SCM-8.3, the proposed DeepSCC reduces the encoding time by 48.81% on average with a negligible Bjøntegaard delta bitrate increase of 1.18% under all-intra configuration. [ABSTRACT FROM AUTHOR]

Extensive efforts have been made to design hardware-based integer motion estimation (IME) that is much faster than software-based IME but suffers from the degradation in the coding efficiency. This is because the strategy for previous efforts was a simple algorithmic modification of the fast IME to facilitate the given hardware design at the expense of coding efficiency. This paper proposes a novel hardware design of the IME that not only offers real-time processing capability but also provides a flexible tradeoff between computational complexity and coding efficiency. First, a prediction unit (PU) loop unrolling scheme is proposed to solve the pipeline stall problem owing to the nature of fast IME algorithms such as the test zone search (TZS). It reduces idle cycles by 89.24%. Next, to further reduce the computational complexity of the TZS algorithm, a computational redundancy among PUs within a coding unit is reduced through a search step synchronization and search point sharing scheme. Thus, the computational complexity is reduced by 72.25%. The proposed schemes eliminate the inefficiency of hardware design; thus, they do not suffer from serious degradation in the coding efficiency. Consequently, the proposed hardware-based IME processes $7680\times4320$ videos at 30 frames per second while increasing the Bjøntegaard delta bitrate by only 0.90% on average. The hardware design is synthesized using a 65 nm general purpose CMOS technology, and its gate count is 268.5K at an operating clock frequency of 500 MHz. [ABSTRACT FROM AUTHOR]

Quadtree with nested multi-type tree (QTMT) partition structure is an efficient improvement in versatile video coding (VVC) over the quadtree (QT) structure in the advanced high-efficiency video coding (HEVC) standard. With the exception of the recursive QT partition structure, recursive multi-type tree partition is applied to each leaf node, which generates more flexible block sizes. Besides, intra prediction modes are extended from 35 to 67 so as to satisfy various texture patterns. These newly developed techniques achieve high coding efficiency but also result in very high computational complexity. To tackle this problem, we propose a fast intra-coding algorithm consisting of low-complexity coding tree units (CTU) structure decision and fast intra mode decision in this paper. The contributions of the proposed algorithm lie in the following aspects: 1) the new block size and coding mode distribution features are first explored for a reasonable fast coding scheme; 2) a novel fast QTMT partition decision framework is developed, which can determine the partition decision on both QT and multi-type tree with a novel cascade decision structure; and 3) fast intra mode decision with gradient descent search is introduced, while the best initial search point and search step are also investigated in this paper. The simulation results show that the complexity reduction of the proposed algorithm is up to 70% compared to VVC reference software (VTM), and averagely 63% encoding time saving is achieved with 1.93% BDBR increasing. Such results demonstrate that our method yields a superior performance in terms of computational complexity and compression quality compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The screen content coding (SCC) extension of high efficiency video coding (HEVC) improves coding gain for screen content videos by introducing two new coding modes, namely, intra block copy (IBC) and palette (PLT) modes. However, the coding gain is achieved at the increased cost of computational complexity. In this paper, we propose a decision tree-based framework for fast intra mode decision by investigating various features in the training sets. To avoid the exhaustive mode searching process, a sequential arrangement of decision trees is proposed to check each mode separately by inserting a classifier before checking a mode. As compared with the previous approaches where both IBC and PLT modes are checked for screen content blocks (SCBs), the proposed coding framework is more flexible which facilitates either the IBC or PLT mode to be checked for SCBs such that computational complexity is further reduced. To enhance the accuracy of decision trees, dynamic features are introduced, which reveal the unique intermediate coding information of a coding unit (CU). Then, if all the modes are decided to be skipped for a CU at the last depth level, at least one possible mode is assigned by a CU-type decision tree. Furthermore, a decision tree constraint technique is developed to reduce the rate–distortion performance loss. Compared with the HEVC-SCC reference software SCM-8.3, the proposed algorithm reduces computational complexity by 47.62% on average with a negligible Bjøntegaard delta bitrate (BDBR) increase of 1.42% under all-intra (AI) configurations, which outperforms all the state-of-the-art algorithms in the literature. [ABSTRACT FROM AUTHOR]

High-efficiency video coding (HEVC) poses a considerable challenge to hardware implementation due to its complexity. Mobile devices are powered by batteries that are limited in capacity. Therefore, reducing the power consumption arising from the implementation of sophisticated coding tools in HEVC is an especially important issue for mobile devices. In particular, motion estimation (ME) is the major contributor to the power consumption of the encoder and the calculation of the sum of absolute difference (SAD) for ME consumes more than 50% of the total ME power. In this paper, a low-power motion estimation VLSI architecture is proposed based on a novel method of calculating the SAD. By reusing the calculation, the computation complexity and, hence, the power consumption are reduced. A low-power systolic processing elements array and a novel memory hierarchy are developed, which enable real-time processing of 8K resolution video with only half of the power consumption when compared with the state-of-the-art design. [ABSTRACT FROM AUTHOR]

Distributed compressive video sensing (DCVS) has great potential for signal acquisition and processing in source-limited communication, e.g., wireless video sensors networks, because it shifts complicated motion estimation and motion compensation from the encoder to the decoder. Known as a state-of-the-art technique in DCVS, multihypothesis (MH) prediction is widely used because of its acceptable performance and low computational complexity. However, this technique is restricted by inaccurate regularizations, which can cause susceptibility to inaccurate hypotheses. In this paper, we present an iterative reweighted Tikhonov-regularized scheme for MH prediction reconstruction. Specifically, to enhance robustness, this scheme proposes a reweighted Tikhonov regularization that synthetically considers three factors that affect the MH prediction performance—accuracy of the hypothesis set, number of hypotheses, and accuracy of regularizations—by utilizing the influence of each hypothesis. Furthermore, to avoid over-iteration in iterative MH prediction reconstruction, we propose a Bhattacharyya coefficient-based stopping criterion for use in the recovery of non-key frames, in which we exploit the similarity to an adjacent key frame rather than a previous iteration result. The simulation results show that the proposed scheme outperforms the state-of-the-art MH methods in terms of robustness to inaccurate hypotheses when there are a limited number of hypotheses. [ABSTRACT FROM AUTHOR]

The growing demand for high-performance ultra-high-definition video coding leads to H.265/high-efficiency video coding (HEVC), where the increased computational complexity and data/timing dependence hinder its coding throughput. To address these challenges, this paper presents four algorithm adaptations and a fully parallel hardware architecture for an H.265/HEVC intra encoder. To the best of our knowledge, this is the first fully parallel H.265/HEVC intra encoder. This design supports 35 prediction modes and all coding tree unit partitions. All PUs are independently processed in four prediction engines for high parallelism. An appropriate set of intra prediction modes, RDO candidates, and CABAC rate estimate instances is assigned to each prediction engine, where internal computational tasks are pipelined and scheduled to maximize the processing throughput. Compared with the HM-15.0 software, the proposed algorithm adaptations lead to a reduction of 27% in computational workload, while the average BD-rate and BD-PSNR are 4.39% and −0.21 dB, respectively. This BD-rate is lower than the existing designs with the same video resolution. FPGA implementation of the proposed design shows that it operates at 120 MHz and supports 45 fps of 1080P video sequences using 201-K logic elements and 120-KB on-chip SRAM. ASIC implementation of the proposed design in TSMC 90-nm technology shows that its clock frequency reaches 320 MHz with a hardware gate count of 2288 K, and that it supports real-time encoding of 30 fps of 4-K video sequences. Compared with the state-of-the-art designs, our proposed design demonstrates advantages in computational complexity, bit rate, video quality, throughput, reliability, and flexibility. [ABSTRACT FROM AUTHOR]

Video summarization is an economical way of representing video contents and is useful for effective and quick browsing of the relevant activity present in the video. Most current video summarization approaches do not provide the correct insight of the events occurring in the video, such as moving the objects in a nonchronological order and tampering with the background and size of the objects. In this paper, we present an approach to create a video summarization that is a precise representation of the video content. First, our approach finds out the salient activities that are taking place in the video using an optimization framework for static and dynamic scenes. Second, the frames with the salient activities are stitched using alpha matting to form a single frame. Third, the summarized frame over multiple video shots obtained by our approach gives superior retrieval performance with image queries while reducing retrieval latency and memory requirement. This paper proposes a single frame-based indexing of the video database instead of multi-frame indexing as it is an interesting and potentially promising approach. Finally, several experiments are carried out to evaluate the proposed approach. The experiments demonstrate that the video summarization produced by our approach gives a realistic view of the events in the video, while the reduction in computational complexity and memory requirement is high. [ABSTRACT FROM AUTHOR]

We present a high-quality temporal frame interpolation (TFI) method that employs piecewise-smooth motion and handles disoccluded regions using the observation that motion discontinuities travel with the foreground object. We derive a “motion discontinuity” likelihood map from the divergence of a motion field between the input frames. Motion which is modeled at the reference frame is mapped to the target frame using a cellular-affine mapping strategy—a process during which regions of disocclusion are readily observed. This information is then used to guide the occlusion-aware, bidirectional FI process. Furthermore, we propose two computationally inexpensive texture optimizations that selectively improve the quality of the interpolated frames in regions around moving objects. The scheme produces very high-quality interpolated frames and outperforms current high-quality state-of-the-art TFI schemes by 2–2.5 dB; the method works with a very low-complexity motion estimation scheme and runs orders of magnitudes faster than its competitors. [ABSTRACT FROM AUTHOR]

High efficiency video coding (HEVC), which is the newest video coding standard currently, achieves the best coding efficiency compared with all the other existing video coding standards. However, the computational complexity of the typical HEVC encoder dramatically increases because of the recursive searching scheme for finding the best coding unit (CU) partitions. In this paper, an adaptive fast CU size decision algorithm for HEVC Intra prediction is proposed based on CU complexity classification (CC) by using machine learning (ML) technology. Firstly, certain image features are extracted to characterize the CU complexity, which has a strong relationship with CU partitions, and then, the support vector machine is employed to analyze and construct the classification model according to the CU complexity. Finally, the proposed adaptive fast CU size decision algorithm, named as CCML, is released based on the complexity classification. The experimental results show that the proposed algorithm could achieve around 60% encoding time reduction for various test video sequences on average with only 1.26% Bjontegaard delta bit rate increase compared with the reference test model HM15.0 of HEVC. [ABSTRACT FROM AUTHOR]

Although advanced motion vector prediction (AMVP) modes based on motion estimation (ME) are selected significantly less due to the merge mode newly adopted in the high-efficiency video coding (HEVC), integer ME (IME) still occupies a large amount of computation in HEVC because the HEVC supports a highly flexible block partitioning structure. Introduction of the merge mode in HEVC substantially affects the optimal search algorithm of IME. Therefore, this gives a chance to reduce the computational complexity of ME with marginal drop of the compression efficiency. This paper proposes a new IME algorithm that significantly reduces its search ranges. Computational complexity of the proposed algorithm is reduced without serious degradation in coding efficiency by obtaining additional accurate motion vector predictors (MVPs) in a bottom-up order and searching narrow regions around multiple MVPs. These bottom-up MVPs are obtained from the prediction units (PUs) in the coding units (CUs) in the hierarchically lower level, which share the pixel area either completely or partially with the current PU. Although the encoder should perform the proposed IME in a bottom-up order from the smallest CUs to the larger CUs, several other processes, such as fractional ME and merges mode are performed in a top-down order to exploit several fast algorithms already adopted in the HEVC test model encoder. To keep compatibility with HEVC standard, the bottom-up MVPs are utilized for only IME. The bitstream is generated using standard AMVP. According to the simulation results, it is confirmed that the proposed algorithm improves the accuracy of MVPs, which leads to the reduction of IME computational complexity up to 81.95% on average with the Bjøntegaard delta bitrate of 0.39%. [ABSTRACT FROM AUTHOR]

In this paper, we propose a fast H.264-to-HEVC transcoder composed of a motion propagation algorithm and a fast mode decision framework. The motion propagation algorithm creates a motion vector candidate list at the coding tree unit (CTU) level and, thereafter, selects the best candidate at the prediction unit level. This method eliminates computational redundancy by pre-computing the prediction error of each candidate at the CTU level and reusing the information for various partition sizes. The fast mode decision framework is based on a post-order traversal of the CTU and includes several mode reduction techniques. In particular, the framework permits the early termination of the rate distortion cost computation, a highly complex task, when a mode is unpromising. Moreover, a novel method exploits the data created by the motion propagation algorithm to determine whether a coding unit must be split. This allows the pruning of unpromising sub-partitions. Compared with a cascaded pixel-domain transcoding approach, the experimental results show that the proposed solution using one reference frame is on average 8.5 times faster, for an average Bjøntegaard delta-rate (BD-Rate) of 2.63%. For a configuration with four reference frames, the average speed-up is 11.77 times and the average BD-Rate is 3.82%. [ABSTRACT FROM AUTHOR]

In high efficiency video coding (HEVC) standard, residuals are partitioned into variable-size transform units, and integer cosine transforms (ICTs) of order-4, 8, 16, and 32 are adopted to optimize the coding performance. Order-64 ICT is not used due to its high complexity and limited gain in general coding applications. This paper proposes a set of low-complexity ICTs (LCICTs), from order-8 to order-64, which have fully factorizable structures. LCICTs achieve similar coding performance as the state-of-art methods on HEVC Test Model 13 while requiring lower complexity, e.g., 71% fewer multiplications than the partially factorizable ICTs in the order-64 case. Moreover, we find that order-64 ICTs achieve around 1% average bitrate reduction at high quantization parameters. [ABSTRACT FROM AUTHOR]

Rate control in scalable video coding (SVC) is a very challenging problem because of the inter-layer prediction structure, which makes developing an efficient rate-control algorithm complex and difficult. Little prior work is available for joint temporal-quality (T-Q) scalability considering the rate-distortion (R-D) dependence among the temporal and quality layers. However, most of the rate-control algorithms in SVC suffer from high computational complexity, growing significantly with the number of layers. In this paper, a single-pass joint temporal-quality rate-control algorithm is presented for H.264/SVC. In this algorithm, by analyzing the R-D dependence of joint T-Q scalability, Cauchy distribution-based rate-quantization, and distortion-quantization models, a set of empirical values are first derived to estimate the initial values of the R-D model parameters for the joint temporal and quality layers. Then, a novel prediction mechanism to update these model parameters is proposed to allocate the bit budgets efficiently among the temporal and quality layers, and hence to improve the performance of the proposed algorithm. Experimental results show that the proposed algorithm achieves better coding efficiency with low computational complexity compared with two other benchmark rate-control algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we study a simplified affine motion model-based coding framework to overcome the limitation of a translational motion model and maintain low-computational complexity. The proposed framework mainly has three key contributions. First, we propose to reduce the number of affine motion parameters from 6 to 4. The proposed four-parameter affine motion model can not only handle most of the complex motions in natural videos, but also save the bits for two parameters. Second, to efficiently encode the affine motion parameters, we propose two motion prediction modes, i.e., an advanced affine motion vector prediction scheme combined with a gradient-based fast affine motion estimation algorithm and an affine model merge scheme, where the latter attempts to reuse the affine motion parameters (instead of the motion vectors) of neighboring blocks. Third, we propose two fast affine motion compensation algorithms. One is the one-step sub-pixel interpolation that reduces the computations of each interpolation. The other is the interpolation-precision-based adaptive block size motion compensation that performs motion compensation at the block level rather than the pixel level to reduce the number of interpolation. Our proposed techniques have been implemented based on the state-of-the-art high-efficiency video coding standard, and the experimental results show that the proposed techniques altogether achieve, on average, 11.1% and 19.3% bits saving for random access and low-delay configurations, respectively, on typical video sequences that have rich rotation or zooming motions. Meanwhile, the computational complexity increases of both the encoder and the decoder are within an acceptable range. [ABSTRACT FROM AUTHOR]

Determining the best partitioning structure of a coding tree unit is one of the most time-consuming operations in High Efficiency Video Coding (HEVC) encoding. Specifically, it is the evaluation of the quadtree hierarchy using the rate-distortion (RD) optimization that has the most significant impact on the encoding time, especially in the cases of high definition (HD) and ultra HD videos. In order to expedite the encoding for low-delay applications, this paper proposes a coding unit (CU) size selection and encoding algorithm for inter prediction in the HEVC. To this end, it describes: 1) two CU classification models based on Inter $N\times N$ mode motion features and RD cost thresholds to predict the CU split decision; 2) an online training scheme for dynamic content adaptation; 3) a motion vector reuse mechanism to expedite the motion estimation process; and 4) finally introduces a computational complexity to coding efficiency tradeoff process to enable flexible control of the algorithm. The experimental results reveal that the proposed algorithm achieves a consistent average encoding time performance ranging from 55%–58% and 57%–61% with average Bjøntegaard delta bit rate increases of 1.93%–2.26% and 2.14%–2.33% compared with the HEVC 16.0 reference software for the low delay P and low delay B configurations, respectively, across a wide range of content types and bit rates. [ABSTRACT FROM PUBLISHER]

The 3D extension of High Efficiency Video Coding significantly improves the coding efficiency of 3D video at the expense of computational complexity. This paper presents a novel fast mode decision algorithm for depth map coding based on the grayscale similarity and inter-view correlation. First, depth map grayscale similarity is adopted to judge whether the reference frame could assist the coding of the current frame. When the difference in the average grayscale between the co-located coding unit (CU) and the current CU is smaller than the similarity threshold, the depth level of the current CU will be restricted by that of the coded reference CU. Second, the grayscale similarity and inter-view correlation are jointly used for dependent views to achieve early decision on the best prediction unit (PU) mode. The mode decision procedure will be determined early when the co-located CU, which has a grayscale similarity with the current CU, selects Merge or Inter $2N \times 2N$ as the best prediction mode. Moreover, when the corresponding CU in the independent view selects Merge or Inter $2N \times 2N$ as the best prediction mode, the current CU will skip other PU modes checking based on the strong inter-view correlation. Finally, different strategies are proposed for the P-frames and B-frames of dependent views in view of the characteristics of different prediction structures. For B frames, the PU mode information of the coded independent view is utilized as reference to skip the unnecessary mode decision processes. For P frames, the spatial–temporal correlation is considered in the process of early mode decision to determine whether to choose the Merge mode or Inter $2N \times 2N$ as the best mode. Experimental results show that our proposed scheme achieves considerable time saving with negligible degradation of coding performance. [ABSTRACT FROM PUBLISHER]

The H.265/MPEG-H High Efficiency Video Coding compliant encoding process faces the challenge of high computational complexity. Particularly, in the case of inter-picture prediction, most of the computational resources are allocated for the motion estimation (ME) process. In turn, ME and motion compensation enable improving coding efficiency by addressing the blocks of video frames as corresponding displacements from one or more reference blocks. These displacements do not necessarily have to be limited to integer sample positions, but may have an accuracy of half sample or quarter sample positions, which are identified during fractional sample refinement. In this paper, a context-based scheme for fractional sample refinement is proposed. The scheme takes the advantage of already obtained information in prior ME steps and provides significant flexibility in terms of parameterization. In this way, it adaptively achieves a desired tradeoff between computational complexity and coding efficiency. According to the experimental results obtained for an example algorithm utilizing the proposed framework, a significant decrease in the number of search points can be achieved. For instance, considering only 6 instead of 16 fractional sample positions results in a tradeoff of only 0.4% Bjøntegaard Delta-rate loss for high-definition video sequences compared with the conventional interpolation-and-search method. [ABSTRACT FROM PUBLISHER]

The second generation of the audio video coding standard (AVS2), which has been issued as IEEE 1857.4, doubles the coding efficiency of AVS1 and H.264/AVC. However, the advanced techniques in AVS2 also dramatically increase the computational complexity. The research on a commercial encoder for AVS2 is still at an early stage. In this paper, we propose the first fast intra-encoding platform for AVS2, which we term iAVS2. The platform uses numerous speedup methods, including code optimization, single-instruction multiple-data acceleration, and fast algorithms for almost all the time-consuming modules. To meet different application scenarios, five different speed levels are designed in iAVS2 to provide flexible tradeoffs between compression efficiency and encoding speed. The experimental results show that the fastest speed level of iAVS2 can accelerate the AVS2 reference software (RD12.0) by $144\times $ on average. Compared with the well-known HEVC encoder platform $\times 265$ , iAVS2 can provide over 10% bit rate saving at a similar coding speed. For the UHD and 1080 sequences, iAVS2 outperforms $\times 265$ at all speed levels. [ABSTRACT FROM PUBLISHER]

The depth-image-based rendering is a key technique for 3D video and free viewpoint video synthesis. One of the critical problems in current synthesis methods is that the background (BG) occluded by the foreground objects might be exposed in the new view, and some holes are produced in the synthesized video. However, most of the traditional hole-filling approaches may bring some blurry effect or artifacts in the virtual view. In this paper, a foreground removal approach for hole filling is proposed, in which the foreground objects are removed from both the 2D video and its corresponding depth map, and then a BG video and its depth map are generated before the 3D warping and used to eliminate the holes in the synthesized video. Moreover, a BG extension method is applied in the reference view to prevent the large holes occurring along the border areas in the virtual view. Our analysis and experimental results have indicated that the proposed approach has better performance compared with the other methods in terms of the quality of synthesized video, computational complexity, and running time in multiview synthesis or multiframe synthesis. [ABSTRACT FROM PUBLISHER]

SHVC, the scalable extension of High Efficiency Video Coding (HEVC), uses advanced inter-layer prediction features in addition to the advanced compression tools of HEVC to improve the compression performance. Using combined features has brought us improved compression performance at the cost of huge computational complexity for the SHVC encoder. This complexity is mainly because of the the inter/intra-prediction mode search of the coding units. The focus of this study is on developing an efficient complexity reduction for quality scalability of SHVC encoder, with the intention to facilitate the adoption of SHVC for real-time applications. In this regard, first, we build a probabilistic model that uses the mode information and motion homogeneity of already encoded blocks in the enhancement layer (EL) and the base layer to predict the probabilities of all the available inter/intra modes of the to-be-coded block in the EL. Then, we propose an online-learning-based fast mode, assigning (FMA) method that uses the proposed probabilistic model to predict the mode of the to-be-coded block in the EL. Performance evaluation shows that our proposed FMA method reduces the total execution time of the SHVC encoder by 45.40% on average compared with unmodified SHVC codec while maintaining the overall video quality. [ABSTRACT FROM PUBLISHER]

This paper presents a new block-based method for the High Efficiency Video Coding (HEVC) intra coding. First, we have found through analysis and test that the prediction errors on some pixels in each prediction block (PB) that are neighboring to the reference pixels would be no bigger than the corresponding coding errors. Based on this observation, the pixels in each PB are divided into two parts: half pixels are coded via a novel padding technique together with a constrained quantization algorithm (leading to around 3 dB gain under the same bit rate), whereas the other half are reconstructed by linear interpolations along a prediction direction by utilizing the neighboring reference pixels and the first half coded pixels. In the final implementation, a competition mechanism is employed between this new method and the original HEVC intra coding in order to choose the best mode for each PB. Experimental results show that about 2% BD-rate reduction has been achieved both for luma and chroma with respect to the original HEVC intra coding, whereas the encoder complexity increases by 130%, but the decoding time remains nearly unchanged. [ABSTRACT FROM AUTHOR]

An approximate kernel for the discrete cosine transform (DCT) of length 4 is derived from the 4-point DCT defined by the High Efficiency Video Coding (HEVC) standard and used for the computation of DCT and inverse DCT (IDCT) of power-of-two lengths. There are two reasons for considering the DCT of length 4 as the basic module. First, it allows computation of DCTs of lengths 4, 8, 16, and 32 prescribed by the HEVC. Second, the DCTs generated by the 4-point DCT not only involve lower complexity, but also offer better compression performance. Fully parallel and area-constrained architectures for the proposed approximate DCT are proposed to have flexible tradeoff between the area and time complexities. In addition, a reconfigurable architecture is proposed where an 8-point DCT can be used in place of a pair of 4-point DCTs. Using the same reconfiguration scheme, a 32-point DCT could be configured for parallel computation of two 16-point DCTs or four 8-point DCTs or eight 4-point DCTs. The proposed reconfigurable design can support real-time coding for high-definition video sequences in the 8k ultrahigh-definition television format ( $7680\times 4320$ at 30 frames/s). A unified forward and inverse transform architecture is also proposed where the hardware complexity is reduced by sharing hardware between the DCT and IDCT computations. The proposed approximation has nearly the same arithmetic complexity and hardware requirement as those of recently proposed related methods, but involves significantly less error energy and offers better peak signal-to-noise ratio than the others when DCTs of length more than 8 are used. A detailed comparison of the complexity, energy efficiency, and compression performance of different DCT approximation schemes for video coding is also presented. It is shown that the proposed approximation provides a better compressed-image quality than other approximate DCTs. The proposed method can perform HEVC-compliant video coding with marginal degradation of video quality and a slight increase the in bit rate, with a fraction of computational complexity of the latter. [ABSTRACT FROM AUTHOR]

The latest High Efficiency Video Coding (HEVC) standard could achieve the highest coding efficiency compared with the existing video coding standards. To improve the coding efficiency of the intra frame, a quad-tree-based variable block size coding structure that is flexible to adapt to various texture characteristics of images and up to 35 intra-prediction modes for each prediction unit (PU) is adopted in HEVC. However, the computational complexity is increased dramatically because all the possible combinations of the mode candidates are calculated in order to find the optimal rate distortion cost using the Lagrange multiplier. To alleviate the encoder computational load, this paper proposes an adaptive mode decision algorithm based on texture complexity and direction for HEVC intra prediction. First, an adaptive coding unit selection algorithm according to each depth levels’ texture complexity is presented to filter out unnecessary coding block. Then, the original redundant mode candidates for each PU are reduced according to its texture direction. The simulation results show that the proposed algorithm could reduce around 56% encoding time on average while maintaining the encoding performance efficiently with only a 1.0% increase in BD-rate compared with the test model HM16 of HEVC. [ABSTRACT FROM AUTHOR]

3-D High Efficiency Video Coding (HEVC) is a new emerging video compression standard for multiview video applications. This standard utilizes advanced interview prediction characteristics in addition to the prediction features of the HEVC standard for efficient encoding of multiview video content. While using combined features improves the compression performance by utilizing the correlation between the views captured from slightly different angles of the same scene, they also increase coding complexity. The focus of this paper is on developing an efficient complexity reduction scheme for 3D-HEVC, with the intention to facilitate the adoption of this upcoming standard, especially for real-time applications. In this regard, first, we introduce two ways to decrease the complexity of the inter-/ intra-mode search process of the to-be-encoded blocks in the dependent texture views ( \mathrm DVt\text{s} ) of 3D-HEVC. Then, we propose a hybrid complexity reduction scheme that utilizes the two-mode prediction approaches, motion information of the base texture view (BVt), and the rate distortion cost of the already encoded blocks in the BVt and DVt. The performance of our proposed scheme is tested for the case with two views (i.e., base view + dependent view). The evaluations confirm that our proposed hybrid complexity reduction scheme reduces the 3D-HEVC codec complexity by 67.70% on average for the DVt compared with the unmodified 3D-HEVC encoder, while maintaining the overall video quality. Compared with the state-of-the-art method, it reduces complexity by 25.74% on average. [ABSTRACT FROM AUTHOR]

High Efficiency Video Coding (HEVC) has brought significant improvement to video compression in terms of coding performance and subjective quality. However, it is computationally more complex than H.264/Advanced Video Coding (AVC) and, hence, adaptation of HEVC to real-time video services has not been promising. In order to reduce the computational cost, we propose a fast mode decision method that is based on the adaptive ordering of modes. Our experimental results show that the proposed method reduces the total encoding time by 47.27%–60.73% and 47.65%–61.31%, with negligible degradation of coding performance, which is a Bjøntegaard delta rate increment of 0.35%–1.04% and 0.64%–1.80% under random-access configuration and low-delay configuration, respectively, compared with HEVC test model 12.1 with fast mode off. [ABSTRACT FROM AUTHOR]

Several different methods have been investigated in recent years, aiming at computational complexity reduction and scaling of High Efficiency Video Coding (HEVC) software implementations. However, maintaining the encoding time per frame or group of pictures (GOPs) below an adjustable upper bound is still an open research issue. A solution for this problem is devised in this paper based on a set of Pareto-efficient encoding configurations, identified through rate-distortion-complexity analysis. The proposed method combines a medium-granularity encoding time control with a fine-granularity encoding time control to accurately limit the HEVC encoding time below a predefined target for each GOP. It is shown that the encoding time can be kept below a desired target for a wide range of encoding time reductions, e.g., up to 90% in comparison with the original encoder. The results also show that compression efficiency loss (Bjøntegaard delta-rate) varies from negligible (0.16%) to moderate (9.83%) in the extreme case of 90% computational complexity reduction. [ABSTRACT FROM AUTHOR]

The emerging High Efficiency Video Coding (HEVC) standard adopts many advanced techniques with flexible combinations, which enables HEVC to achieve about 50% bit-rate reduction for similar perceptual video quality relative to the prior video coding standard H.264/Advanced Video Coding. However, the enormously increased encoding complexity of HEVC inevitably becomes one of the greatest challenges for real-time applications. Among all the factors resulting in the increase in encoding complexity of HEVC, the quad-tree structure for coding units (CUs) with different sizes and accordingly a large number of prediction modes is one critical reason. Thus, it is greatly desired to develop a fast mode decision method for HEVC to reduce the computational complexity. In this paper, considering that HEVC employs the quad-tree structure, and the distortion of each sub-CU can indicate whether the current mode is suitable for current CU, we explore the relationship between the impossible modes and the distribution of the distortions to help the encoder skip checking the unnecessary modes. Besides, since the residual values can reflect the prediction result directly, we propose a method to skip some motion estimation operations according to the distribution of the residuals. Experimental results show that the proposed method can save about 77% of encoding time with only about a 4.1% bit-rate increase compared with HM16.4 anchor, while compared with the fast mode decision method adopted in HM16.4, the proposed algorithm can save about 48% of encoding time with only about a 2.9% bit-rate increase. [ABSTRACT FROM PUBLISHER]

Mode decision has a significant effect on the quality and complexity of video coding. It is even more challenging when generating multiple bitstreams with different bitrates (BRs) in, for example, dynamic adaptive streaming for HTTP or transrating systems. Full search and simplified fast search mode decision (MD) methods either suffer from a high computational complexity or have a negative impact on quality. Furthermore, mode selection in conventional approaches strongly depends on the quantization parameter (QP). Hence, modes that have been selected for high BR compression may not be suitable for low BR when transrating a bitstream. In this paper, we propose a rate-distortion (R-D)-optimized criterion for fast MD algorithms. The proposed cost function, when adopted in different fast MD algorithms, not only improves the R-D performance by up to 6.6% in terms of Bjøntegaard delta rate, but also reduces the execution time of the encoder by up to 6.8%. We also show that modes selected by the proposed criterion are less sensitive to changes in BR or QP. As a result, the same modes in an encoded bitstream may be used even after transrating using requantization, resulting in a significant R-D performance improvement of up to 33.3%. [ABSTRACT FROM PUBLISHER]

While the rate–distortion optimized quantization (RDOQ) technique provides nontrivial coding gain in High Efficiency Video Coding (HEVC), it also involves considerable computations, whereby the complexity of quantization is significantly increased. In this paper, two schemes (the RDOQ bypass decision and the simplified level adjustment) are investigated to reduce the complexity of the quantization process in HEVC with RDOQ. The RDOQ bypass decision method initially selects the transform blocks for which the RDOQ is expected to give less/or no coding gain and enables the conventional uniform scalar quantization to be applied to these transform blocks instead of the RDOQ. The simplified level adjustment method only estimates the difference in rate–distortion costs among the candidate quantization levels to enable the encoder to select an optimal quantization level at a much reduced computational cost. Furthermore, the proposed simplified level adjustment scheme is designed so that it can be implemented in lookup tables. Experimental results show that the proposed fast method achieves 14.3% quantization complexity reduction in all intra main conditions, 15.2% in the random access main condition, and 14.9% in the low delay main condition on average with virtually no coding loss compared with the conventional quantization process with the RDOQ. [ABSTRACT FROM PUBLISHER]

High Efficiency Video Coding (HEVC) is a video coding standard that offers higher performance than previous video coding standards such as H.264/AVC. Merge mode is one of the new tools adopted in HEVC to improve the inter-frame coding efficiency. Merge mode saves the bits for the motion vector (MV) by sharing the MV with neighboring blocks. Merge mode estimation (MME) is the process of finding a merge mode candidate, which requires extensive computations and memory accesses due to the associated motion compensation. Although MME is very similar to motion estimation (ME) in many ways, previous research on ME cannot be directly applied to solve many difficulties in designing MME hardware. In this paper, the characteristics of and the computational complexity involved in MME are discussed. To improve the throughput of the MME hardware, partially increased parallelism is efficiently exploited. Furthermore, the $M$ -of- $N$ -pixel combination and flexible memory access schemes are proposed to maximize the scalability to support various block sizes of HEVC and to reduce the time for fetching reference data. The proposed schemes are applied to the MME hardware design in this paper. The proposed hardware can process 56074 of $64\times 64$ coding tree units per second with a clock frequency of 366 MHz, and its gate count is 585.4k with 2 kB of dual-port static RAM. [ABSTRACT FROM PUBLISHER]

High Efficiency Video Coding (HEVC) was developed by the Joint Collaborative Team on Video Coding to replace the current H.264/Advanced Video Coding (AVC) standard, which has dominated digital video services in all segments of the domestic and professional markets for over ten years. Therefore, there is a lot of legacy content encoded with H.264/AVC, and an efficient video transcoding from H.264/AVC to HEVC will be needed to enable gradual migration to HEVC. In terms of rate-distortion (RD) performance, HEVC roughly doubles the RD compression performance of H.264/AVC at the expense of a high computational cost. HEVC adopts a quadtree-based coding unit (CU) block partitioning structure that is flexible in adapting various texture characteristics of images. However, this causes a dramatic increase in computational complexity due to the necessity of finding the best CU partitions. This paper presents an adaptive fast quadtree level decision algorithm that is designed to exploit the information gathered at the H.264/AVC decoder in order to make faster decisions on CU splitting in HEVC using a Naïve-Bayes probabilistic classifier that is determined by a supervised data mining process. The experimental results show that the proposed algorithm can achieve a good tradeoff between coding efficiency and complexity compared with the anchor transcoder; moreover, it outperforms other related works available in the literature. [ABSTRACT FROM PUBLISHER]

The latest High Efficiency Video Coding (HEVC) standard significantly increases the encoding complexity for improving its coding efficiency, compared with the preceding H.264/Advanced Video Coding (AVC) standard. In this paper, we present a novel subjective-driven complexity control (SCC) approach to reduce and control the encoding complexity of HEVC. Through reasonably adjusting the maximum depth of each largest coding unit (LCU), the encoding complexity can be reduced to a target level with minimal visual distortion. Specifically, the maximum depths of different LCUs can be varied through solving the proposed optimization formulation of complexity control, based on two explored relationships: 1) the relationship between the maximum depth and encoding complexity and 2) the relationship between the maximum depth and visual distortion. Besides, the subjective visual quality is favored with a novel subjective-driven constraint imposed in the formulation, on the basis of a visual attention model. Finally, the experimental results show that our approach can achieve a wide range of encoding complexity control (as low as 20%) for HEVC, with the smallest complexity bias being 0.2%. Meanwhile, our SCC approach outperforms other two state-of-the-art complexity control approaches, in terms of both control accuracy and visual quality. [ABSTRACT FROM PUBLISHER]

High Efficiency Video Coding (HEVC) is the state-of-the-art video coding standard. It adopts a hierarchical quad-tree-based coding unit (CU) partitioning structure that is flexible in various texture and motion characteristics of a video signal. However, the exhaustive partitioning process for finding optimal CU partitions requires a dramatic increase in computational complexity of the HEVC encoder compared with previous video coding standards. In this paper, a fast CU partitioning algorithm is proposed for HEVC encoder, which early on terminates the CU partitioning process based on the Bayesian decision rule using joint online and offline learning. An online learning method is first presented based on the minimum error Bayesian decision rule using a training picture selection method with scene change detection. Next, a joint online and offline learning method is presented, which additionally trains the loss of decision making of the proposed method based on the minimum risk Bayesian decision rule. The proposed method is implemented on an HEVC test software 15.0. Experimental results show that the proposed method reduces the computational complexity of HEVC encoder to 53.6% on an average with a 0.71% acceptable Bjøntegaard delta bitrate loss in random access configuration. For other configurations, 48.4%, 48.5%, and 54.2% encoding time saving are obtained on an average for low delay, low delay-P, and all intra-configurations, respectively. [ABSTRACT FROM PUBLISHER]

With a lot of enhanced coding tools introduced, High Efficiency Video Coding (HEVC) achieves significant improvement in coding efficiency at the cost of increased computational complexity. To efficiently reduce the encoding time of HEVC, a wavefront-based high parallel (WHP) solution integrating novel data-level and task-level methods is proposed in this paper. On data level, optimal single-instruction-multiple-data algorithms are designed for the enhanced coding tools, i.e., replacing the multiplication in motion compensation by add and shift operations with reduced instruction cycles, removing the transpose in transform via realignment of coefficients, and minimizing the memory access in sum of absolute difference/sum of squared differences calculation by fully reusing the registers. On task level, a novel inter-frame wavefront (IFW) method is developed by effectively decreasing the dependence of wavefront parallel processing (WPP). In addition, a coding tree block level parallelism analysis method is presented to prove the superior of IFW method compared with other HEVC representative parallel methods. Besides, a three-level thread management scheme is proposed to best exploit the parallelism of IFW method and achieve corresponding encoding speedup. Extensive experimental results show that, the overall WHP solution can bring up to $57.65\times $ , $65.55\times $ , and $88.17\times $ speedup for HEVC encoding of Wide Video Graphics Array, 720p and 1080p standard test sequences, while maintaining the same coding performance as with WPP. The proposed solution is also applied in several leading video companies in China, providing HEVC video service for more than 1.3 million users everyday. [ABSTRACT FROM PUBLISHER]

The High Efficiency Video Coding standard provides improved compression ratio in comparison with its predecessors at the cost of large increases in the encoding computational complexity. An important share of this increase is due to the new flexible partitioning structures, namely the coding trees, the prediction units, and the residual quadtrees, with the best configurations decided through an exhaustive rate-distortion optimization (RDO) process. In this paper, we propose a set of procedures for deciding whether the partition structure optimization algorithm should be terminated early or run to the end of an exhaustive search for the best configuration. The proposed schemes are based on decision trees obtained through data mining techniques. By extracting intermediate data, such as encoding variables from a training set of video sequences, three sets of decision trees are built and implemented to avoid running the RDO algorithm to its full extent. When separately implemented, these schemes achieve average computational complexity reductions (CCRs) of up to 50% at a negligible cost of 0.56% in terms of Bjontegaard Delta (BD) rate increase. When the schemes are jointly implemented, an average CCR of up to 65% is achieved, with a small BD-rate increase of 1.36%. Extensive experiments and comparisons with similar works demonstrate that the proposed early termination schemes achieve the best rate-distortion-complexity tradeoffs among all the compared works. [ABSTRACT FROM AUTHOR]

High Efficiency Video Coding (HEVC) employs a coding unit (CU), prediction unit (PU), and transform unit (TU) based on the quadtree coding tree unit (CTU) structure to improve coding efficiency. However, the computational complexity increases greatly because the rate-distortion (RD) optimization process should be performed for all CUs, PUs, and TUs to obtain the optimal CTU partition. In this paper, a fast CU size decision algorithm is proposed to reduce the encoder complexity of HEVC. Based on the statistical analysis, three approaches with SKIP mode decision (SMD), CU skip estimation (CUSE), and early CU termination (ECUT) are considered. In SMD, it is determined that the remaining modes except for SKIP mode are preformed or not. CUSE and ECUT determine that larger CU sizes and smaller CU sizes are coded or not, respectively. Thresholds for SMD, CUSE, and ECUT are designed based on Bayes’ rule with a complexity factor. Update process is performed to estimate the statistical parameters for SMD, CUSE, and ECUT considering the characteristic of RD cost. The experimental results demonstrate that the proposed CU size decision algorithm significantly reduces computational complexity by 69% on average with 2.99% Bjøntegaard difference bitrate (BDBR) increase for random access. The complexity reduction and BDBR increase for low delay are 68% and 2.46%, respectively. The experimental results also show that our proposed scheme performs well for various characteristic of sequences and outperforms the two previous state-of-the-art works. [ABSTRACT FROM PUBLISHER]

Motion compensation (MC) is a critical component in terms of computational complexity and memory bandwidth. The MC complexity of High Efficiency Video Coding (HEVC) for UHD contents significantly increased more than that of AVC/H.264. This paper reveals and analyzes a feature of generalized P and B pictures in HEVC and introduces a simple and effective method for MC complexity reduction that can be exploited at both the encoder and decoder without affecting the compression performance. The proposed method bypasses the \(L1\) interpolation process when the \(L0\) and \(L1\) motion information of a bipredicted block are identical. The simulation results show that the time reductions of 14.5% and 6.4% for the encoder and decoder, respectively, were achieved for LD-B configuration without any changes in coding results. The proposed method was adopted in the HEVC test model as a non-normative complexity reduction tool. [ABSTRACT FROM AUTHOR]