Search

Consumer electronics have started to support 8K/4K/UHD/SUHD videos in order to improve the perceived QoE. It is beneficial to assess the consumer’s perceived quality degradation at the output prior to transmitting the video in an error prone channel, to include concealment support and error resilient features to minimize these artifacts. While existing error resilient schemes have attempted to include source coding error and the channel’s error propagation, they have ignored the error resilience and concealment accuracy at the decoder when encoding the video which allows for end-to-end error capture and mitigation to improve consumer QoE. This paper proposes a two-pass codec encoding scheme that determines the motion vectors and the coding modes which not only mitigate the source coding error and the channel error propagation, but also improve the accuracy of the error concealment operation. The proposed method incorporates an end-to-end probabilistic distortion model and utilizes the concealment error in the subsequently encoding frames while estimating the motion vectors for the current frame. The experiment achieves up to 1.47dB PSNR gain for diverse video contents in comparison to the state-of-the-art techniques at a 10% packet error rate. This objective gain was also supported by the user perceived QoE.

Consumer Electronics (CE) devices have started to support 360° videos in order to improve the perceived video quality by consumers. Higher compression in 360° videos not only lead to improved consumer perception, but also efficient energy and storage usage in CE devices. Consequently, efficient compression of such video content is becoming a critical design consideration of CE devices. To this end, this paper first proposes a residual weighting algorithm that utilizes the features of Weighted Craster Parabolic Projection Peak Signal-to-Noise Ratio (WCPPPSNR), to instigate the first stage of magnitude reduction of the residuals. The model parameters associated with the residual weighting algorithm are determined in the initial experimental stage. Next, the paper proposes a Quantization Parameter (QP) optimization technique which introduces the second stage of magnitude reduction for residuals with minimal impact on the overall video quality. The proposed method is tested on the Versatile Video Coding (VVC) test model with All-Intra configuration. In this context, the empirical results demonstrate that the proposed method improves the coding efficiency by 3.34% on average and outperforms the state-of-the-art techniques.

The hierarchical quadtree partitioning of Coding Tree Units (CTU) is one of the striking features in HEVC that contributes towards its superior coding performance over its predecessors. However, the brute force evaluation of the quadtree hierarchy using the Rate-Distortion (RD) optimisation, to determine the best partitioning structure for a given content, makes it one of the most time-consuming operations in HEVC encoding. In this context, this paper proposes an intelligent fast Coding Unit (CU) size selection algorithm to expedite the encoding process of HEVC inter-prediction. The proposed algorithm introduces (i) two CU split likelihood modelling and classification approaches using Support Vector Machines (SVM) and Bayesian probabilistic models, and (ii) a fast CU selection algorithm that makes use of both offline trained SVMs and online trained Bayesian probabilistic models. Finally, (iii) a computational complexity to coding efficiency trade-off mechanism is introduced to flexibly control the algorithm to suit different encoding requirements. The experimental results of the proposed algorithm demonstrate an average encoding time reduction performance of 53.46%, 61.15%, and 58.15% for Low Delay B, Random Access, and Low Delay P configurations, respectively, with Bjøntegaard Delta-Bit Rate (BD-BR) losses of 2.35%, 2.9%, and 2.35%, respectively, when evaluated across a wide range of content types and quality levels.

Inter-frame prediction in HEVC uses two types of reference pictures: short-term and long-term. Out of these, long-term reference (LTR) pictures enable exploiting correlation among frames with extended temporal distances. In addition, LTR pictures improve the inter-frame prediction where video scenes are repeated such as in TV-series episodes, news broadcasts and movies. In this context, this paper proposes an algorithm to calculate LTR pictures using artificially generated virtual reference frames for static-camera scenes. The experimental results demonstrate an average coding improvement of 2.34% in terms of Bjontegaard Delta Bit Rate(BDBR), when compared with the HEVC reference encoder HM16.8.

Modern video compression solutions rely on intra-prediction to exploit spatial redundancy within a frame. In this paper, a new intra-prediction scheme is presented whereby one or more prediction directions are inferred at the decoder side by means of a texture analysis process. The inferred directions are then combined by means of prediction fusion, to compute the final predicted samples. No further information is required in the bitstream, considerably limiting the amount of bits necessary to be transmitted in order to derive the prediction block at the decoder. Experiments in the context of the state-of-the-art Versatile Video Coding standard show the proposed method can improve coding efficiency by -0.63% BD-rates on average, and up to -1.04% on larger resolution sequences, with limited impact on the decoder complexity.

It is vital that video encoders remove redundant information present in the spherically projected 360° video frames. To this end, this paper proposes a novel weighted residual technique and an integrated technique between the proposed weighted residual technique and a state-of-the-art adaptive quantization technique. The proposed methods adapt to the spherical characteristics in order to reduce the magnitude of the redundant information. The tests of the weighted residual technique and the integrated technique with All-Intra configuration using Versatile Video Coding (VVC) test model, produce average bit rate savings between 0.91% and 1.35% with regards to various spherical objective quality metrics.

High Efficiency Video Coding(HEVC) employs both past or future frames when encoding the current frame in a video sequence. This paper proposes a framework for using virtual reference frames, to achieve increased coding gains in the long-term for repetitive scenes in static camera scenarios.

It is vital that video encoders understand and remove the redundant information present in spherically projected 360° videos. To this end, this paper formulates a spherically adaptive objective function that incorporates novel adaptive quantization and weighted residual techniques to reduce the magnitude of redundant information while minimizing the distortion. Furthermore, this paper identifies the residual weighting function and the most optimal quantization parameters that are used to encode 360° videos. The proposed method is tested with test model of Versatile Video Coding (VVC) with All-Intra configurations. The obtained results exhibit an average bit rate saving of 3.18%.

360° videos are represented in spherical projection formats and the video quality of such videos is assessed using spherical objective quality metrics. Furthermore, the objective video quality between two different spherical projection formats can be evaluated using Cross projection metrics. Craster parabola, is a 2D cross projection format which is used by Craster Parabolic Peak Signal-to-Noise Ratio (CPPPSNR) metric. The existing CPPPSNR measurements do not consider the subsampling locations during the quality assessment to match the pixel density of a sphere. Nevertheless, it is vitally important to account for the oversampled projection formats and the sphere in order to be compatible with the existing video encoding architectures. To this end, the proposed improvements to the CPPPSNR locates the subsample points during craster parabolic projection and use nearest neighbor interpolation to assign pixels from the craster parabolic projection. Furthermore, in order to compensate the occurrence of oversampling, appropriate weights are applied to the corresponding pixels. The proposed method was tested with Shanghai Jiao Ton University (SJTU) Virtual Reality (VR) sequences for projection conversion. The comparison between Spherical PSNR (SPSNR) and existing CPPPSNR, validate the proposed CPPPSNR as an objective quality metric for cross projections.

The brute force rate-distortion optimisation based approach used in the High Efficiency Video Coding(HEVC) encoders to determine the best block partitioning structure for a given content demands an excessive amount of computational resources. In this context, this paper proposes a novel algorithm to reduce the computational complexity of HEVC inter-prediction using Support Vector Machines. The proposed algorithm predicts the Coding Unit (CU) split decision of a particular block enabling the encoder to directly encode the selected block, avoiding the unnecessary evaluation of the remaining CU size combinations. Experimental results demonstrate encoding time reductions of ∼58% ∼50%2.27%1.89% Bjontegaard Delta Bit Rate (BDBR) losses for Random Access and Low-Delay B configurations, respectively.

This paper proposes a content adaptive fast CU size selection algorithm for HEVC intra-prediction using weighted support vector machines. The proposed algorithm demonstrates an average encoding time reduction of 52.38% with 1.19% average BDBR increase compared to HM16.1 reference encoder.

The use of machine learning techniques for encoding complexity reduction in recent video coding standards such as High Efficiency Video Coding (HEVC) has received prominent attention in the recent past. Yet, the dynamically changing nature of the video contents makes it evermore challenging to use rigid traditional inference models for predicting the encoding decisions for a given content. In this context, this paper investigates the resulting implications on the coding efficiency and the encoding complexity, when using offline trained and online trained machine-learning models for coding unit size selection in the HEVC intra-prediction. The experimental results demonstrate that the ground truth encoding statistics of the content being encoded, is crucial to the efficient encoding decision prediction when using machine learning based prediction models.

Even though the latest video compression techniques such as High Efficiency Video coding (HEVC) have succeeded in significantly alleviating the bandwidth consumption during high resolution video transmission, they have become severely susceptible to transmission errors. Overcoming the resulting temporal impact of the transmission errors on the decoded video requires efficient error resilient schemes that can introduce robustness features to the coded video in order to mitigate the negative impact on the viewer. To this end, this paper proposes a rate-controlled error resilient bit allocation scheme, together with an encoding parameter selection process, to adaptively determine the most robust video coding parameters and the decoder error concealment operations during the encoding itself. Consequently, the proposed method has demonstrated 0.48dB-0.62dB PSNR gain over the state-of-the art methods at the same bit rate.

Along with the rapid increase in the availability of high-quality video formats such as high definition (HD), ultra HD, and high dynamic range, a huge demand for data rates during their transmission has become inevitable. Consequently, the role of video compression techniques has become crucially important in the process of mitigating the data rate requirements. Even though the latest video codec high efficiency video coding (HEVC) has succeeded in significantly reducing the data rate compared to its immediate predecessor H.264/advanced video coding, the HEVC coded videos in the meantime have become even more vulnerable to network impairments. Therefore, it is equally important to assess the consumers’ perceived quality degradation prior to transmitting HEVC coded videos over an error-prone network, and to include error resilient features so as to minimize the adverse effects of those impairments. To this end, this paper proposes a probabilistic model which accurately predicts the overall distortion of the decoded video at the encoder followed by an accurate QP– $\lambda $ relationship which can be used in the rate-distortion optimization (RDO) process. During the derivation process of the probabilistic model, the impacts from the motion vectors, the pixels in the reference frames, and the clipping operations are accounted, and consequently, the model is capable of minimizing the prediction error to as low as 3.11%, whereas the state-of-the-art methods cannot reach below 20.08%, under identical conditions. Furthermore, the enhanced RDO process has resulted in 21.41%–43.59% improvement in the BD-rate compared to the state-of-the-art error resilient algorithms.

Rate-Distortion optimization is updated to improve HEVC coding performance for HDR content. The proposed method employs Perceptually Uniform encoding as a distortion measure and shows on average performance improvement of 1.06% for HDR-VDP-2.2 Q factor and 4.02% in HDR-VQM with respect to the reference encoder. Further, with proposed change, better bit budget estimation is achieved showing performance improvement in Rate-Control. In comparison to NRMSE values from 0.5991 to 3.7670 achieved by the reference encoder, the proposed method is able to achieve improved NRMSE values from 0.2896 to 2.7754.

Although High Dynamic Range (HDR) content can provide an enhanced immersive experience for end-users, the impact of channel errors on its perception is unclear due to the lack of a standardized HDR video distribution framework. This paper presents an assessment of the robustness of the two main HDR video distribution architectures, the single-layer 10-bit scheme and the two-layer 8-bit backward-compatible scheme, in terms of their perceived quality in an error prone transmission network. In order to approximate a realistic distribution scenario, the state-of-the-art error concealment approaches are implemented at the HDR decoder with drop packets during the transmission. Results suggest that the single-layer architecture is more robust to channel errors compared to the two-layer architecture. However, the results reveal that with an efficient concealment approach in an enhancement layer, the backward-compatible two-layer coding architecture has potential to outperform the single-layer coding architecture which can lead to new directions of HDR research.

This paper proposes an error resilience framework that can be devised to compress HDR (High Dynamic Range) video content using HEVC (High Efficiency Video Coding) codec. The proposed method predicts the most robust motion vectors, coding modes and decoder concealment methods under erroneous environments considering the Perceptually Uniform (PU) domain quality impact. Results show that an average of 1dB PUPSNR gain can be achieved over the existing methods.

Error concealment techniques such as motion copying require significant changes to HEVC (High Efficiency Video Coding) motion estimation process when incorporated in error resilience frameworks. This paper demonstrates a novel motion estimation mechanism incorporating the concealment impact from future coding frames to achieve an average 0.73 dB gain over the state-of-the-art.

New algorithms are being investigated in the context of next generation video coding to achieve higher compression efficiency. In particular, intra block copy is well known to accurately predict screen content and artificially generated content where patterns and edges may repeat within the frame. On the other hand, such type of video content is often subject to geometrical transformations such as transitions, rotations, zooms etc, which may not be accurately captured by simply copying neighbouring pixels. A new intra-prediction scheme is presented in this paper whereby blocks of samples from already reconstructed areas are processed by means of an affine-type transformation. This allows more accurate prediction of blocks which improve compression efficiency in a variety of conditions. Experiments in the context of Versatile Video Coding standard show the proposed method can improve intra-coding compression efficiency by 2.01% BD-rates on average, and up to 4.81%, with negligible impact on the decoder complexity.