Search

Consumer electronics have started to support 8K/4K/UHD/SUHD videos in order to improve the perceived QoE. It is beneficial to assess the consumer’s perceived quality degradation at the output prior to transmitting the video in an error prone channel, to include concealment support and error resilient features to minimize these artifacts. While existing error resilient schemes have attempted to include source coding error and the channel’s error propagation, they have ignored the error resilience and concealment accuracy at the decoder when encoding the video which allows for end-to-end error capture and mitigation to improve consumer QoE. This paper proposes a two-pass codec encoding scheme that determines the motion vectors and the coding modes which not only mitigate the source coding error and the channel error propagation, but also improve the accuracy of the error concealment operation. The proposed method incorporates an end-to-end probabilistic distortion model and utilizes the concealment error in the subsequently encoding frames while estimating the motion vectors for the current frame. The experiment achieves up to 1.47dB PSNR gain for diverse video contents in comparison to the state-of-the-art techniques at a 10% packet error rate. This objective gain was also supported by the user perceived QoE.

Consumer Electronics (CE) devices have started to support 360° videos in order to improve the perceived video quality by consumers. Higher compression in 360° videos not only lead to improved consumer perception, but also efficient energy and storage usage in CE devices. Consequently, efficient compression of such video content is becoming a critical design consideration of CE devices. To this end, this paper first proposes a residual weighting algorithm that utilizes the features of Weighted Craster Parabolic Projection Peak Signal-to-Noise Ratio (WCPPPSNR), to instigate the first stage of magnitude reduction of the residuals. The model parameters associated with the residual weighting algorithm are determined in the initial experimental stage. Next, the paper proposes a Quantization Parameter (QP) optimization technique which introduces the second stage of magnitude reduction for residuals with minimal impact on the overall video quality. The proposed method is tested on the Versatile Video Coding (VVC) test model with All-Intra configuration. In this context, the empirical results demonstrate that the proposed method improves the coding efficiency by 3.34% on average and outperforms the state-of-the-art techniques.

New video compression technology is being developed to meet the increasing demands for more content at higher resolutions. To this end, the Versatile Video Coding (VVC) standard was recently finalised by the Joint Video Experts Team (JVET). Even though VVC can considerably reduce the bitrate requirements of compressed videos, the complexity of encoders has also drastically increased. In order to allow practical implementations to meet complexity requirements, this paper proposes a new block partitioning scheme for VVC inter-coding which can significantly reduce the complexity of the encoding. The paper is based on a novel feature selection scheme to speed up the partitioning decisions in VTM. The features are fed into classifiers where both a new random forest tree selection process and an early termination criteria are applied to better specialise the predictions to block characteristics, and allowing uniform distribution of complexity savings throughout the sequence. The proposed method was shown to consistently reduce complexity of VVC inter-coding with marginal increases in bitrate.

Modern video compression solutions rely on intra-prediction to exploit spatial redundancy within a frame. In this paper, a new intra-prediction scheme is presented whereby one or more prediction directions are inferred at the decoder side by means of a texture analysis process. The inferred directions are then combined by means of prediction fusion, to compute the final predicted samples. No further information is required in the bitstream, considerably limiting the amount of bits necessary to be transmitted in order to derive the prediction block at the decoder. Experiments in the context of the state-of-the-art Versatile Video Coding standard show the proposed method can improve coding efficiency by -0.63% BD-rates on average, and up to -1.04% on larger resolution sequences, with limited impact on the decoder complexity.

It is vital that video encoders remove redundant information present in the spherically projected 360° video frames. To this end, this paper proposes a novel weighted residual technique and an integrated technique between the proposed weighted residual technique and a state-of-the-art adaptive quantization technique. The proposed methods adapt to the spherical characteristics in order to reduce the magnitude of the redundant information. The tests of the weighted residual technique and the integrated technique with All-Intra configuration using Versatile Video Coding (VVC) test model, produce average bit rate savings between 0.91% and 1.35% with regards to various spherical objective quality metrics.

It is vital that video encoders understand and remove the redundant information present in spherically projected 360° videos. To this end, this paper formulates a spherically adaptive objective function that incorporates novel adaptive quantization and weighted residual techniques to reduce the magnitude of redundant information while minimizing the distortion. Furthermore, this paper identifies the residual weighting function and the most optimal quantization parameters that are used to encode 360° videos. The proposed method is tested with test model of Versatile Video Coding (VVC) with All-Intra configurations. The obtained results exhibit an average bit rate saving of 3.18%.

360° videos are represented in spherical projection formats and the video quality of such videos is assessed using spherical objective quality metrics. Furthermore, the objective video quality between two different spherical projection formats can be evaluated using Cross projection metrics. Craster parabola, is a 2D cross projection format which is used by Craster Parabolic Peak Signal-to-Noise Ratio (CPPPSNR) metric. The existing CPPPSNR measurements do not consider the subsampling locations during the quality assessment to match the pixel density of a sphere. Nevertheless, it is vitally important to account for the oversampled projection formats and the sphere in order to be compatible with the existing video encoding architectures. To this end, the proposed improvements to the CPPPSNR locates the subsample points during craster parabolic projection and use nearest neighbor interpolation to assign pixels from the craster parabolic projection. Furthermore, in order to compensate the occurrence of oversampling, appropriate weights are applied to the corresponding pixels. The proposed method was tested with Shanghai Jiao Ton University (SJTU) Virtual Reality (VR) sequences for projection conversion. The comparison between Spherical PSNR (SPSNR) and existing CPPPSNR, validate the proposed CPPPSNR as an objective quality metric for cross projections.

This paper proposes a content adaptive fast CU size selection algorithm for HEVC intra-prediction using weighted support vector machines. The proposed algorithm demonstrates an average encoding time reduction of 52.38% with 1.19% average BDBR increase compared to HM16.1 reference encoder.

The use of machine learning techniques for encoding complexity reduction in recent video coding standards such as High Efficiency Video Coding (HEVC) has received prominent attention in the recent past. Yet, the dynamically changing nature of the video contents makes it evermore challenging to use rigid traditional inference models for predicting the encoding decisions for a given content. In this context, this paper investigates the resulting implications on the coding efficiency and the encoding complexity, when using offline trained and online trained machine-learning models for coding unit size selection in the HEVC intra-prediction. The experimental results demonstrate that the ground truth encoding statistics of the content being encoded, is crucial to the efficient encoding decision prediction when using machine learning based prediction models.

Even though the latest video compression techniques such as High Efficiency Video coding (HEVC) have succeeded in significantly alleviating the bandwidth consumption during high resolution video transmission, they have become severely susceptible to transmission errors. Overcoming the resulting temporal impact of the transmission errors on the decoded video requires efficient error resilient schemes that can introduce robustness features to the coded video in order to mitigate the negative impact on the viewer. To this end, this paper proposes a rate-controlled error resilient bit allocation scheme, together with an encoding parameter selection process, to adaptively determine the most robust video coding parameters and the decoder error concealment operations during the encoding itself. Consequently, the proposed method has demonstrated 0.48dB-0.62dB PSNR gain over the state-of-the art methods at the same bit rate.

Along with the rapid increase in the availability of high-quality video formats such as high definition (HD), ultra HD, and high dynamic range, a huge demand for data rates during their transmission has become inevitable. Consequently, the role of video compression techniques has become crucially important in the process of mitigating the data rate requirements. Even though the latest video codec high efficiency video coding (HEVC) has succeeded in significantly reducing the data rate compared to its immediate predecessor H.264/advanced video coding, the HEVC coded videos in the meantime have become even more vulnerable to network impairments. Therefore, it is equally important to assess the consumers’ perceived quality degradation prior to transmitting HEVC coded videos over an error-prone network, and to include error resilient features so as to minimize the adverse effects of those impairments. To this end, this paper proposes a probabilistic model which accurately predicts the overall distortion of the decoded video at the encoder followed by an accurate QP– $\lambda $ relationship which can be used in the rate-distortion optimization (RDO) process. During the derivation process of the probabilistic model, the impacts from the motion vectors, the pixels in the reference frames, and the clipping operations are accounted, and consequently, the model is capable of minimizing the prediction error to as low as 3.11%, whereas the state-of-the-art methods cannot reach below 20.08%, under identical conditions. Furthermore, the enhanced RDO process has resulted in 21.41%–43.59% improvement in the BD-rate compared to the state-of-the-art error resilient algorithms.

Rate-Distortion optimization is updated to improve HEVC coding performance for HDR content. The proposed method employs Perceptually Uniform encoding as a distortion measure and shows on average performance improvement of 1.06% for HDR-VDP-2.2 Q factor and 4.02% in HDR-VQM with respect to the reference encoder. Further, with proposed change, better bit budget estimation is achieved showing performance improvement in Rate-Control. In comparison to NRMSE values from 0.5991 to 3.7670 achieved by the reference encoder, the proposed method is able to achieve improved NRMSE values from 0.2896 to 2.7754.

Emerging internet based social and interactive video applications are expected to replace the traditional TV broadcasting concept. However, dynamic network conditions together with varying user expectations, pose many challenges for resource allocation in such applications. This paper proposes a heuristic solution to the problem, where the energy incurred by the applications is minimized while simultaneously satisfying the user- and network-driven constraints.

Although High Dynamic Range (HDR) content can provide an enhanced immersive experience for end-users, the impact of channel errors on its perception is unclear due to the lack of a standardized HDR video distribution framework. This paper presents an assessment of the robustness of the two main HDR video distribution architectures, the single-layer 10-bit scheme and the two-layer 8-bit backward-compatible scheme, in terms of their perceived quality in an error prone transmission network. In order to approximate a realistic distribution scenario, the state-of-the-art error concealment approaches are implemented at the HDR decoder with drop packets during the transmission. Results suggest that the single-layer architecture is more robust to channel errors compared to the two-layer architecture. However, the results reveal that with an efficient concealment approach in an enhancement layer, the backward-compatible two-layer coding architecture has potential to outperform the single-layer coding architecture which can lead to new directions of HDR research.

This paper proposes an error resilience framework that can be devised to compress HDR (High Dynamic Range) video content using HEVC (High Efficiency Video Coding) codec. The proposed method predicts the most robust motion vectors, coding modes and decoder concealment methods under erroneous environments considering the Perceptually Uniform (PU) domain quality impact. Results show that an average of 1dB PUPSNR gain can be achieved over the existing methods.

Three main approaches exist to solve the optimization problem described in the paper; linear programming methods, greedy resource allocation methods and heuristic methods.

Error concealment techniques such as motion copying require significant changes to HEVC (High Efficiency Video Coding) motion estimation process when incorporated in error resilience frameworks. This paper demonstrates a novel motion estimation mechanism incorporating the concealment impact from future coding frames to achieve an average 0.73 dB gain over the state-of-the-art.

Though ubiquitous, the full potential of consumer electronic devices in the home, as content creators, remains underutilized due to the limited interaction between the consumers and the existing on-demand application and media services. Although services such as interactive television could change this, the geographic distribution of groups of consumers and the need for on-the-fly media processing that this entails, makes the efficient utilization of resources a complex optimization task requiring mechanisms to simultaneously allocate processing and network resources to groups of users. However, these technologies have not yet been developed, and brute force methods remain prohibitively complex. In order to overcome this problem, this paper proposes heuristic algorithms to both generate end-to-end delay bound multicast trees for individual groups of users and to co-locate multiple multicast trees, such that a minimum group quality metric can be satisfied. The performance of the proposed heuristic solution is evaluated in terms of the serving probability, i.e., the resource utilization efficiency, and computation time of the resource allocation decision making process. Simulation results show that improvements in the serving probability of up to 50%, in comparison with existing generic resource allocation schemes, and several orders of magnitude reduction of the computation time, in comparison to an optimal linear programming solution approach, can be achieved.

New algorithms are being investigated in the context of next generation video coding to achieve higher compression efficiency. In particular, intra block copy is well known to accurately predict screen content and artificially generated content where patterns and edges may repeat within the frame. On the other hand, such type of video content is often subject to geometrical transformations such as transitions, rotations, zooms etc, which may not be accurately captured by simply copying neighbouring pixels. A new intra-prediction scheme is presented in this paper whereby blocks of samples from already reconstructed areas are processed by means of an affine-type transformation. This allows more accurate prediction of blocks which improve compression efficiency in a variety of conditions. Experiments in the context of Versatile Video Coding standard show the proposed method can improve intra-coding compression efficiency by 2.01% BD-rates on average, and up to 4.81%, with negligible impact on the decoder complexity.