Search

In an increasingly connected world, consumer video experiences have diversified away from traditional broadcast video into new applications with increased use of non-camera-captured content such as computer screen desktop recordings or animations created by computer rendering, collectively referred to as screen content. There has also been increased use of graphics and character content that is rendered and mixed or overlaid together with camera-generated content. The emerging Versatile Video Coding (VVC) standard, in its first version, addresses this market change by the specification of low-level coding tools suitable for screen content. This is in contrast to its predecessor, the High Efficiency Video Coding (HEVC) standard, where highly efficient screen content support is only available in extension profiles of its version 4. This paper describes the screen content support and the five main low-level screen content coding tools in VVC: transform skip residual coding (TSRC), block-based differential pulse-code modulation (BDPCM), intra block copy (IBC), adaptive color transform (ACT), and the palette mode. The specification of these coding tools in the first version of VVC enables the VVC reference software implementation (VTM) to achieve average bit-rate savings of about 41% to 61% relative to the HEVC test model (HM) reference software implementation using the Main 10 profile for 4:2:0 screen content test sequences. Compared to the HM using the Screen-Extended Main 10 profile and the same 4:2:0 test sequences, the VTM provides about 19% to 25% bit-rate savings. The same comparison with 4:4:4 test sequences revealed bit-rate savings of about 13% to 27% for $Y'C_{B}C_{R}$ and of about 6% to 14% for $R'G'B'$ screen content. Relative to the HM without the HEVC version 4 screen content coding extensions, the bit-rate savings for 4:4:4 test sequences are about 33% to 64% for $Y'C_{B}C_{R}$ and 43% to 66% for $R'G'B'$ screen content.

In this Special Section of the IEEE Transactions on Circuits and Systems for Video Technology, it is our honor to introduce the Versatile Video Coding (VVC) standard, the latest of the historic partnership collaborations between the International Telecommunication Union Telecommunication Standardization Sector (ITU-T), the International Organization for Standardization (ISO), and the International Electrotechnical Commission (IEC) in the field of video coding standardization.

Proceedings of the IEEE 109(9), 1463-1493 (2021). doi:10.1109/JPROC.2020.3043399, Published by Institute of Electrical and Electronics Engineers, New York, NY [u.a.]

IEEE transactions on circuits and systems for video technology : CSVT 30(5), 1226-1240 (2019). doi:10.1109/TCSVT.2019.2949619, Published by Institute of Electrical and Electronics Engineers, New York, NY

This paper provides an overview of the coding algorithms of the Joint Exploration Model (JEM) for video compression with capability beyond HEVC, which was developed by the Joint Video Exploration Team (JVET) of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG). The goal of the JEM development and experimentation was to provide evidence that sufficient coding efficiency improvement over the High Efficiency Video Coding (HEVC) standard can be achieved, which would justify the need for a new video coding standard with a compression capability significantly exceeding that of HEVC. The development of the JEM provided an ability to conduct studies toward that goal in a verifiable and collaborative manner and led to the launching of the project to develop the new Versatile Video Coding (VVC) standard. Objective metric gains exceeding 30% were measured for most of the tested high-resolution video content that represents current demanding new applications, and subjective testing using human observers showed even more benefit.

Standardization for digital video compression has shown significant evolution over the last three decades. Starting in 1988 with ITU-T H.261 as the first such standard that was practical for consumer use, ISO/IEC MPEG-1 and H.262/MPEG-2 video (the latter jointly standardized by ITU-T and ISO/IEC) were developed very soon thereafter, creating the first wave of broad usage of digital technology in consumer video, such as broadcast and disc player applications. Later, the H.264/MPEG-4 Advanced Video Coding (AVC) standard was again developed jointly by ITU-T and ISO/IEC experts, with its High Profile becoming dominant from 2004 in HD broadcast and storage, as well as network-based streaming services and private capture of video. With ever-increasing demands for higher quality and the advent of flat-panel displays, the H.265/MPEG-H High Efficiency Video Coding (HEVC) standard became the next generation of video compression standard; with its first version defined in 2013, HEVC has been especially instrumental for the recent deployment of Ultra High Definition (UHD, a.k.a. 4K video). As time has moved forward, video content has continued to become an increasing presence in our lives, with an ever-growing diversification of usage models and continuing demands for higher quality. For example, flat panels evolved towards support of high dynamic range (HDR) video with a wider color gamut, and new modalities for consuming video have appeared, such as head-mounted displays (HMDs).

There is currently the widest breadth of video codecs available for the massive $200B video services industry, comprising broadcast, streaming, and other services, in history: MPEG-2, MPEG-4, AVC, HEVC, VVC, VP8, VP9, AV1, EVC, and LC-EVC. While these codecs compete in the marketplace for share of streams, the consumer surely benefits from having advanced services at lower rates. Is 4K HDR HEVC going to become the new norm for broadcast/streaming? But this is a challenging environment for developers and service provides. In this panel, we explore the breadth of consumer services that are enabled by these technologies, including high resolution: 4K, 8K, and beyond, as well as HDR, and AR/VR – will these finally take off and fulfill their promise? And is 8K the end of the line for consumer devices such as TVs, and even computers, tablets, and smartphones?

IEEE transactions on circuits and systems for video technology 31(10), 3736-3764 (2021). doi:10.1109/TCSVT.2021.3101953, Published by Institute of Electrical and Electronics Engineers, New York, NY

Versatile Video Coding (VVC), a.k.a. ITU-T H.266 | ISO/IEC 23090-3, is the new generation video coding standard that has just been finalized by the Joint Video Experts Team (JVET) of ITU-T VCEG and ISO/IEC MPEG at its $19^{\mathrm {th}}$ meeting ending on July 1, 2020. This paper gives an overview of the VVC high-level syntax (HLS), which forms its system and transport interface. Comparisons to the HLS designs in High Efficiency Video Coding (HEVC) and Advanced Video Coding (AVC), the previous major video coding standards, are included. When discussing new HLS features introduced into VVC or differences relative to HEVC and AVC, the reasoning behind the design differences and the benefits they bring are described. The HLS of VVC enables newer and more versatile use cases such as video region extraction, composition and merging of content from multiple coded video bitstreams, and viewport-adaptive 360° immersive media.

Video comprises as much as three-fourths of IP traffic, 1 and has a huge impact on our lives by other measures as well, such as the multiple hours per day spent watching it in typical households and the amount of storage capacity devoted to it on the world’s servers and personal devices. Moreover, the percentage of video traffic has been increasing and is expected to further increase, 1 as consumer applications continually move toward higher resolution, higher fidelity, higher frame rates, higher dynamic range, higher bit depth, etc. Compressing video better continues to be a major market need, and a very active area of research and product development.

High Efficiency Video Coding (HEVC) is the most recent standard in the series of major video coding standards jointly produced by the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG). HEVC was first approved in 2013 in the ITU-T as Recommendation H.265 and in ISO/IEC as International Standard 23008-2, and it offers an unprecedented degree of compression capability for a very wide variety of applications. In the three years since its initial completion, it has been extended in several important ways to further broaden its scope. This special issue on HEVC features two sections: 1) HEVC extensions and 2) efficient HEVC implementations.

This paper reports on verification testing of the coding performance of the screen content coding (SCC) extensions of the High Efficiency Video Coding (HEVC) standard (Rec. ITU-T H.265 | ISO/IEC 23008-2 MPEG-H Part 2). The coding performance of HEVC screen content model (SCM) reference software is compared with that of the HEVC test model (HM) without the SCC extensions, as well as with the Advanced Video Coding (AVC) joint model (JM) reference software, for both lossy and mathematically lossless compression using All-Intra (AI), Random Access (RA), and Lowdelay B (LB) encoding structures and using similar encoding techniques. Video test sequences in 1920×1080 RGB 4:4:4, YCbCr 4:4:4, and YCbCr 4:2:0 colour sampling formats with 8 bits per sample are tested in two categories: “text and graphics with motion” (TGM) and “mixed” content. For lossless coding, the encodings are evaluated in terms of relative bit-rate savings. For lossy compression, subjective testing was conducted at 4 quality levels for each coding case, and the test results are presented through mean opinion score (MOS) curves. The relative coding performance is also evaluated in terms of Bjontegaard-delta (BD) bit-rate savings for equal PSNR quality. The perceptual tests and objective metric measurements show a very substantial benefit in coding efficiency for the SCC extensions, and provided consistent results with a high degree of confidence. For TGM video, the estimated bit-rate savings ranged from 60–90% relative to the JM and 40–80% relative to the HM, depending on the AI/RA/LB configuration category and colour sampling format.

This paper describes extensions to the High Efficiency Video Coding (HEVC) standard that are active areas of current development in the relevant international standardization committees. While the first version of HEVC is sufficient to cover a wide range of applications, needs for enhancing the standard in several ways have been identified, including work on range extensions for color format and bit depth enhancement, embedded-bitstream scalability, and 3D video. The standardization of extensions in each of these areas will be completed in 2014, and further work is also planned. The design for these extensions represents the latest state of the art for video coding and its applications.

Geometry partitioning for video coding involves establishing a partition line boundary within each block-shaped region and applying motion-compensated prediction to the two sub-regions created by the partition line. This paper presents techniques for enhancing the effectiveness and reducing the complexity of geometry partitioning schemes. A texture-difference-based approach is described to simplify the process of selecting the partition lines. Applying this approach together with a described skipping strategy for blocks with uniform texture can achieve a 94% reduction of encoding time while retaining a similar rate-distortion (R-D) performance to the full-search partitioning approach, when implemented for wedge-based geometry partitioning (WGP) in the context of H.264/MPEG-4 AVC JM 16.2. A bit rate improvement of approximately 6% is shown relative to not using geometry partitioning. For further R-D improvement, we describe a background-compensated prediction scheme to reduce the number of overhead bits used for motion vectors. Additionally, for systems in which high-quality depth maps are available, we incorporate depth map usage into the described approaches to generate a more accurate partitioning. Using these approaches with object-boundary-based geometry partitioning can achieve about 9% bit rate savings relative to using WGP, while keeping a similar computational complexity to the described complexity-reduced WGP.

High Efficiency Video Coding (HEVC) is a new video compression standard developed jointly by ITU-T Video Coding Experts Group (VCEG) and ISO/IEC Moving Pictures Expert Group (MPEG) through their Joint Collaborative Team on Video Coding (JCT-VC). The first version of HEVC will be finalized by the JCT-VC in January 2013. The HEVC project was launched to achieve major savings-e.g., reduction by about half for 1,280 × 720 high-definition (HD) and higher-resolution progressives can video-for equivalent visual quality relative to the bit rate needed by the widely used H.264/ MPEG-4 Advanced Video Coding (AVC) standard. For high resolution video where such additional compression is most urgently required, implementations of the current draft standard are already meeting or exceeding the targeted goal. We review the architecture and building blocks of HEVC, which were carefully selected with regard to compression capability versus complexity and to enable parallelism for the signal processing operations. Given the benefits that HEVC provides, it is likely to become the new primary reference for video compression.

The compression capability of several generations of video coding standards is compared by means of peak signal-to-noise ratio (PSNR) and subjective testing results. A unified approach is applied to the analysis of designs, including H.262/MPEG-2 Video, H.263, MPEG-4 Visual, H.264/MPEG-4 Advanced Video Coding (AVC), and High Efficiency Video Coding (HEVC). The results of subjective tests for WVGA and HD sequences indicate that HEVC encoders can achieve equivalent subjective reproduction quality as encoders that conform to H.264/MPEG-4 AVC when using approximately 50% less bit rate on average. The HEVC design is shown to be especially effective for low bit rates, high-resolution video content, and low-delay communication applications. The measured subjective improvement somewhat exceeds the improvement measured by the PSNR metric.

High Efficiency Video Coding (HEVC) is currently being prepared as the newest video coding standard of the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group. The main goal of the HEVC standardization effort is to enable significantly improved compression performance relative to existing standards-in the range of 50% bit-rate reduction for equal perceptual video quality. This paper provides an overview of the technical features and characteristics of the HEVC standard.

Perspective motion is commonly represented in video content that is captured and compressed for various applications including cloud gaming, vehicle and aerial monitoring, etc. Existing approaches based on an eight-parameter homography motion model cannot deal with this efficiently, either due to low prediction accuracy or excessive bit rate overhead. In this paper, we consider the camera motion model and scene structure in such video content and propose a joint global and local homography motion coding approach for video with perspective motion. The camera motion is estimated by a computer vision approach, and camera intrinsic and extrinsic parameters are globally coded at the frame level. The scene is modeled as piece-wise planes, and three plane parameters are coded at the block level. Fast gradient-based approaches are employed to search for the plane parameters for each block region. In this way, improved prediction accuracy and low bit costs are achieved. Experimental results based on the HEVC test model show that up to 9.1% bit rate savings can be achieved (with equal PSNR quality) on test video content with perspective motion. Test sequences for the example applications showed a bit rate savings ranging from 3.7 to 9.1%.

The High Efficiency Video Coding (HEVC) standard (ITU-T H.265 and ISO/IEC 23008-2) has been developed with the main goal of providing significantly improved video compression compared with its predecessors. In order to evaluate this goal, verification tests were conducted by the Joint Collaborative Team on Video Coding of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29. This paper presents the subjective and objective results of a verification test in which the performance of the new standard is compared with its highly successful predecessor, the Advanced Video Coding (AVC) video compression standard (ITU-T H.264 and ISO/IEC 14496-10). The test used video sequences with resolutions ranging from 480p up to ultra-high definition, encoded at various quality levels using the HEVC Main profile and the AVC High profile. In order to provide a clear evaluation, this paper also discusses various aspects for the analysis of the test results. The tests showed that bit rate savings of 59% on average can be achieved by HEVC for the same perceived video quality, which is higher than a bit rate saving of 44% demonstrated with the PSNR objective quality metric. However, it has been shown that the bit rates required to achieve good quality of compressed content, as well as the bit rate savings relative to AVC, are highly dependent on the characteristics of the tested content.

This paper discusses about video telephony. So equipment manufacturers turned to the corporate world, introducing pricey videoconferencing systems designed to replace on-site meetings and reduce travel costs. While many companies invested in the technology in the 1990s, it typically gathered dust, unused. It looked as though people didn't want it anywhere.

The five papers in this special section were among those submitted in response to the joint call for proposals on high efficiency video coding (HEVC) standardization. Although at this point of development it is still unclear which specific elements the final HEVC standard will contain, the selection of the papers was made such that together they would cover most of the promising tools and technologies that seem likely to be included in the standard.

JPEG XR is the newest image coding standard from the JPEG committee. It primarily targets the representation of continuous-tone still images such as photographic images and achieves high image quality, on par with JPEG 2000, while requiring low computational resources and storage capacity. Moreover, it effectively addresses the needs of emerging high dynamic range imagery applications by including support for a wide range of image representation formats.

The thirteen papers in this special issue are devoted to the standardization and development of scalable video coding techniques and applications.

A scalable extension to the H.264/AVC video coding standard has been developed within the joint video team (JVT), a joint organization of the ITU-T video coding group (VCEG) and the ISO/IEC moving picture experts group (MPEG). The extension allows multiple resolutions of an image sequence to be contained in a single bit stream. In this paper, we introduce the spatially scalable extension within the resulting scalable video coding standard. The high-level design is described and individual coding tools are explained. Additionally, encoder issues are identified. Finally, the performance of the design is reported.

This article provides a description of the structure, technology, performance, and resources of H.264/AVC, which is referred to formally as ITU-T Recommendation H.264 and ISO/IEC 14496-10 (MPEG-4 Part 10)