Search

Deep learning has achieved great success in the past decade, especially in the fields of computer vision and image processing. After witnessing such success, video coding experts are motivated to consider whether deep learning can also benefit video coding, and if so, they seek to discover why and how. Indeed, a number of research studies have been conducted to explore deep learning for image and video coding, which has been an active and fast-growing research area especially since the year 2015. These prior arts can be divided into two categories: new coding schemes that are built solely upon deep networks (deep schemes), and deep network-based coding tools that are embedded into traditional coding schemes (deep tools). Moreover, in the responses to the joint call for proposals on video compression with capability beyond High Efficiency Video Coding (HEVC), a number of deep tools have been proposed, and some of them are further studied for the upcoming Versatile Video Coding (VVC). In this paper, we summarize the ongoing efforts in the Joint Video Experts Team about the proposed deep tools, and we discuss several promising tools in much detail, including neural network-based intra prediction, convolutional neural network (CNN) based in-loop filtering, and CNN-based block-adaptive-resolution coding. A series of experimental results are provided to demonstrate the capability of these tools in achieving higher compression efficiency than the VVC or HEVC anchor. These results shed light on the promising direction of deep learning-based future video coding, towards which a lot of open problems call for further study. [ABSTRACT FROM AUTHOR]

To save manufacturing cost, most color digital video cameras employ a single-sensor technology with a red–green–blue (RGB) color filter array (CFA) to capture real-world scenes. Due to only one primary color measured at each pixel location, the captured videos are usually referred to as the mosaic videos. For the purposes of economical storage and transmission, it is very important to achieve a good tradeoff between the quality and bitrate when compressing mosaic videos with different RGB-CFA structures. In this paper, based on mathematical optimization technique, a novel chroma subsampling strategy is presented for compressing mosaic videos with arbitrary RGB-CFA structures in H.264/AVC and High Efficiency Video Coding (HEVC). For each $2 \times 2$ YUV block to be subsampled with 4:2:0 format, the proposed strategy determines the proper sampled $U$ and $V$ components by minimizing, prior to compression, the quality distortion between the original colocated mosaic block and the mosaic block conversed from the current subsampled YUV block. Through the mathematical optimization formulated in the proposed strategy, the significance of the sampled $U$ and $V$ components for reconstructing R, G, and B pixels can be simultaneously taken into consideration. The experimental results demonstrate that the proposed chroma subsampling strategy has the best quality and bitrate tradeoff at a similar execution time requirement for compressing mosaic videos with arbitrary RGB-CFA structures in H.264/AVC and HEVC compared with the state-of-the-art ones by Chen et al. and Yang et al. as well as the three commonly used ones. [ABSTRACT FROM AUTHOR]