Search

Gaming video streaming services have grown tremendously in the past few years, with higher resolutions, higher frame rates and HDR gaming videos getting increasingly adopted among the gaming community. Since gaming content as such is different from non-gaming content, it is imperative to evaluate the performance of the existing encoders to help understand the bandwidth requirements of such services, as well as further improve the compression efficiency of such encoders. Towards this end, we present in this paper GamingHDRVideoSET, a dataset consisting of eighteen 10-bit UHD-HDR gaming videos and encoded video sequences using four different codecs, together with their objective evaluation results. The dataset is available online at [to be added after paper acceptance]. Additionally, the paper discusses the codec compression efficiency of most widely used practical encoders, i.e., x264 (H.264/AVC), x265 (H.265/HEVC) and libvpx (VP9), as well the recently proposed encoder libaom (AV1), on 10-bit, UHD-HDR content gaming content. Our results show that the latest compression standard AV1 results in the best compression efficiency, followed by HEVC, H.264, and VP9., 14 pages, 8 figures, submitted to IEEE journal

In this paper, the design and implementation of low power and high-efficiency filterbank for MPEG-2/4 AAC system is presented. Since filterbank represents the most computation-intensive kernel of AAC codec, we design it with algorithm and architecture aspects. We supply the dedicated algorithm for filterbank. The derived algorithm and the hardware shared engine (HSE) we proposed for AAC can reduce the computation power and implement as a codec used in encoder and decoder. We also optimize the performance, hardware resources, and power consumption thoroughly. It is designed as an intellectual property (IP) to construct the overall decoder in an embedded system. The hardware cost is with 16.1 k logic gates, 2 k-word local memory, and 1 K-word coefficient ROM. The proposed design has a real-time operation at only 1.25 MHz with a sampling rate of 48 kHz. It can achieve 0.70 mW power consumption in TSMC 0.18 μm CMOS technology. Furthermore, we use a programmable chip (SOPC) platform which includes the software and hardware engine. Throughout the computation analysis, the bitstream parser and lower complexity part are performed by a software solution, and the higher complexity part is computed by a hardware solution. Several design techniques are needed including the wrapper design, embedded CPU, and IP to construct the system. Based on this co-design approach, a whole AAC audio decoder is also established.

Content-weighted compression scheme for high-resolution remote-sensing (RS) images can be well modeled by Markov random field (MRF)-oriented attention. This article addresses high-resolution RS image compression by incorporating MRF into attention mechanism. To this end, we reformulate the attention mechanism with MRF-based probabilistic graph modeling implicitly and combine the target of image compression and parameter learning of MRF in a unified framework, namely high-order MRF-oriented attention (HMA) network. Specifically, HMA extends key-value query (KVQ) pairwise terms of the vanilla attention to high-order terms, by which the prior information could be expressed effectively to boost performance of high-resolution RS image compression. It is noted that several superiorities of HMA are listed. First, unlike the vanilla attention network that apt to yield coarse features, HMA is capable of output more pleasing decoding results. Second, HMA can accelerate the convergence in the training of the deep neural networks (DNNs), thus facilitating deploying it on resource-limited IOT devices. Third, HMA demonstrates its potential of processing semantic joint task. Moreover, We thoroughly evaluate our approach on standard data sets of varying resolutions, the proposed framework performs favorably against most image coding standards and DNN-based codecs on the ISPRS Vaihingen data set and the USC-SIPI data set especially at low bit rates.

Recently, more interest in the different plenoptic formats, including digital holograms, has emerged. Aside from other challenges that several steps of the holographic pipeline, from digital acquisition to display, have to face, visual quality assessment of compressed holograms is particularly demanding due to the distinct nature of this 3D image modality when compared to regular 2D imaging. There are few studies on holographic data quality assessment, particularly with respect to the perceptual effects of lossy compression. This work aims to study the quality evaluation of digital hologram reconstructions presented on regular 2D displays in the presence of compression distortions. As there is no established or generally agreed on compression methodology for digital hologram compression on the hologram plane with available implementations, a set of state-of-the-art compression codecs, namely, HEVC, AV1, and JPEG2000, were used for compression of the digital holograms on the object plane. Both computer-generated and optically generated holograms were considered. Two subjective tests were conducted to evaluate distortions caused by compression. The first subjective test was conducted on the reconstructed amplitude images of central views, while the second test was conducted on pseudovideos generated from the reconstructed amplitudes of different views. The subjective quality assessment was based on mean opinion scores. A selection of objective quality metrics was evaluated, and their correlations with mean opinion scores were computed. The VIFp metrics appeared to have the highest correlation.

The Dynamic Adaptive Streaming over HTTP (MPEG-DASH) ensures online videos display of good quality and without interruption. It provides an adequate streaming for each display device and network transmission. This can be verfield of Massive Open Online Courses (MOOCs). In fact, MPEG-DASH tracks the bandwidth fluctuations so hat the learner profits from continuous streaming, without worrying about frequent interruption of courses videos. Therefore, the learners profit from an exceptional visual experience that improves their commitment level and eases the course assimilation. These MPEG-DASH assets can become more and more advantageous if a good choice of its parameters is made. Being a recent branch, the MPEG-DASH adaptive diffusion presents a research field where the efforts are still limited, even more for MOOCs videos. Most of the work published in this sense focus on the Quality of Service (QoS) and the technical specifications of the network transmission. In this paper, we aim to consider the quality of the streamed content that directly impacts the learners Quality of Experience (QoE). For this, we develop a content-aware dataset that includes several MOOCs videos of different characteristics and types. These videos are firstly encoded using the latest codecs then dashified according to a coding scheme of several combinations of bitrates and display resolutions. Then, and in order to enhance the learners QoE, the so generated MPEG-DASH manifest files and segments are subsequently exploited to study the most appropriate codecs, bitrates and segment durations for each type of MOOCs videos.

With the rapidly increasing resolutions of 360° cameras, head-mounted displays, and live-streaming services, streaming high-resolution panoramic videos over limited-bandwidth networks is becoming a critical challenge. Foveated video streaming can address this rising challenge in the context of eye-tracking-equipped virtual reality head-mounted displays. However, conventional log-polar foveated rendering suffers from a number of visual artifacts such as aliasing and flickering. In this paper, we introduce a new log-rectilinear transformation that incorporates summed-area table filtering and off-the-shelf video codecs to enable foveated streaming of 360° videos suitable for VR headsets with built-in eye-tracking. To validate our approach, we build a client-server system prototype for streaming 360° videos which leverages parallel algorithms over real-time video transcoding. We conduct quantitative experiments on an existing 360° video dataset and observe that the log-rectilinear transformation paired with summed-area table filtering heavily reduces flickering compared to log-polar subsampling while also yielding an additional 10% reduction in bandwidth usage.

High-Throughput JPEG 2000 (HTJ2K) is a royalty-free image compression standard published in 2019 that enhances JPEG 2000 by replacing its slow block coder with a fast block coder. The resulting speedup (e.g., >30× for lossless coding) accelerates the encoding and decoding of images, which can have a great impact on users. A less obvious benefit of the speedup is the enabling of low-latency applications, which was not possible with the original JPEG 2000 standard. Examples of these new potential applications are live, high-quality, low-latency broadcast video contribution, remote production, and Internet Protocol (IP)-based production on-premises and on cloud. This paper examines various HTJ2K encoding parameters and their impact on quality, bitrate, latency, and multigeneration encode/decode cycle performance in the context of requirements of broadcast and IP-based applications. Configurations include a number of different wavelet filters, code-block sizes, and wavelet decomposition structures that are available with HTJ2K. The performance of low-latency HTJ2K is also compared to other low-latency wavelet codecs like VC-2, JPEG XS, and JPEG 2000 Part-1. The JPEG 2000 Part-1 comparisons use the full-frame broadcast profile as well as the ultra-low latency (ULL) configuration as per Video Services Forum (VSF) Technical Recommendation (TR)-01:2018. A tradeoff between latency and compression quality is discussed .

In this paper, a wavelet-based multimodal compression method is proposed. The method jointly compresses a medical image and an ECG signal within a single codec, i.e., JPEG-2000 in an effective and simple way. The multimodal scheme operates in two main stages: the first stage, consists of the encoder and involves a mixing function, aiming at inserting the samples of the signal in the image according to a predefined insertion pattern in the wavelet domain. The second stage represented by a separation function, consists of the extraction process of the ECG signal from the image after performing the decoding stage. Both the cubic spline and the median edge detection (MED) predictor have been adopted to conduct the interpolation process for estimating image pixels. Intensive experiments have been conducted to evaluate the performance of the multimodal scheme using objective distortion criteria. Results show clear superiority of the proposed scheme over the conventional separate compression approach involving two codecs: JPEG-2000 for images and ECG SPIHT-1D as well as other competing multimodal compression schemes in terms of both PRD and SNR at the signal decompression stage while maintaining good image quality and exhibiting a reduced computational complexity. Improvements in terms of average PRD and SNR values are as significant as 0.7 and 6 dB at low bit rates and 0.06 and 2 dB at higher bit rates on a number of test ECG signals and medical images.

We present Genozip, a universal and fully featured compression software for genomic data. Genozip is designed to be a general-purpose software and a development framework for genomic compression by providing five core capabilities—universality (support for all common genomic file formats), high compression ratios, speed, feature-richness and extensibility. Genozip delivers high-performance compression for widelyused genomic data formats in genomics research, namely FASTQ, SAM/BAM/CRAM, VCF, GVF, FASTA, PHYLIP and 23andMe formats. Our test results show that Genozip is fast and achieves greatly improved compression ratios, even when the files are already compressed. Further, Genozip is architected with a separation of the Genozip Framework from file-format-specific Segmenters and data-type-specific Codecs. With this, we intend for Genozip to be a general-purpose compression platform where researchers can implement compression for additional file formats, as well as new codecs for data types or fields within files, in the future. We anticipate that this will ultimately increase the visibility and adoption of these algorithms by the user community, thereby accelerating further innovation in this space. Availability and implementation Genozip is written in C. The code is open-source and available on http://www.genozip.com. The package is free for non-commercial use. It is distributed through the Conda package manager, github, and as a Docker container on DockerHub. Genozip is tested on Linux, Mac and Windows. Supplementary information Supplementary data are available at Bioinformatics online.

The industrial consortium AOMedia has recently developed the AV1 (AOMedia Video 1) format considered as a successor of VP9. One of the main goals with the video coding and compression format is to enable efficient Web based video content delivery, as in the case of OTT (Over-the-top) services. Nowadays, there are different AV1 software implementations available for experimental analysis besides the reference tool libaom. The Scalable Video Technology for AV1, or SVT-AV1 CODEC, is oriented towards further optimizations for specific Intel hardware. In this paper, we have performed a video quality assessment experiment with some of the SVT-AV1 implementations using 4k like video. The results show the advantages of SVT-AV1 format, as well as its performance for different quantization factors.

In this paper, we present a set of tools designed to simplify the distribution, playback, and adaptation of high-quality immersive content for large-screen immersive environments. The tools are based on a high-performance high-efficiency video coding (HEVC) codec that allows one to encode, decode, and play ultrahigh-resolution video in 8K and 16K resolutions using standard computing systems. A flexible video-rendering engine allows the content to be adapted to different output display environments. In addition, an immersive audio workflow, based on high-order ambisonics, provides support for 3D and interactive audio. The new tools have been validated in two advanced immersive installations using new immersive content created using various techniques that has been produced in the context of the Immersify European Project. The results show that the new tools enable ultrahigh-resolution immersive video playback combined with 3D audio, and first evaluations indicate that a better immersive experience for final users can be achieved, while at the same time using a simpler workflow.

File carving is a powerful technique for both digital forensics investigations and data recovery. It offers the flexibility to recover data stored on digital media independent of the underlying file system. Thus, it can be used in the cases where we need to recover deleted files or when we have a corrupted, overwritten, or unknown file system. In this paper, we present a novel video file carving (VidCarve) framework to recover and reassemble fragmented video files into playable video files. VidCarve consists of four main components: identification and recovery, weight assignment, reassembly, and file construction. This paper focuses on the weight assignment and reassembly processes where the codec specification parameters of carved fragments were overwritten. We propose several weight assignment techniques to estimate the probability of adjacency between video fragments. Based on these weights, the reassembly algorithm recovers the video files by constructing their correct sequences. We provide experimental results for the proposed techniques of weight assignment and reassembly. We claim that the overall accuracy rate can produce forensically sound evidence and play a critical role in the process of digital evidence recovery in many criminal cases.

The increase of video content and video resolution drive more exploration of video compression techniques recently. Meanwhile, learning-based video compression is receiving much attention over the past few years because of its content adaptivity and parallelable computation. Although several promising reports were introduced, there is no breakthrough work that can further go out of the research area. In this work, we provide an up-to-date overview of learning-based video compression research and its milestones. In particular, the research idea of recent works on learning-based modules for conventional codec adaption and the learning-based end-to-end video compression are reported along with their advantages and disadvantages. According to the review, compare to the current video compression standard like HEVC or VVC, from 3% to 12% BD-rate reduction have been achieved with integrated approaches while outperformed results on perceptual quality and structure similarity were reported for end-to-end approaches. Furthermore, the future research suggestion is provided based on the current obstacles. We conclude that, for a long-term benefit, the computation complexity is the major problem that needed to be solved, especially on the decoder-end. Whereas the rate-dependent and generative designs are optimistic to provide a more low-complex efficient learning-based codec.

This paper describes a new measurement method (VS model) for determining the quality of online video services relying on modern H.265/HEVC and VP9 codecs. The said method has been developed on the basis of VQuad-HD curves (according to ITU-T J.341). This model does not refer to signal analysis, but protocol analysis instead. The parameters used are: type of video codec, encoding rate, transport technique, packet loss and burst size. The method may be implemented quickly and easily, which is one of the great advantages when using this method to measure QoS.

A procedure for multi-criteria selection of optimal design solutions is proposed, taking into account the set of contradictory indicators of designed telecommunication system quality. In the procedure, the formal numerical method of weighting factors for solving the MCO problem is supplemented with an informal procedure by the method of narrowing the Pareto set for choosing the only best solution using additional expert information from a decision maker (DM). The inclusion of the decision maker in the interactive mode allows to refine the solution and bring it closer to the goal. In contrast to formal numerical methods, this allows changing the values of some partial criteria in dynamics to achieve the required quality of the optimized system. The problem of choosing a speech codec for IP networks is solved using the proposed procedure. The selection of a set of partial criteria has been substantiated. Solutions are obtained for different values of the weights of partial criteria with the involvement of a decision maker. The results obtained allow to conclude that the proposed procedure can be used by popular IP-PBXs when justifying the choice of an optimal codec version for the user.

In this paper, we propose a deep semantic segmentation-based multiple description coding (DSSMDC). In the proposed scheme, the input image is divided into two different subsets and getting two descriptions, which is named multiple description pre-processing (MDP). Then, two descriptions are encoded and decoded respectively by utilizing the deep semantic segmentation codec, in which the semantic segmentation label is as side information for improving image reconstruction quality. We can get one side reconstruction, when only one description is received at the decoder. If both descriptions are received at the decoder, we can get the central reconstruction. Experimental results show that the proposed scheme achieves better performance than other existing compression methods.

Feature transformation is the most primary step in biometric template protection whose effectiveness is directly dependent upon the test and trained samples. The minutiae points are segregated to retrieve the attributes with the singular point data to enlarge the secure user template. The user fingerprint information could be revealed to produce the secured template with the help of the fingerprint image. An attempt of feature transformation is experimented in this paper by DNA based encoding methodology. The general formation of the proposed technique is constructed from the Biometric data to generate the feature extraction; the Z pattern generation is used for implementing the transformation and produced the DNA codec. The evaluation of each step in the proposed scheme is theoretically evaluated and experimented with. The revocability, diversity and the security parameters are analyzed and compared with the relevant methods using the images from FVC2004 dataset that the accuracy for biometric template is achieved through the enhanced recognition rate. The complexity and the performance of the proposed method proved challenging to the state of art techniques.

High-Efficiency Video Coding (HEVC) is the most recent video codec standard. It is substantial to analyze the HEVC steganography process due to its practical and academic significance. Thus, a secure HEVC steganography approach is introduced in this paper to study the possibility of hiding an encrypted secret audio message within a cover compressed video frame in a secure and complicated manner. In the preliminary stage, the secret audio message is compressed utilizing the Discrete Cosine Transform (DCT) to achieve a high capacity performance for the HEVC steganography process. After that, the suggested approach implies two-cascaded encryption layers to encrypt the compressed secret message before embedding it within a cover HEVC frame. In the first encryption layer, a novel encryption technique based on random projection and Legendre sequence in the Discrete Wavelet Transform (DWT) domain is introduced to cipher the compressed secret audio message. In the second encryption layer, the yielded encrypted audio message is represented in a form of quaternion numbers using the Quaternion Fast Fourier Transform (QFFT) technique. Each cover HEVC frame is also represented in a quaternion form. In the suggested approach, some straightforward quaternion mathematical operations are employed on the encrypted secret message and the cover HEVC frames to represent them in a quaternion form in the frequency domain, then the encrypted secret audio message is hidden within the cover HEVC frame. At the receiver, the secret message can be retrieved and extracted from the cover HEVC frame utilizing the same methodology of the employed quaternion mathematical operations. The major contributions of the suggested HEVC steganography scheme are: (1) it allows hiding of massive amount of secret information within cover video frames, and (2) it has higher robustness against multimedia attacks and steganalysis contrasted to the conventional and literature schemes. Furthermore, the proposed approach is evaluated utilizing different assessment metrics like Feature Similarity Index Measure (FSIM), Peak Signal-to-Noise Ratio (PSNR), correlation coefficient, and Structural Similarity Index Measure (SSIM) to evaluate the efficiency of the stego HEVC frames compared to the original ones. The achieved outcomes demonstrate that the suggested steganography scheme is straightforward to implement, more secure, and robust in the presence of steganalysis multimedia attacks compared to the literature approaches.

Despite the current global health crisis, 2020 will be an important year in the Motion Picture Experts Group’s (MPEG) history. MPEG is working hard to finalize multiple new standards in the coming months, including three new codecs standards

To deliver three-dimension (3D) video services through the current two-dimension (2D) broadcasting systems, the frame compatible packing formats of views with depth maps will be an effective approach. With texture and depth information, the 3D videos with a depth image-based rendering engine can easily support all glasses and glasses-free 3D displays. In this paper, we propose a new YCbCr color depth packing method based on the centralized texture-depth packing (CTDP) formats to deliver effective 3D video services. Simulations show that the CTDP formats with YCbCr color depth packing method can achieve better objective and subjective texture and depth quality than the 2D-plus-depth packing (2DDP) formats. The benefits from using CTDP include light complexity and maturity and wide application of existing video codec. The packing scheme is compatible with any video codecs. Before the 3D-HEVC-based video broadcasting system, the proposed CTDP formats with YCbCr color depth packing method could help to deliver 3D videos in the current 2D broadcasting systems simply and efficiently.

Current image compression techniques such as JPEG and WebP are widely applied in the age of information. However, lossy compression like JPEG in its nature will introduce visually annoying artifacts while saving Internet bandwidth and storage space. The artifacts such as blocking and ringing are especially sharp at low bitrates. In this paper, we propose a novel dual-residual network to reduce compression artifacts caused by lossy compression codecs. This network directly learns an end-to-end mapping between the distorted image processed by JPEG or other compression methods and the original image, which takes decompressed images with blocking artifacts as input and produces clearer images with less artifacts. Experiments results on the test dataset demonstrate the efficiency of the proposed model, especially at very low bitrates.

With the advent of smartphones and tablets, the amount of online video traffic has increased enormously. This, together with the growing popularity of high-definition video, motivated the development of the High Efficiency Video Coding (HEVC) standard, released in 2013, with the aim of achieving a 50% bitrate reduction with respect to its predecessor, namely H.264/MPEG-4 Advanced Video Coding (AVC). However, new contents with greater resolutions and requirements arise every day, making it necessary to reduce the bitrate to a further extent. In this regard, the efforts to become the leading video codec in the market resulted in two main contenders: the Joint Video Experts Team (JVET), which leads the development of the Versatile Video Coding (VVC) standard, and the Alliance for Open Media (AOMedia), which spearheads the AOMedia Video 1 (AV1) project. In this context, this paper presents a rate-distortion/complexity analysis of HEVC, VVC and AV1 main video codecs using objective measures of assessment in order to analyze their real capabilities. The analysis, which was done using well-defined test conditions, reveals that VVC considerably outperforms both HEVC and AV1 in terms of coding efficiency.

El objetivo fue realizar el estudio y análisis de la calidad de servicio en la transmisión am digital mediante el uso de códecs en USRP. La Metodología de desarrollo muestra claramente cómo se realizó el prototipo para la transmisión y recepción, empezando con la selección de dispositivos, el diseño, la ejecución y el análisis de los resultados obtenidos. Se basa en el estudio de las condiciones particulares en la transmisión y recepción de señal en AM digital (DRM), la implementación del transmisor en GNU Radio, la configuración del receptor en HDSDR y Dream SDR. Además de las tarjetas Ettus USRP N210, Ettus USRP B200 y RTL-SDR REALTEK RTL2832U. El resultado de este estudio fue la calidad de los niveles de servicio evaluados en la velocidad de transmisión (Kbps) y la relación señal / ruido (SNR) para determinar la fidelidad en la recepción de señales digitales. En base a los resultados se define que las características de funcionamiento de cada modo de robustez para un determinado canal de transmisión varían los valores de la velocidad de transmisión y la relación señal a ruido, ofreciendo el modo B una mejor protección de la señal con respecto al ruido, mientras que el modo A ofrece mayor velocidad de transmisión. El sistema DRM emplea las mejores tecnologías de compresión de sonido ya que usa los algoritmos de audio y voz del MPEG-4 que brinda excelente calidad de sonido a bajas tasas de bit como la codificación AAC.

This paper is dedicated to the analysis of the existing approaches to video codecs comparisons. It includes the revealed drawbacks of popular comparison methods and proposes new techniques. The performed analysis of user-generated videos collection showed that two of the most popular open video collections from media.xiph.org which are widely used for video-codecs analysis and development do not cover real-life videos complexity distribution. A method for creating representative video sets covering all segments of user videos the spatial and temporal complexity is also proposed. One of the sections discusses video quality estimation algorithms used for video codec comparisons and shows the disadvantages of popular methods VMAF and NIQE. Also, the paper describes the drawbacks of the BD-rate – generally used method for video codecs final ranking during comparisons. A new ranking method called BSQ-rate which considers the identified issues is proposed. The results of this investigation were obtained during the series of research conducted as part of the annual video-codecs comparisons organized by video group of computer graphics and multimedia laboratory at Moscow State University.

In video compression class projects, students may observe some strange behaviors when using video codecs. Some performance metrics from a mediocre codec such as motion JPEG-2000 (or simply JPEG2000) may have exceptionally high values at certain compression ratios as compared to other high performing codecs. This strange behaviors may be overlooked by instructors and students may never understand why this is happening. In this paper, we will first highlight the strange behaviors. We will then use experiments to systematically determine the root cause. Our experiments show that, if one uses a previously compressed and decompressed video in some compression experiments, then it is highly likely that some strange behaviors will show up. Some advice will be provided to instructors, tutors, and students on how one can prevent such behaviors from occurring.

Recently, learning-based lossy image compression has achieved notable breakthroughs with their excellent modeling and representation learning capabilities. Comparing to traditional image codecs based on block partitioning and transform, these data-driven approaches with artificial-neural-network (ANN) structures bring significantly different distortion patterns. Efficient objective image quality assessment (IQA) measures play the key role in quantitative evaluation and optimization of image compression algorithms. In this paper, we construct a large-scale image database for quality assessment of compressed images. In the proposed database, 100 reference images are compressed to different quality levels by 10 codecs, involving both traditional and learning-based codecs. Based on this database, we present a benchmark for existing IQA methods and reveal the challenges of IQA on learning-based compression distortions. Furthermore, we develop an objective quality assessment framework in which a self-attention module is adopted to leverage multi-level features from reference and compressed images. Extensive experiments demonstrate the superiority of our method in terms of prediction accuracy. The subjective and objective study of various compressed images also shed lights on the optimization of image compression methods.

The study examines the impact of multi-algorithm fusion over audio compression with reference to the traditional exercises. For emotion recognition, here the most prominent features ‘Mel Frequency Cepstral Coefficients’ (MFCC) and ‘Discrete Wavelet Transform’ (DWT) features are extracted from prevalent speech samples of Berlin emotional database and Telugu (a south Indian language) database, we proposed automatic emotion recognition system (AERS) based on multi-algorithms fusion. AERS means to monitor and identify unit psychological/emotional state. The extracted features are analyzed using support vector machine, K-NN algorithms used for the classification of different states of emotion. Using two state-of-art mp3, Speex codec with different bit-rates investigated to ensure specific emotional intelligibility. MP3 codec configuration with 96 kbps bit-rate is recommended to pull off high compression for all emotions. Fusion algorithms also performed well compared with individual algorithms. Accuracy of 94.2% using fusion DWT and MFCC compared to 89.1% using DWT and 91.38% using MFCC separately. The accuracy of the proposed method increased further to 94% through a multiresolution approach by approximating frequency along with time information.

The past decade has witnessed great success of deep learning technology in many disciplines, especially in computer vision and image processing. However, deep learning-based video coding remains in its infancy. This paper reviews the representative works about using deep learning for image/video coding, which has been an actively developing research area since the year of 2015. We divide the related works into two categories: new coding schemes that are built primarily upon deep networks (deep schemes), and deep network-based coding tools (deep tools) that shall be used within traditional coding schemes or together with traditional coding tools. For deep schemes, pixel probability modeling and auto-encoder are the two approaches, that can be viewed as predictive coding scheme and transform coding scheme, respectively. For deep tools, there have been several proposed techniques using deep learning to perform intra-picture prediction, inter-picture prediction, cross-channel prediction, probability distribution prediction, transform, post- or in-loop filtering, down- and up-sampling, as well as encoding optimizations. In the hope of advocating the research of deep learning-based video coding, we present a case study of our developed prototype video codec, namely Deep Learning Video Coding (DLVC). DLVC features two deep tools that are both based on convolutional neural network (CNN), namely CNN-based in-loop filter (CNN-ILF) and CNN-based block adaptive resolution coding (CNN-BARC). Both tools help improve the compression efficiency by a significant margin. With the two deep tools as well as other non-deep coding tools, DLVC is able to achieve on average 39.6\% and 33.0\% bits saving than HEVC, under random-access and low-delay configurations, respectively. The source code of DLVC has been released for future researches.

Deep Neural Networks (DNN) have emerged in recent year as a best-of-breed alternative for performing various classification, prediction and identification tasks in images and other fields of study. In the last few years, various research groups are exploring the option to harness them to improve video coding with the primary purpose of reducing video compression rates while retaining same video quality. Evolving neural-networks based video coding research efforts are focused on two different directions: (1) improving existing video codecs by performing better predictions that are incorporated within the same codec framework, and (2) holistic methods of end-to-end image/video compression schemes. While some of the results are promising and the prospects are good, no breakthrough has been reported as of yet. This paper provides an overview of state-of-the-art research work, providing examples of few prominent publications that illustrate and further explain the different highlighted topics in the field of using DNNs for video compression. Our conclusion is that the benefits have not been fully explored yet and additional work is expected to accomplish the next generation, neural networks based codecs.

The JPEG2000 still image compression standard, while providing a remedy for the many shortcomings of its predecessor JPEG, is still slow to establish itself on the Internet. This problem is mainly due to the complexity of the COder-DECoder (CODEC) which implies its non-adoption by large firms and platforms in the field of image acquisition, processing and transmission. Indeed, the encoding and decoding process consumes a lot of CPU, memory and energy resources and takes a lot of computing time. The objective of this paper is to propose a model for decoding jpeg2000 on lightweight devices running on the Android mobile operating system. This implementation uses coroutines, which are a lightweight process model with reduced resource consumption costs compared to conventional AsyncTask threadsets. The model minimizes decoding time while minimizing CPU and memory usage, resulting in a fast and energetically economical decoded image. The results of integrating the coroutines from the main thread into the decoding process instead of the AsyncTask from the main thread produced better performance in terms of computation time, CPU and memory utilization. Indeed, the use of our model has led to a gain of around 23.41% in execution time, 9.8% in CPU utilization rate and 18.56% in memory utilization rate, compared to the model proposed in the literature which uses the threads.

Video compression is the core technology in mobile (mHealth) and electronic (eHealth) health video streaming applications. With global video traffic projected to reach 82% of all Internet traffic by 2022, there is a strong need to develop efficient compression algorithms to accommodate expected future growth. For the first time in decades, and especially since ISO/IEC MPEG and ITU-T VCEG expert groups strategically joined forces to develop the highly successful H.264/AVC standard, we have two distinct initiatives competing for the best performing video codec. On the one hand, we have the Alliance for Open Media (AOM) that support a new, royalty free video codec generation, termed AV1, based on VP8 and VP9 efforts. On the other hand, the Joint Video Exploration Team (JVET) has been developing the Versatile Video Codec (VVC) as the successor of the High Efficiency Video Coding (HEVC) standard. At the same time, the breadth of applications utilizing video codecs, involving significant content variability and moving across the video resolution ladder, to satisfy different constraints, have resulted in mixed literature results, with respect to the best performing codec. In this paper, we compare the performance of emerging VVC and AV1 codecs, along with popular HEVC implementations, namely the HEVC Test Model (HM) and x265, as well as earlier, VP9 codec, and investigate their suitability for medical applications. To the best of our knowledge, this is the first performance comparison of emerging VVC and AV1 video codecs for use in the healthcare domain. Experimental evaluation based on three datasets (ultrasound, emergency scenery, and general-purpose videos) demonstrate that VVC outperforms all rival codecs while AV1 achieves better compression efficiency than HEVC in all cases but low-resolution ( $560\times448$ @40Hz) ultrasound videos of the common carotid artery. Furthermore, the use of video despeckling prior to ultrasound video compression can provide significant bitrate savings.

In this modernistic age of innovative technologies like big data processing, cloud computing, and Internet of things, the utilization of multimedia information is growing daily. In contrast to other forms of multimedia, videos are extensively utilized and streamed over the Internet and communication networks in numerous Internet of Multimedia Things (IoMT) applications. Consequently, there is an immense necessity to achieve secure video transmission over modern communication networks due to the third-party exploitation and falsification of transmitted and stored digital multimedia data. The present methods for secure communication of multimedia content between clouds and mobile devices have constraints in terms of processing load, memory support, data size, and battery power. These methods are not the optimum solutions for large-sized multimedia content and are not appropriate for the restricted resources of mobile devices and clouds. The High-Efficiency Video Coding (HEVC) is the latest and modern video codec standard introduced for efficiently storing and streaming of high-resolution videos with suitable size and higher quality. In this paper, a novel hybrid cryptosystem combining DNA (Deoxyribonucleic Acid) sequences, Arnold chaotic map, and Mandelbrot sets is suggested for secure streaming of compressed HEVC streams. Firstly, the high-resolution videos are encoded using the H.265/HEVC codec to achieve efficient compression performance. Subsequently, the suggested Arnold chaotic map ciphering process is employed individually on three channels (Y, U, and V) of the compressed HEVC frame. Then, the DNA encoding sequences are established on the primary encrypted frames resulted from the previous chaotic ciphering process. After that, a modified Mandelbrot set-based conditional shift process is presented to effectively introduce confusion features on the Y, U, and V channels of the resulted ciphered frames. Massive simulation results and security analysis are performed to substantiate that the suggested HEVC cryptosystem reveals astonishing robustness and security accomplishment in contrast to the literature cryptosystems.

The codec landscape has rapidly accelerated, especially with the recent announcement by Apple to fully support High-Efficiency Video Coding (HEVC) and join the Alliance for Open Media (AOM) consortium. At the 2018 National Association of Broadcasters (NAB) Show, the AOM froze its specification of the AV1 codec. In addition, the Joint Video Exploration Team (JVET), a collaborative initiative between the Motion Picture Experts Group (MPEG) and International Telecommunication Union - Telecommunication (ITU-T), is making progress with the development of the next generation of MPEG compression, which will address future applications such as 8K and virtual reality (VR) six degrees of freedom (6DoF). This paper provides a thorough comparison of different codec technologies, including Advanced Video Coding (AVC), HEVC, AV1, and Versatile Video Coding (VVC). The total cost of ownership (TCO) and compression efficiency of each codec will be evaluated based on objective testing methods [i.e., peak signal-to-noise ratio (PSNR) and Video Multimethod Assessment Fusion (VMAF)] and subjective testing. Based on the factual data presented, readers will be able to make a better decision about which codec to use.

In this paper, we present a short review of some well-known image codecs in the literature and summarize a systematic study that determines the root cause of some puzzling observations in image compression experiments. Moreover, we propose a methodology to determine whether an image is genuine or not, meaning that whether or not a given image has been compressed and decompressed before and by which codec. In image compression class projects, students may observe some strange behaviors when they use some images with unknown quality in compression experiments. That is, some performance metrics from a mediocre codec such as JPEG may have exceptionally high values at certain compression ratios as compared to other high performing codecs. This confusing behavior may be overlooked by instructors, and students may never understand why this is happening. We will first highlight this anomalous behavior. We will then use experiments to systematically determine the root cause, which is due to image quality. In other words, if one uses a previously compressed and decompressed image in some compression experiments, it is highly likely that some strange behaviors in the performance metrics will show up. Our findings include the determination of the root cause of a puzzling phenomenon in image compression experiments and some sound advice to instructors, tutors, and students on how one can prevent such behaviors from occurring. We also developed a methodology to determine whether an image is genuine or not.

Multiple description coding (MDC) is able to stably transmit the signal in the un-reliable and non-prioritized networks, which has been broadly studied for several decades. However, the traditional MDC doesn't well leverage image's context features to generate multiple descriptions. In this paper, we propose a novel standard-compliant convolutional neural network-based MDC framework in term of image's context features. Firstly, multiple description generator network (MDGN) is designed to produce appearance-similar yet feature-different multiple descriptions automatically according to image's content, which are compressed by standard codec. Secondly, we present multiple description reconstruction network (MDRN) including side reconstruction network (SRN) and central reconstruction network (CRN). When any one of two lossy descriptions is received at the decoder, SRN network is used to improve the quality of this decoded lossy description by removing the compression artifact and up-sampling simultaneously. Meanwhile, we utilize CRN network with two decoded descriptions as inputs for better reconstruction, if both of lossy descriptions are available. Thirdly, multiple description virtual codec network (MDVCN) is proposed to bridge the gap between MDGN network and MDRN network in order to train an end-to-end MDC framework. Here, two learning algorithms are provided to train our whole framework. In addition to structural similarity loss function, the produced descriptions are used as opposing labels with multiple description distance loss function to regularize the training of MDGN network. These losses guarantee that the generated description images are structurally similar yet finely diverse. Experimental results show a great deal of objective and subjective quality measurements to validate the efficiency of the proposed method., Comment: 13 pages, 3 tables, and 6 figures. Accepted by IEEE Trans. CSVT. Our another MDC paper based on deep learning is "Deep Multiple Description Coding by Learning Scalar Quantization", which was accepted by DCC-2019. (arXiv:1811.01504)

Machine Learning (ML)-enabled software systems have been incorporated in many public demonstrations for automated driving (AD) systems. Such solutions have also been considered as a crucial approach to aim at SAE Level 5 systems, where the passengers in such vehicles do not have to interact with the system at all anymore. Already in 2016, Nvidia demonstrated a complete end-to-end approach for training the complete software stack covering perception, planning and decision making, and the actual vehicle control. While such approaches show the great potential of such ML-enabled systems, there have also been demonstrations where already changes to single pixels in a video frame can potentially lead to completely different decisions with dangerous consequences in the worst case. In this paper, a structured analysis has been conducted to explore video degradation effects on the performance of an ML-enabled pedestrian detector. Firstly, a baseline of applying “You only look once” (YOLO) to 1,026 frames with pedestrian annotations in the KITTI Vision Benchmark Suite has been established. Next, video degradation candidates for each of these frames were generated using the leading video compression codecs libx264, libx265, Nvidia HEVC, and AV1: 52 frames for the various compression presets for color frames, and 52 frames for gray-scale frames resulting in 104 degradation candidates per original KITTI frame and in 426,816 images in total. YOLO was applied to each image to compute the intersection-over-union (IoU) metric to compare the performance with the original baseline. While aggressively lossy compression settings result in significant performance drops as expected, it was also observed that some configurations actually result in slightly better IoU results compared to the baseline. Hence, while related work in literature demonstrated the potentially negative consequences of even simple modifications to video data when using ML-enabled systems, the findings from this work show that carefully chosen lossy video configurations preserve a decent performance of particular ML-enabled systems while allowing for substantial savings when storing or transmitting data. Such aspects are of crucial importance when, for example, video data needs to be collected from multiple vehicles wirelessly, where lossy video codecs are required to cope with bandwidth limitations for example.

There is currently the widest breadth of video codecs available for the massive $200B video services industry, comprising broadcast, streaming, and other services, in history: MPEG-2, MPEG-4, AVC, HEVC, VVC, VP8, VP9, AV1, EVC, and LC-EVC. While these codecs compete in the marketplace for share of streams, the consumer surely benefits from having advanced services at lower rates. Is 4K HDR HEVC going to become the new norm for broadcast/streaming? But this is a challenging environment for developers and service provides. In this panel, we explore the breadth of consumer services that are enabled by these technologies, including high resolution: 4K, 8K, and beyond, as well as HDR, and AR/VR – will these finally take off and fulfill their promise? And is 8K the end of the line for consumer devices such as TVs, and even computers, tablets, and smartphones?

Today, a huge variety of video codecs are available for compressing video, for a wide range of applications. Besides the traditional MPEG-2 and H.264 codecs, we now have H.265, VP9, AV1, H.266, EVC, and LC-EVC. H.266 succeeds H.265; EVC is essentially a light version of H.266/VVC; AV1 is a more advanced version of VP9; and LC-EVC is not a codec at all (nor related to EVC), but a codec enhancer. So of the remaining contenders for the next-generation video codec, we focus on perhaps the two most relevant: VVC and AV1. We take a practical approach, and focus on a specific application: high-quality, 4K aerial HDR video. Consumers love 4K UHD HDR. That moniker sells TVs. And among the most common way consumers can get a hold of capturing such content is with a high-end consumer drone. Modern drones can already capture 4K (and even 8K) in HDR, using the HLG transfer function, in 10-bit H.265, at 100 Mbs, creating clean content. We have collected some representative aerial HDR video clips to test with. In this context, we consider three issues: (a) convenience of conversion; (b) coding efficiency; and (c) playback support. We mention IP licensing issues in passing but focus on the technical merits as well as business prospects. In short, we find that both formats are usable, and moderately convenient to convert to. VVC has a slight edge in terms of coding efficiency (~15%). However, only one of these has even a possibility of wide playback support at this time: AV1. But that will change in time (~3-5 years). AV1 dates to March, 2018, VVC only to July, 2020.

State-of-the-art video compression techniques play a vital role in coding and decoding of video images that consume enormous memory space. In general, video compression is widely adapted for all future mobile applications based on recent developed technologies that are inbuilt into architectures to establish communication links with minimum consumption of power and memory for visual audio and video information. High-quality video consumes more memory for a server/user which is a major drawback in video compression. We present a conceptually generic, pliable and prevalent approach to overcome this issue in video compression with minimum memory space. However, storing and backup maintenance of these large data in an efficient way have been provided the researchers a roadmap in developing novel algorithms and techniques. A video compression technique plays a vital role in storage and transmission of data through the limited bandwidth capability. In this paper, authors mentioned advanced technologies in video compression by providing identical quality and recognizing more content of video. In this paper, authors focus on compressors and decompressors (CODECs) which are there like high-efficiency video coding (HEVC/H.265) that is capable to compress the video of any resolutions like 8192 * 4320 including 8K ultra-high definition (UHD) (John Singh et al. in Int J Pure Appl Math 119:3709–3724, [1]). The author highlights the algorithmic approach that describes the prerequisites for HEVC on playback on the web browser and compression of high-definition video for live video streaming, key factors in developments in dividing the video frames into a few subsections to high compression ratio and to retrieve the same video frames with high quality and more accurately (Garcia-Pineda et al. in Comput Commun, [2]).

In this paper, we propose a hybrid NR video quality metric with decodable payload. Although NR video quality metric has a strong need from the industry, developing these metrics has been a challenging task. Recently, hybrid models that use both processed video sequences and bit-stream data have shown promising results. In particular, from bitstream data, useful information can be extracted for video quality estimation, which includes codec information and impairment due to transmission errors. In the proposed method, we used packet information from the bit-stream data and estimated transmission impairment by decoding the payload to estimate the video quality. The proposed method was tested for the H.264 standard and it showed promising results. Also, the proposed method was independently evaluated using large datasets and it produced good performance.

International audience; This short paper describes a TensorFlow toolbox for point cloud geometry coding based on deep neural networks. This coding method employs a deep auto-encoder trained with a focal loss to learn good representations for voxel occupancy. The software provides several coding parameters to achieve different rate-distortion trade-offs, and comes with pre-trained models to reproduce the results of the published paper. It also offers a number of utility functions for evaluating and comparing the codec. To our knowledge, this is the first publicly available open-source toolbox for deep learning- based point cloud coding.

Compressive Video Sensing (CVS) is the video coding framework that uses compressive sensing (CS) theory into video codec to reduce the burden of encoding. Various existing methods are developed for CVS, but enhancing reconstruction speed and reconstruction quality remains a great challenge in video compression. Hence, an optimal method is developed using the proposed Differential Pulse Code Modulation+Conditional Autoregressive-Salp Swarm (DPCM+CA-SS) model for CVS system by considering the characteristics of CVS frame. The spatial redundancy of video frames is explored by partitioning the video frames into several blocks such that each block is measured using the measurement matrix. The measurement vectors are quantized using DPCM. The key frames are compressed into bits using Huffman coding, and the packet is transmitted to the decoder side. The temporal redundancy of CVS model is explored by encoding the frames using CA-SS-based prediction model to achieve Motion Aligned (MA) optimal reconstruction. However, the proposed CA-SS is designed by the integration of Conditional Autoregressive Value at Risk (CAViaR) and Salp Swarm Algorithm (SSA), respectively. However, the proposed DPCM+CA-SS obtained maximum PSNR and SSIM of 39.6649 db and 0.9120, and the minimum total time of 4.9523 s, respectively.

In preparation of the Call for Proposals on JPEG Pleno Holography, multiple exploration studies are ongoing to define the general procedure for performance assessment of proposed coding solutions. The performance of proposals will be compared against JPEG 2000 and H.265/HEVC intra coding, both serving as anchor codecs. In this paper, we report on the results of a dynamic subjective visual quality assessment procedure. A subset of 8 holograms, selected from the JPEG Pleno Database, is compressed both in hologram and object plane at three different bit rates. Chosen bit rates vary per hologram depending on their content characteristics. Then, for each hologram and its compressed versions, pseudo-video sequences are generated from the reconstructed views along a scan path that involves focus and viewing angle changes and that is hologram specific. A double stimulus simultaneous test, combined with a 5-level impairment-scale scoring protocol is deployed where videos created from the reference and the decoded holograms are visualized side by side on a professional 4K display. Results demonstrate that this test procedure requires deep scenes and sufficient scene complexity throughout the depth stack to allow for adequate stress testing of the codecs under test, particularly those solutions that adhere to compression in the object plane.

Video traffic comprises the majority of today's Internet traffic, and HTTP Adaptive Streaming (HAS) is the preferred method to deliver video content over the Internet. Increasing demand for video and the improvements in the video display conditions over the years caused an increase in the video coding complexity. This increased complexity brought the need for more efficient video streaming and coding solutions. The latest standard video codecs can reduce the size of the videos by using more efficient tools with higher time-complexities. The plans for integrating machine learning into upcoming video codecs raised the interest in applied machine learning for video coding. In this doctoral study, we aim to propose applied machine learning methods to video coding, focusing on HTTP adaptive streaming. We present four primary research questions to target different challenges in video coding for HTTP adaptive streaming.

DNA is a promising data storage medium due to its remarkable durability and space-efficient storage. Early bit-to-base transcoding schemes have primarily pursued information density, at the expense however of introducing biocompatibility challenges or at the risk of decoding failure. Here, we propose a robust transcoding algorithm named the “Yin-Yang Codec” (YYC), using two rules to encode two binary bits into one nucleotide, to generate DNA sequences highly compatible with synthesis and sequencing technologies. We encoded two representative file formats and stored them in vitro as 200-nt oligo pools and in vivo as an ~54-kb DNA fragment in yeast cells. Sequencing results show that YYC exhibits high robustness and reliability for a wide variety of data types, with an average recovery rate of 99.94% at 104 molecule copies and an achieved recovery rate of 87.53% at 100 copies. In addition, the in vivo storage demonstration achieved for the first time an experimentally measured physical information density of 198.8 EB per gram of DNA (44% of the theoretical maximum for DNA).

In this work we propose a novel deep learning approach for ultra-low bitrate video compression for video conferencing applications. To address the shortcomings of current video compression paradigms when the available bandwidth is extremely limited, we adopt a model-based approach that employs deep neural networks to encode motion information as keypoint displacement and reconstruct the video signal at the decoder side. The overall system is trained in an end-to-end fashion minimizing a reconstruction error on the encoder output. Objective and subjective quality evaluation experiments demonstrate that the proposed approach provides an average bitrate reduction for the same visual quality of more than 80% compared to HEVC., Comment: 5 pages

The document describes the procedure by which applications of JPEG 2000 CODECs will be evaluated with respect to conformance to the NIST guidance for compression of 1000 ppi friction ridge images as detailed in NIST Special Publication 500-289 [NIST3]. The document describes the attributes of a set of fingerprint images selected for conformance testing and the rationale for selection of these images based on both examiner assessment of image quality over increasing degrees of JPEG 2000 compression and relative fidelity based on computational metrics described SP 500-289 and supporting studies. The document provides background behind the conformance testing, describes the CODEC pathways to be tested and the metrics used to measure conformance, and provides instructions on how to run the protocol and submit results to NIST for evaluation.

Recently, due to the pandemic of coronavirus (known as COVID-19), travel options for people are very limited. However, technology Virtual Reality (VR), one of the emerging fields in the multimedia services, can help to overcome these barriers. Providing of a VR-based video content in a form of online streaming has very high requirements on the network capacity. Hence, advanced video compression algorithms with high efficiency for delivery of a video in the format of VR are vital. In this paper, we present an objective-based performance study of different implementations of the AOMedia Video 1 (AV1) video codec for 360-degree videos with a resolution of UHD. Different quality presets that set the balance between the visual quality and encoding speed for the same bitrate are also considered. The Video Multimethod Assessment Fusion (VMAF) objective metric, developed by Netflix, is used to measure the quality of the video sequences. Results show that all the tested implementations of the AV1 encoder have a problem with the real-time encoding of the 360-degree videos.

Security industry is a sunrise industry. Video surveillance system is a subsystem of security system and an important part of technical defense. IT has matured with the development of IT technologies such as multimedia technology, codec technology and network technology. Nowadays, most of the video analysis and management processes are manual or semi-automatic, which is difficult to meet the real-time requirements, and the utilization rate of video information is low. In this paper, the architecture of mobile edge computing technology is analyzed, and a cascaded two-tier support vector machine is designed. The first tier uses two-class support vector machine to find local features in test images, and the second tier uses multi-class support vector machine to find the categories of images. Using the advantages of cloud computing architecture, it provides users with scalable, flexible and centrally integrated video management, video access and video storage services.