Search

• The first study to improve decoder pixel throughput by limiting the number of memory access. • A memory access number constraint-based string prediction (SP) framework is proposed. • A specific memory access number constraint rule is designed for SP-coded coding units. • A constrained rate-distortion-optimization-based string searching method is presented. String prediction (SP) is a highly efficient screen content coding (SCC) tool that has been adopted in international and Chinese video coding standards. SP exhibits a highly flexible and efficient ability to predict repetitive matching patterns. However, SP also suffers from low throughput of decoded display output pixels per memory access, which is synchronized with the decoder clock, due to the high number of memory accesses required to decode an SP coding unit for display. Even in state-of-the-art (SOTA) SP, the worst-case scenario involves two memory accesses for decoding each 4-pixel basic string unit across two memory access units, resulting in a throughput as low as two pixels per memory access (PPMA). To solve this problem, we are the first to propose a technique called memory access number constraint-based string prediction (MANC-SP) to achieve high throughput in SCC. First, a novel MANC-SP framework is proposed, a well-designed memory access number constraint rule is established on the basis of statistical data, and a constrained RDO-based string searching method is presented. Compared with the existing SOTA SP, the experimental results demonstrate that MANC-SP can improve the throughput from 2 to 2.67 PPMA, achieving a throughput improvement of 33.33% while maintaining a negligible impact on coding efficiency and complexity. [ABSTRACT FROM AUTHOR]

Previous research demonstrated that filtering Macro-Pixels (MPs) in a decoded Light Field Image (LFI) sequence can effectively enhances the quality of the corresponding Sub-Aperture Images (SAIs). In this paper, we propose a deep-learning-based quality enhancement model following the MP-wise processing approach tailored to LFIs encoded by the Versatile Video Coding (VVC) standard. The proposed novel Res2Net Quality Enhancement Convolutional Neural Network (R2NQE-CNN) architecture is both lightweight and powerful, in which the Res2Net modules are employed to perform LFI filtering for the first time, and are implemented with a novel improved 3D-feature-processing structure. The proposed method incorporates only 205K model parameters and achieves significant Y-BD-rate reductions over VVC of up to 32%, representing a relative improvement of up to 33% compared to the state-of-the-art method, which has more than three times the number of parameters of our proposed model. • Filtering macro-pixels in light field images compressed by versatile video coding. • Novel improved 3D Res2Net module with introduced spatial attention. • Lightweight quality enhancement network which incorporates only 205K parameters. • Achieving up to 32% BD-rate savings over versatile video coding. [ABSTRACT FROM AUTHOR]

Due to the high resolution of 360-degree video, it needs more time to compress using the latest Versatile Video Coding (VVC) compared to traditional video. Meanwhile, the introduction of quad-tree plus multi-type tree (QTMT) in VVC enhances compression performance, but the coding complexity also increases significantly. This paper proposes a fast intra coding unit (CU) partition algorithm for 360-degree video. First, we create a large-scale CU partition mode (CPM) dataset. Next, we design an adaptive CU partition decision network (ACPD-Net) based on adaptive QP, which can simultaneously predict the optimal CPM of four 32 × 32 sub-CUs from the global texture information of 64 × 64 CU. Then, based on the partitioning characteristics of the mapped 360-degree video, the partition structure of 32 × 32 CU is simplified from QTMT to QTBT. The experimental findings indicate that the proposed method can decrease the complexity of intra-frame coding by 61.10% and increase BDBR by 1.99% on average. [ABSTRACT FROM AUTHOR]

Current training data-dependent physical attacks have limited applicability to privacy-critical situations when attackers lack access to neural networks' training data. To address this issue, this paper presents a hierarchical adversarial patch generation framework considering data privacy, utilizing proxy datasets while assuming that the training data is blinded. In the upper layer, Average Patch Saliency (APS) is introduced as a quantitative metric to determine the best proxy dataset for patch generation from a set of publicly available datasets. In the lower layer, Expectation of Transformation Plus (EoT+) method is developed to generate patches while accounting for perturbing background simulation and sensitivity alleviation. Evaluation results obtained in digital settings show that the proposed proxy-data-based framework achieves comparable targeted attack results to the data-dependent benchmark method. Finally, the framework's validity is comprehensively evaluated in the physical world, where the corresponding experimental videos and code can be found at here. • A proxy-data-based physical attack framework to reconcile robustness evaluation and private data protection. • A novel metric for determining the proxy dataset form publicly available datasets. • A new adversarial patch generation method that allows for lower limit improvement on adversarial strength. [ABSTRACT FROM AUTHOR]

Video content tampering is a growing concern. While tampering, perpetrator needs to decompress and re-compress a video. Thus, detecting whether a video has undergone double compression is an important issue in video forensics. In this paper, a novel method is proposed for the detection of High Efficiency Video Coding (HEVC) double compression. Firstly, theoretical and statistical analysis are stated on the quality degradation during double compression, and the impact of Quantization Parameter (QP) on Transforming Unit (TU) and Discrete Cosine Transform (DCT) coefficients distribution. Then 3 sub-features based on Boundary Effect of TU and Non-Zero DCT Coefficients are calculated. Further, 3 sub-features are combined into a 26-dimension feature as the proposed detection feature, which is finally fed to the Multilayer Perceptron (MLP) classifier. Experiments are conducted on video sets with different settings, and the result proves that our method achieves better performance under several situations compared with two existing methods. • A novel method for HEVC video double compression detection under different QPs. • Boundary effect of TU is proposed to construct sub-feature. • Non-zero DCT coefficient distribution is analyzed for constructing sub-features. • Experimental results address the effectiveness of various video setting. [ABSTRACT FROM AUTHOR]

• IDCT and FDCT written in Halide. • 200 Lines of Halide replace over 20,000 lines of Assembly Language. • 5-25% faster for decoding. • 10-40% faster for encoding. • Significantly easier to maintain and port to new architectures. The Discrete Cosine Transform (DCT) is commonly used for image and video coding and very efficient implementations of the forward and inverse transforms are of great importance. The popular libjpeg-turbo library contains handwritten, highly-optimised assembly language DCT implementations utilizing SIMD instruction sets for a variety of architectures. We present an alternative approach, implementing the 8x8 IDCT and FDCT written in the functional image processing language Halide. We show how less than 200 lines of Halide can replace over 20,000 lines of code the libjpeg-turbo library to perform JPEG encoding and decoding. The Halide implementation is compared for ARMv8 NEON and x86-64 SIMD extensions and shows a 5–25 percent performance improvement over the SIMD code in libjpeg-turbo for decoding and a 10–40 percent improvement for encoding. The Halide code is significantly easier to maintain and port to new architectures than the existing code. [ABSTRACT FROM AUTHOR]

Animals use vigilance to detect or monitor threats. While numerous aspects of vigilance have been studied across a wide range of species, little work has explored the methodological variation that has emerged across these studies. Different approaches in sampling designs, statistical analyses and definitions can make cross-study comparisons challenging and potentially obscure our understanding of animal vigilance. In this study we explore two important components of vigilance definitions and ask (1) whether definitions vary in their interobserver agreement, and (2) whether using different definitions can create varied results within and across observers. Separate groups of 'experienced' and 'inexperienced' observers extracted data from video focal observations of wild chacma baboons, Papio ursinus , using four different definitions representative of the variation found within primate vigilance literature. In the first stage of analysis, we found that the four definitions varied in their interobserver agreement, with only an operational-looking definition performing well across both duration and frequency assessments, and an experienced/inexperienced dichotomy. This suggests definitions vary in how well observers can converge on similar interpretations of the same definition. The second part of the analysis used the experienced group's data in a typical primate vigilance analysis and found results varied within observers across definitions, i.e. definition effects, and across observers within definitions, i.e. interpretation effects. Together these results suggest that variation in definitions and their interpretation could have a fundamental role in producing between-study differences in results. Future vigilance research must consider these factors and explore working towards a single framework for studying vigilance, particularly within taxonomic families. Without consistency, cross-study comparisons are likely to be challenging and future observational work on other behaviours may also benefit from exploring these types of definitional issues. For baboons, operationalized definitions appear the most consistent across observers; however, future research should explore its application in other taxa. • Variation in vigilance definitions has grown, both within and between species. • Observers coded videos of baboons using definitions derived from primate research. • Definitions varied in their interobserver agreement and the results produced. • Operationalized definitions improved consistency of estimates across observers. • Similar assessment frameworks are required in other taxa. [ABSTRACT FROM AUTHOR]

As the latest generation of video coding standards, versatile video coding (VVC) introduces several new coding tools in transform coding to concentrate the energy of residual blocks. In this work, we propose a regular multiplier (RM) based hardware architecture for 2D transform that can process different transform sizes and types using a unified architecture. This architecture consists of a 1D row transform, a transpose memory, and a 1D column transform, allowing for processing 16 coefficients per cycle. For the 1D row/column transform, the unified calculation structure consists of 512 regular multipliers supporting various transform sizes and types. For the transpose memory, we design an SRAM-based diagonal storage approach, along with a FIFO for storing the block information. Compared with the state-of-the-art MCM-based design, the proposed RM-based design shows a 30.9% reduction in area while can operate at a higher frequency. [ABSTRACT FROM AUTHOR]

Nowadays, the use of video data for machine (VCM) tasks has become increasingly prevalent, with deep learning and computer vision requiring large volumes of video data for object detection, object tracking, and other tasks. However, the features required for machine tasks are different from those used by humans, and a new approach is needed to encode and compress video data for machine consumption. Video coding for machines has received considerable attention, with many approaches focusing on compressing features rather than the video itself. However, a key challenge in this process is repacking the features in an efficient and effective manner. This paper proposes a distance-based patch tiling and intra-block quilting method to repack feature sequences in a manner that is better suited for existing video codecs, based on statistical analysis of feature characteristics in the channel dimension. Experimental results demonstrate that our method achieves an 65.54% BD-rate gain compared to benchmark methods. This research has significant implications for improving the efficiency of video coding for machine applications, and future work could explore the use of feature dimensionality reduction and combination of neural network (NN) codec to optimize the repacking of features for compression. [Display omitted] • Video data accounts for most internet traffic, video consumed by machines outnumbers video consumed by human. • Feature compression is an effective solution for coding for Machines with advantages of high compression ratio, computational offloading and privacy protection, while maintaining task accuracy. • Repacking features into a pseudo video to fit into existing video codecs demonstrates a performance gain. • Features is repacked based on statistical characteristics in the channel dimension. • Feature similarity is calculated by Mean Squared Error or binary distance. [ABSTRACT FROM AUTHOR]

High Efficiency Video Coding (HEVC) −based steganography has gained attention as a prominent research focus. Especially, block structure based HEVC video steganography has received increasing attention due to commendable performance. However, current block structure- based steganography algorithms confront with challenges such as reduced coding efficiency and limited capacity. To avoid these problems, an adaptive video steganography algorithm based on Prediction Unit (PU) partition mode in I-frames is proposed. This is done through the analysis of the block division process and the visual distortion resulting from the modification of the PU partition mode in HEVC. The PU block structure is utilized as steganographic covers, and the Rate Distortion Optimization (RDO) technique is introduced to establish an adaptive distortion function for Syndrome-trellis code (STC). Further comparison is performed between the proposed method and the state-of-the-art steganography algorithms, confirming its advantages in embedding capacity, compression efficiency, visual quality, and resistance to video steganalysis. [ABSTRACT FROM AUTHOR]

Eating behaviour in children is a matter of study for which diverse tools have been designed. Coding systems for videotaped meals allow the extraction of detailed in vivo information; however, there is no tool available for infants following a Baby-Led Weaning (BLW) method. This study aimed to create and validate a new tool to assess eating behaviour in infants during weaning, applicable regardless of the complementary feeding method. The Baby Eating Behaviour Coding System (BEBECS) was developed comprising time variables, behaviours, feeder-led actions, and other meal-related variables. Sixty videos of infants aged 6–18 months following spoon-feeding (SF) or BLW methods were coded by two trained coders. These scores were analysed together with intake and maternal ratings of liking and calmness. Additionally, combined analysis and internal comparison assessed the possible differences in BEBECS variables between SF and BLW. Inter-rater and test-retest reliability had good to excellent agreement: Cohen's Kappa >0.75, Lin's CCC >0.70, and Intraclass Correlation Coefficient >0.75, for almost all variables. Infants' liking and intake of the offered food correlated positively with meal duration and total count of mouth approaches but negatively with having leftovers and time between mouth approaches. Infants' calmness and tiredness were negatively correlated. More food than initially offered was available during the meal in BLW but not in SF. There was a tendency towards more autonomous behaviour in BLW infants regarding changes observed in the time the food was in the mouth at each stage (6, 12, and 18 months). In conclusion, BEBECS has the potential to be a valid tool for application in the research of infant eating behaviour during weaning by trained coders. [ABSTRACT FROM AUTHOR]

The aim of video anomaly detection is to detect anomalous events in a video sequence. In an unsupervised setting, enhancing detection accuracy hinges on the ability to learn normal features during the training phase and subsequently generate large errors when abnormal video frames are encountered during the testing phase. The transformer is an innovative neural network that utilizes a self-attention mechanism to extract intrinsic features, thereby proving more effective in extracting normal features. When paired with convolutional neural networks (CNNs), known for their proficiency in local information extraction, this hybrid architecture becomes particularly adept at handling numerous vision tasks. However, research exploring the full potential of such a hybrid architecture network for video anomaly detection is still in its early stages. In this paper, we introduce a novel approach to integrating transformers and CNNs for video anomaly detection. Here, the transformer functions as a memory module (TransMem) that processes latent features and incorporates them into CNN-based autoencoders (AEs). This approach significantly reduces computational complexity compared to directly processing video frames. Moreover, unlike other similarity-based memory methods, the proposed memory module is learnable. TransMem is a lightweight, plug-and-play module that can be seamlessly integrated into other complex frameworks to further enhance detection accuracy. Extensive experiments have demonstrated the effectiveness of our proposed method. • We modify the memory module by using transformer architecture. • The training procedure will generate adaptive memory items. • Our memory module is efficient and plug-and-play. • The results prove the generated memory items contain more global normal patterns. [ABSTRACT FROM AUTHOR]

Thus far, Multiview High-Efficiency Video Coding (MV-HEVC) can only use a central processing unit (CPU) to perform decompression on a personal computer (PC) or workstation. Because MV-HEVC is much more complex than High-Efficiency Video Coding (HEVC), decompressors need higher parallelism to decompress in real time. Therefore, this study presents a parallel method based on MV-HEVC. Inter-view complete parallelism is realized according to the dependent relationship between other MV-HEVC views, and a search range is not required to avoid the data dependence between frames. Based on the dependencies of each task in MV-HEVC, an advanced wavefront parallel processing method is proposed to achieve higher intra-frame parallelism. The parallel structure of the proposed method is compatible with that of the single-instruction multiple-data acceleration method. The results showed that the proposed method can decompress MV-HEVC with 20 threads in real time for 1088p video with three views. • We presented a parallel method based on Multiview High Efficiency Video Coding. An advanced WPP method is proposed to achieve higher intra-frame parallelism. • The proposed method is compatible with that of the SIMD acceleration method. • The proposed method can achieve real-time decompression of MV-HEVC with three views. [ABSTRACT FROM AUTHOR]

• A fast coding method is proposed for HEVC inter-frame. • First, the characteristics of HEVC inter-frame coding are analyzed. • Then, an Inter-Frame Feature Transfer-LSTM (IFFT-LSTM) model is designed to accelerate CTU partition decision. • Finally, an algorithm flowchart for fast HEVC inter-frame coding is formulated. • The proposed method can reduce HEVC inter-frame coding complexity significantly with acceptable subjective video quality. High Efficiency Video Coding (HEVC) is the most commonly used video coding standard. However, its high coding complexity is a heavy burden for real-time video applications. But, coding tools designed based on traditional coding frameworks have reached limits. Furthermore, existing low-complexity video coding methods have not thoroughly analyzed the characteristics of compressed video, making it impossible to develop targeted models to reduce coding complexity. Therefore, in this research, a fast HEVC inter-frame coding technique is proposed. Firstly, we perform a characteristics analysis of HEVC inter-frame coding to explore the correlation between video frames. Secondly, we develop an Inter-Frame Feature Transfer-Long Short-Term Memory (IFFT-LSTM) model to obtain the optimal coding tree unit (CTU) partition structure. Thirdly, we embed the IFFT-LSTM model into the HEVC test platform. The experimental results show that suggested method can effectively reduce the HEVC inter-frame coding complexity with a small amount of rate-distortion (RD) performance loss while maintaining acceptable subjective video quality. [ABSTRACT FROM AUTHOR]

• Conditionally and directly skip all non-first pre-coding. • In I picture, for a CU with top-left location (x, y), if both x and y are multiples of 64, then each non-first pre-coding is skipped. • In I picture, for a CU with top-left location (x, y), if at least one of x and y is not multiples of 64 and a set of conditions is satisfied by a non-first pre-coding, then the non-first pre-coding is skipped. • In non-I picture, for CUs at any location, if a set of conditions is satisfied by a non-first pre-coding, then the non-first pre-coding is skipped. • Direct Non-First Pre-coding Skip (DNFPS) significantly reduce the coding complexity with negligible impact on coding efficiency. The third generation of Audio Video Coding Standard (AVS3) adopts sophisticated block partitioning structure with a multi-type tree, which includes Quad-Tree (QT), Binary-Tree (BT), and Extend Quad-Tree (EQT). During multi-type tree recursive block partitioning and pre-coding process of encoding, each Coding Unit (CU) of each Size at each Location (CUeSeL) of the picture is repeatedly pre-coded many times when visiting each branch of each tree. Although there are early termination mechanism, the number of visits for each CUeSeL is still very high and up to more than fifty. This process brings very high coding complexity. In order to reduce the coding complexity of AVS3, this paper proposes a method named Direct Non-First Pre-coding Skip (DNFPS) for a CUeSeL to directly and completely skip all non-first pre-coding under specific conditions. The conditions include, but are not limited to, those related to the first pre-coding. The contributions of the paper are as follows. In I pictures, for a CUeSeL with top-left location (x, y), if both x and y are multiples of 64 or if a set of conditions related to first pre-coding is satisfied, then each non-first pre-coding is skipped, and the coding result of the first pre-coding is reused for each non-first pre-coding skipped. In non-I pictures, each non-first pre-coding is skipped, and the coding result of the first pre-coding is reused for each non-first pre-coding skipped, provided a set of conditions related to first pre-coding is satisfied. The experimental results demonstrate that DNFPS has a negligible impact on coding efficiency under All Intra (AI) configurations, and coding runtime is reduced by 16 %. [ABSTRACT FROM AUTHOR]

With the developing explosion in video data delivery, perceptual video coding (PVC) plays an increasingly significant role in video compression. The just noticeable distortion (JND) reflects the tolerance limit of human visual system (HVS) to coding distortion directly, resulting in that JND-based PVC is the most important branch of video coding. This paper provides an extensive overview of JND estimation and JND-based PVC, so as to make interested readers aware of the status quo. The main contribution of this article can be briefly outlined as follows. Firstly, the general description of JND concepts and most existing computation models for JND are to be reviewed systematically. Secondly, most related works about JND-based perceptual image and video coding schemes are introduced, including JND-based coding preprocessing and JND-based codec embedding. Thirdly, in addition to a thorough summary of JND estimation and JND-based PVC, possible future directions and opportunities are analyzed and discussed. • Firstly, the general description of JND concepts and most existing computation models for JND are to be reviewed systematically. • Secondly, most related works about JND-based perceptual image or video coding schemes are introduced, including JND-based coding preprocessing and JND-based codec embedding. • Thirdly, in addition to a thorough summary of JND estimation and JND-based PVC, possible future directions and opportunities are analyzed and discussed. [ABSTRACT FROM AUTHOR]

Reversible data hiding (RDH), also known as lossless data hiding, ensures the losslessly recovery of both the cover medium and the secret message at the decoder. For the video medium, RDH can also provide the effective protection and has attracted a lot of attention due to its unique reversibility. This paper aims to present a systematic review about the existing video RDH methods and the corresponding practical applications. We summarize the basic frameworks and the current optimization aspects in video RDH methods, including (1) the determination for the suitable embedding position, (2) the drift distortion minimization, (3) the extension for security improvement. Then, the fundamental trade-off for the evaluation metrics is quantified, and the performance comparison of the classical methods is given. Finally, the future research directions are discussed to exploit the further idea for this area. • We provide a comprehensive survey of reversible data hiding (RDH) in H.264 and HEVC. • A detailed analysis of representative methods in video RDH is given and the typical frameworks are summarized. • The optimization directions for video RDH are discussed and the valuable future researches are estimated. [ABSTRACT FROM AUTHOR]

• An adaptive HEVC video steganography algorithm is designed, which can effectively resist the PU shift-based steganalysis. • The introduction of multivariate STC allows for smaller modifications with the same embedding capacity. • The superiority of this algorithm over the state-of-the-art video steganography algorithm is verified through extensive experiments. Calibration-based steganalysis methods are a fatal threat to current HEVC steganography algorithms. The current steganography algorithms based on prediction unit (PU) still lack resistance to this type of steganalysis. The reason is that the improvements in HEVC codec provide more potential steganographic space as well as introduce more detectable distortion. Therefore, a HEVC video steganography method that has resistance to PU shift-based steganalysis is proposed in this paper. First, the PU shift phenomenon that exists in PU-based steganography is introduced. Then the existing calibration-based steganalysis method is enhanced. According to the encoding rules of HEVC, the distortion cost function according to the difference of rate distortion cost (RD cost) during recompression is designed, which aims to obtain better resistance to calibration-based steganalysis. Experimental results demonstrate that the steganography algorithm designed in this paper provides excellent resistance ability for calibration-based steganalysis. [ABSTRACT FROM AUTHOR]

• Novel excitation waveform is introduced for enhanced optical infrared thermography. • The waveform is discretely modulated with frequency and phase modulation codes. • Depth resolvability is evaluated based on the concept of thermal wave radar. • Optimal modulation codes are explored for maximized depth resolvability. • The optimized waveforms show superior performance compared to existing waveforms. Thermal wave radar (TWR) is a state-of-the-art non-destructive testing method, inspired by radio wave radar systems, in order to increase depth resolution and signal to noise ratio of optical infrared thermography through pulse compression. Analogue frequency modulation (i.e. frequency sweep) and Barker binary phase modulation are the two popular and widely researched pulse compression techniques in TWR among which Barker coding has shown the highest performance. This paper introduces a novel modulated waveform with variable discrete frequency-phase modulation (FPM) which distinctively enhances the depth resolvability of TWR compared to the existing techniques. The pulse compression quality and depth resolvability of the novel FPM waveform is initially evaluated through a 1D analytical solution. The analogue frequency modulated and discrete phase modulated waveforms as well as mono-frequency excitation (i.e. lock-in thermography) are also evaluated at the same central frequency as the reference. Objective functions are defined and a large search space is explored for optimal modulation codes. Two FPM waveforms are selected based on their maximized depth resolvability through resultant lag and phase in the output channel of TWR. Furthermore, the excellent performance of the selected FPM waveforms is validated by 3D finite element simulation. A delaminated glass fiber reinforced polymer (GFRP) laminate is simulated in order to evaluate the impact of a dominant lateral heat diffusion on the performance of the novel FPM waveforms. The superior depth resolvability of the introduced FPM waveforms is confirmed and their robustness at various noise levels is demonstrated. [ABSTRACT FROM AUTHOR]

This paper introduces a novel video saliency model for salient object detection in videos. Firstly, we generate multi-level deep features via a symmetrical convolutional neural network, in which the inputs are the current frame and the optical flow image. Then, the multi-level deep features are integrated in a hierarchical manner using a fusion network, which deploys attention module to make a selection for deep features. Lastly, the integrated deep feature is combined with the boundary information originated from shallow layer of the feature extraction networks, and the saliency map is generated by the saliency prediction step. The key advantages of our model lie on the attention module, the hierarchical integration and the boundary information, in which the former one acts as weight filter and is used to select the most salient regions in deep features, the middle one gives an effective integration manner for features from different layers and the last one provides well-defined boundaries for saliency map. Extensive experiments are performed on two challenging video dataset, and the experimental results show that our model consistently outperforms the state-of-the-art saliency models in a large margin. [ABSTRACT FROM AUTHOR]

• A study on the possibility of avoiding the HEVC intra-frame drift effect. • Theoretically computing a mask multiplicative matrix cancelling the intra-frame drift effect. • Experimental illustrations and benchmarking over state-of-the-art methods. • Future work on the applicative integration under the watermarking framework. • Paving the way towards studying the intra-drift for the HEVC extensions. This paper investigates the possibility of avoiding the HEVC intra-frame drift effect induced by additive modifications of the luma DCT/DST coefficients. The main novelty consists in solving the intra-drift problem in its general form: starting from the equations defining the 35 intra-prediction modes and the DCT/DST computation, a mask multiplicative matrix cancelling the intra-frame drift effect is computed. The relevance of the advanced method with respect to the state of the art studies is illustrated, discussed and quantitatively assessed. [ABSTRACT FROM AUTHOR]

Highlights • Nonapeptides regulate prairie vole pair bonds but are understudied in other species. • Pair maintenance mechanisms are understudied. • Oxytocin antagonist (OTA) reduced following in zebra finches during pair maintenance. • Allopreening bouts declined in OTA females and OTA males reduced song bouts. • Pecking and clumping (side-by-side perching) were unaffected by OTA in either sex. Abstract The nonapeptides (oxytocin, vasopressin, and their non-mammalian homologs) regulate a number of social behaviors across vertebrates including monogamous pair bonds in mammals. Recent work on zebra finches has shown an important role for these neurohormones in establishing avian pair bonds as well. However, studies on the role of nonapeptides in maintaining pair bonds after pair formation are lacking. The goal of the present study was to investigate the effects of an oxytocin receptor antagonist (OTA) on pair maintenance behaviors in the monogamous zebra finch. I injected established zebra finch pairs over three days with either 5 μg of an OTA or a vehicle control, and separated the partners for one hour, after which partners were reunited and their reunion recorded on video for 30 min. Videos were then coded to measure singing, affiliative (allopreening, clumping, following), and aggressive (pecking) behaviors. These behaviors were also measured both on the day before injections to establish a pre-treatment level and two days after the last injection. Control and antagonist treated birds did not differ in the amount of time spent clumping or the frequency of pecking across the experiment. However, both male and female zebra finches that received OTA significantly reduced the amount of time spent following their partner. Females given the OTA treatment reduced allopreening and males given the OTA treatment reduced the frequency of singing bouts directed at their partners relative to controls. These results suggest that the nonapeptides play a role in regulating some, but not all, pair maintenance behaviors in experienced zebra finches. [ABSTRACT FROM AUTHOR]

With the rapid development of network and multimedia technology, a large number of sports and national fitness information are stored in various fitness guidance systems in the form of video and pictures. In order to better promote public fitness and facilitate learning and viewing, sports video has a variety of needs of editing, segmentation and integration. Aiming at the shortcomings of current sports video image segmentation methods, such as rough segmentation results and high spatial distortion rate, a sports video image segmentation method based on fuzzy clustering algorithm is proposed. This paper introduces the basic theory of fuzzy clustering algorithm, establishes second-order fuzzy attributes with normal distribution and gray value by means of time-domain difference images, assigns the fuzzy attribute S membership function, then performs fuzzy clustering on time-domain difference images, and obtains the segmentation results of moving video images by edge detection. The experimental results show that the method has high spatial accuracy, good noise iteration performance and low spatial distortion rate, and can accurately segment complex moving video images to obtain high-definition images. [ABSTRACT FROM AUTHOR]

Abstract This paper presents a video coding scheme tailored for traffic surveillance videos, which features a pre-built library that is utilized in both encoder and decoder to pursue higher compression efficiency. We are motivated by the observation that, in traffic surveillance videos, not only the background is steady for a long while, but also the foreground (e.g. vehicles) contains high redundancy. For example, in the video taken by a traffic monitoring camera, we can observe that the passing-through vehicles are usually similar. However, the redundancy in the vehicles and the background is not fully exploited in the current video coding schemes. To address this problem, we propose a library-based video coding scheme. Specifically, for each static monitoring camera, we can collect video in a period, and build a library of vehicles and background from that video. When encoding the following video of that camera, we utilize the pre-built library by searching similar vehicles and background from the library and using the retrieved vehicles and background to help encode. Accordingly, the decoder also refers to the same library to reconstruct the video. We design efficient algorithms for building the library, searching in the library, as well as utilizing the library for encoding/decoding, to fulfill the proposed scheme. Our scheme is implemented upon the state-of-the-art video coding system–High Efficiency Video Coding (HEVC), and is tested on our own collected traffic surveillance videos. Experimental results show that, compared to the HEVC anchor, our proposed scheme achieves as high as 47.0%, 37.1%, and 34.8% BD-rate reduction, under random-access, low-delay B, and low-delay P settings, respectively. Our scheme also achieves higher compression efficiency than the existing background-based methods for surveillance video coding. [ABSTRACT FROM AUTHOR]

Abstract Perceptual quality evaluation of multiply distorted images has become a very challenging research topic. In this paper, we present a novel and efficient deep blind quality evaluator for multiply distorted images based on monogenic binary coding (MBC). Local complementary structural information and a deep learning method are employed to blindly evaluate the quality of multiply distorted images. First, a monogenic signal representation is utilized to decompose a multiply distorted image into three complementary components: orientation, phase, and magnitude. The quality-predictive features are then determined from the complementary components. Finally, the features are mapped to the human quality score of the multiply distorted image based on the deep neural network. The results on two newly established multiply distorted image subjective databases confirm that our metric has a better prediction performance than existing state-of-the-art full-reference and classical blind metrics. [ABSTRACT FROM AUTHOR]

Highlights • The R-D relationship of the screen content video is related to video contents. On the basis of this fact, CTUs in screen content videos are classified into three types: T-CTUs, SI-CTUs and NI-CTUs according to their pixel activity and DCT coefficient distribution. • Disparate CTUs present vastly different R-D relationships. For this reason, instead of the unitive parameter update method in the R- λ model, the parameters used to code specific kind of CTU are updated according to the information of the last same kind of coded CTU rather than the last CTU. • Before assigning bits to each CTU, the bits allocated to text region, screen image region and nature image region are determined. When a scene change occurs, the bit allocation method which relied on the distortion and CTU number changes into the one only related to the number of CTUs. Abstract The popularity of 3D screen content videos like 3D animations calls for better coding performance of screen content videos. The exiting R- λ rate control model is designed to enable the coding performance of conventional nature videos, which can not accurately describe the Rate-Distortion (R-D) relationships of the screen content videos, especially for videos containing limited colors and sharper edges. This paper proposes a new rate control (RC) scheme which takes the divergent characteristics of text contents, screen images and nature images in screen content videos into consideration. In particular, the content-based rate control algorithm is developed at Coding Tree Unit (CTU) level, where CTUs are divided into three categories including Text-CTUs (T-CTUs), Screen Image-CTUs (SI-CTUs) and Nature Image-CTUs (NI-CTUs). Three R- λ models reflecting different R-D relationships of different CTUs are established, and the model parameters are updated on the basis of their own models. Furthermore, in view of the fact that continuity and mutability simultaneously exist in screen content videos, two bit allocation schemes for regions containing only one content are determined in this paper. Experimental results show that the proposed RC scheme achieves 0.749 dB BDPSNR increase under the Low Delay configuration and 0.637 dB BDPSNR increase under the Random Access configuration on average, compared to the default R- λ model in HEVC Screen Content Coding extension (HEVC-SCC) reference software. [ABSTRACT FROM AUTHOR]

Highlights • We model the RD cost comparison as a MDP for low complexity video coding. • We learn a CU early termination classifier with end-to-end actor-critic RL algorithm. • A CU early termination algorithm independent of depths is derived for whole CTU. Abstract In this paper, we propose a Reinforcement Learning (RL) based Coding Unit (CU) early termination algorithm for High Efficiency Video Coding (HEVC). RL is utilized to learn a CU early termination classifier independent of depths for low complexity video coding. Firstly, we model the process of CU decision as a Markov Decision Process (MDP) according to the Markov property of CU decision. Secondly, based on the MDP, a CU early termination classifier independent of depths is learned from trajectories of CU decision across different depths with the end-to-end actor-critic RL algorithm. Finally, a CU decision early termination algorithm is introduced with the learned classifier, so as to reduce computational complexity of CU decision. We implement the proposed scheme with different neural network structures. Two different neural network structures are utilized in the implementation of RL based video encoder, which are evaluated to reduce video coding complexity by 34.34 % and 43.33 %. With regard to Bj ø ntegaard delta peak signal-to-noise ratio and Bj ø ntegaard delta bit rate, the results are −0.033 dB and 0.85 % , −0.099 dB and 2.56 % respectively on average under low delay B main configuration, when compared with the HEVC test model version 16.5. [ABSTRACT FROM AUTHOR]

Abstract Active defense technology is very important in intelligent systems and video surveillance. In some important fields, active defense system can effectively find intruders. Many intelligent video surveillance systems were proposed in recent years. They achieved good performance to some extent. Since power station is an important field, it is important to develop an intelligent video surveillance. Considering that detecting the whole surveillance video is time consumption and computation. So in this paper, we propose an active defense system to find intruders automatically. First, a key frame selection algorithm based on adaptive features is presented to select key frames. These key frames can cover the main content of videos and detecting these key frames can also reduce time consumption and computation. Then, a probabilistic model is proposed to learn the training data distribution. Finally, our system can achieve active defense based on probabilistic model. Experimental results show that our active defense system can achieve finding intruders effectively. [ABSTRACT FROM AUTHOR]

Highlights • In this paper, we propose a novel reduced-reference quality assessment metric for image super-resolution (RRIQA-SR) by two components: the energy change in low-frequency regions, which can be used to capture the global distortion in SR images, and texture variation in high-frequency regions, which can be used to capture the local distortion in SR images. • The overall quality of SR images is estimated by perceptual similarity calculated by energy change and texture variation between local image patches of LR and HR images. Abstract In this paper, we propose a novel reduced-reference quality assessment metric for image super-resolution (RRIQA-SR) based on the low-resolution (LR) image information. With the pixel correspondence, we predict the perceptual similarity between image patches of LR and SR images by two components: the energy change in low-frequency regions, which can be used to capture the global distortion in SR images, and texture variation in high-frequency regions, which can be used to capture the local distortion in SR images. The overall quality of SR images is estimated by perceptual similarity calculated by energy change and texture variation between local image patches of LR and HR images. Experimental results demonstrate that the proposed method can obtain better performance of quality prediction for SR images than other existing ones, even including some full-reference (FR) metrics. [ABSTRACT FROM AUTHOR]

Highlights • In this paper, a novel multi-view NMF algorithm for scene categorization. • Human gaze shifting path reflect the sequence of objects that human focus on. Abstract Accurately discriminating complicated sceneries from different categories is a useful technique in multimedia and computer vision. In this work, we propose a novel multi-view non-negative matrix factorization to detect human gaze behavior, which is subsequently integrated into an image kernel machine for scene categorization. More specifically, we first project regions from each scenery into the so-called perceptual space, which is established by combining color, texture, and semantic features. Then, a novel non-negative matrix factorization (NMF) algorithm is developed which decomposes the regions' feature matrix into the product of the basis matrix and the sparse codes. The sparse codes indicate the saliency level of different regions which is used to constructed gaze shifting path. Thereby, the path from each scenery is derived and further incorporated into an image kernel for scene categorization. Comprehensive experiments on six scenery data sets have demonstrated the superiority of our method over a series of recognition models. [ABSTRACT FROM AUTHOR]

In response to the increased demand for high-resolution video, the new generation of video standards, High Efficiency Video Coding (HEVC) and its scalable extension (SHVC) have been finalized. The compression of HEVC/SHVC is efficiently improved and supports ultra-high resolution (UHD). Therefore, the coding complexity of HEVC/SHVC is much higher than those of previous standards. The framework of SHVC is based on HEVC and is divided into several types of scalable video. SHVC can be decoded into various video resolutions, frame rates and qualities, and only needs to be encoded once, but with higher complexity than HEVC. Thus, how to reduce the coding complexity of SHVC is the purpose of this paper. Our proposed algorithm accelerates the enhancement layer (EL) prediction by utilizing encoded Coding Unit (CU) sizes, prediction modes, motion vectors and Rate-Distortion Costs (RD-Costs) of the base layer (BL) and encoded CU sizes of the enhancement layer for quality scalability of SHVC. Experimental results show that the proposed algorithm can save lots of time while maintaining good video quality, and the performance is better than those of previous works. [ABSTRACT FROM AUTHOR]

The goal of this work is to address two limitations in autoencoder-based models: latent space interpretability and compatibility with unstructured meshes. This is accomplished here with the development of a novel graph neural network (GNN) autoencoding architecture with demonstrations on complex fluid flow applications. To address the first goal of interpretability, the GNN autoencoder achieves reduction in the number nodes in the encoding stage through an adaptive graph reduction procedure. This reduction procedure essentially amounts to flowfield-conditioned node sampling and sensor identification, and produces interpretable latent graph representations tailored to the flowfield reconstruction task in the form of so-called masked fields. These masked fields allow the user to (a) visualize where in physical space a given latent graph is active, and (b) interpret the time-evolution of the latent graph connectivity in accordance with the time-evolution of unsteady flow features (e.g. recirculation zones, shear layers) in the domain. To address the goal of unstructured mesh compatibility, the autoencoding architecture utilizes a series of multi-scale message passing (MMP) layers, each of which models information exchange among node neighborhoods at various lengthscales. The MMP layer, which augments standard single-scale message passing with learnable coarsening operations, allows the decoder to more efficiently reconstruct the flowfield from the identified regions in the masked fields. Analysis of latent graphs produced by the autoencoder for various model settings are conducted using unstructured snapshot data sourced from large-eddy simulations in a backward-facing step (BFS) flow configuration with an OpenFOAM-based flow solver at high Reynolds numbers. • A graph neural network autoencoder is developed for latent space interpretability. • Latent graphs can be accessed as time-evolving coherent structures in physical space. • Multi-scale message passing layers are employed to improve reconstructions. • Demonstrations performed using unstructured unsteady turbulent fluid flow dataset. [ABSTRACT FROM AUTHOR]

Corrosion morphology prediction for civil infrastructure using physics-based simulation is computationally challenging due to the coupled multi-physics simulations involved in long-term corrosion prognostics. Machine learning (ML)-based surrogate modeling provides a promising way of overcoming this challenge. This paper presents a physics-constrained ML method for surrogate modeling of a high-fidelity multi-physics pitting corrosion simulation, which is solved using a phase-field method. The proposed surrogate model consists of a convolutional variational autoencoder to reduce the dimension of pitting corrosion shape images and a Bayesian multi-layer perceptron network (also known as the Bayesian Latent Space Time Evolution Network) to model the evolution of the corrosion pit morphology over time. To account for the fact that corrosion damage without repair is an irreversible process, a physics constraint is added to the surrogate model to ensure that the corrosion rate is strictly negative. The proposed physics-constrained surrogate modeling method is compared with a purely data-driven surrogate modeling method which integrates a convolutional neural network autoencoder with a Gaussian process regression (GPR) model-based nonlinear autoregressive exogenous model. The results show that the proposed physics-constrained surrogate model is significantly more accurate than the purely data-driven GPR-based surrogate model. [ABSTRACT FROM AUTHOR]

Affine motion estimation (AME) which is newly introduced in versatile video coding (VVC) plays a significant role in the bit-rate reduction for rotation and zooming scenes. However, it brings the complexity extremely increased. In this paper, a fast AME algorithm is designed to cope with the high-complexity problem. Three crucial stages are involved in the fast algorithm, which is the optimal inter-mode decision based on the coding unit (CU) partition, the fast algorithm in affine motion search (AMS), and the fast inter-mode decision based on a decision tree. Specifically, the optimal inter-mode will be determined straightforwardly when the parent CU is checked whether it is translational motion estimation (TME). Then, three fast algorithms are made in AMS, which are early termination based on control point motion vector (C P M V), early termination of the iteration process, and fast fine granularity C P M V search. Finally, the optimal inter-mode is predicted using a decision tree based on eight essential features. Experimental results show that the proposed algorithm achieves 10.20% and 10.33% encoding time-saving on average under Low Delay B (LDB) and Random Access (RA) configuration, while the BD-Rate loss is only 0.12% and 0.14%, respectively. • Propose a three-stage scheme to decrease the complexity of the affine ME in VVC. • Utilize the CU partition including parent-CU and child-CU to reduce complexity. • Propose a three-step fast algorithm for the fast algorithm in affine motion search. • Find eight key features to predict the optimal inter-mode based on the decision tree model. [ABSTRACT FROM AUTHOR]

Highlights • We proposed a new representation projection method for VR video. • We proposed optimal rotation angle prediction for the proposed projection method. • We proposed a new objective video quality assessment metric for VR video. Abstract VR video is increasingly popular due to the recent advances in VR technology and hardware. The bulky size of VR video, however, impose new challenges in its storage and processing. In this paper, we focus on the research problems of VR video representation and objective quality assessment. Distinct from traditional 2D video, a VR video is displayed and represented in a form of spherical surface. For encoding purpose, a VR video frame needs to be projected to a 2D flat plane. Existing projection methods usually lead to much redundancy or significant violation in the correlation of neighbor pixels, which are not encoding friendly or are creating visible edge artifacts. To alleviate these problems, we propose in this paper our Quadrangle Affine Square Projection (QASP). QASP is a novel representation for VR video frames, which can reduce the redundant pixels over the traditional projections. In particular, all the inner pixels in QASP remain connected, i.e. the correlations between neighbors pixels are well maintained, which is a desirable feature for video encoding and edge-artifact-free viewing experience. Besides, we also investigated in predicting the optimal rotation angle for QASP frames for higher encoding efficiency. In addition to VR video representation, we also investigate in this paper accurate objective quality measurement for VR video. The traditional video quality measurements, e.g. PSNR, are not suitable to measure the quality of VR videos since they will take the redundant pixels into account. In this paper, we propose a new quality measurement for VR videos, named as Resized-PSNR (R-PSNR). With R-PSNR, only the "meaningful" pixels are considered for quality measurement while the redundant pixels are discarded. To evaluate QASP and R-PSNR, experiments are conducted based on standard VR video sequences. The experimental results show that the proposed projection method achieves noticeable improvement over traditional methods, and the proposed quality assessment method outperforms the traditional measurements in terms of consistency with subjective evaluation. [ABSTRACT FROM AUTHOR]

Highlights • Design of adaptive non separable vector lifting scheme for stereo image coding. • Sparse optimization of the lifting operators used with the left and right images. • Development of independent full optimization and joint optimization algorithms. • Benefits of sparse optimization techniques in the context of stereo image coding. Abstract One of the potential 3D imaging techniques relies on the use of stereoscopic systems. The great interest in these systems has resulted in huge amount of data which needs to be compressed for storage and transmission purposes. In this context, vector lifting scheme has been found to be an efficient approach for stereo image coding. For instance, the coding performance depends on the design of the involved lifting operators referred to as prediction and update filters. For this reason, while a non separable vector lifting structure is retained, we investigate different techniques for optimizing sparse criteria to design the filters used with both views. More precisely, an independent full optimization algorithm as well as a joint algorithm will be developed and studied. Simulations performed on different stereo images demonstrate the effectiveness of the proposed sparse optimization algorithms in terms of quality of reconstruction and bitrate saving. [ABSTRACT FROM AUTHOR]

Abstract Motion compensation is the key technique to reduce temporal redundancy in video coding. Interpolation filters are adopted to generate the inter frame prediction for motion compensation with fractional pixel accuracy. In existing video coding standards such as H.264/AVC and HEVC, a set of predefined interpolation filters is adopted in motion compensation. However, predefined interpolation filters cannot adapt to the video content, which may compromise the coding efficiency. In this paper, a content adaptive interpolation scheme is proposed for motion compensation. In the proposed scheme, a set of adaptive interpolation filters is derived for each frame as additional interpolation filters to minimize the inter prediction difference. Rate-distortion optimization is employed to choose between the predefined interpolation filters and the derived adaptive interpolation filters to achieve the best coding performance at the low bit rates. The proposed scheme is implemented into the HM 12.1 software. Experimental results show that the proposed scheme achieves 5.13 percent, 3.42 percent and 4.07 percent bit rate saving on average compared with HEVC under the "low delay P", the "low delay B" and the "random access" configurations respectively. [ABSTRACT FROM AUTHOR]

Highlights • Novel unit prediction method for efficient HEVC-to-SHVC transcoding is proposed. • Two proposed techniques are early termination and adaptive confidence interval. • We are among the first to propose a fast HEVC-to-SHVC transcoding framework. • Proposed method reduces encoding time of SHVC by 74.14% almost without quality loss. To support good video quality of experiences in heterogeneous environments, transcoding an existed HEVC (high efficiency video coding) video bitstream to a SHVC (scalability extension of HEVC) bitstream with quality scalability is highly required. A straightforward way is to first fully decode the input HEVC bitstream and then fully re-encode it with the SHVC encoder, which requires a tremendous computational complexity. To solve the problem, in this paper, a coding unit complexity (CUC)-based prediction method for predictions of CU (coding unit) depth and PU (prediction unit) mode for efficient HEVC-to-SHVC transcoding with quality scalability is proposed to significantly reduce the transcoding complexity. The proposed method contains two prediction techniques, including (i) early termination and (ii) adaptive confidence interval, and predicts the CU depth and PU mode relying on the decoded information from the input HEVC bitstream. Experimental results have shown that the proposed method significantly outperforms the traditional HEVC-to-SHVC method by 74.14% on average in reductions of encoding time for SHVC enhancement layer. [ABSTRACT FROM AUTHOR]

In 3D video coding systems, depth maps are not displayed to the viewers, but provide the geometric information to generate virtual views. To ensure the quality of virtual views, the rate-distortion optimization (RDO) in depth map coding adopts the virtual view distortion as the distortion item. The virtual view distortion comes from the reconstructed color video distortion and depth distortion. It is usually recognized that the virtual view distortion caused by reconstructed color video distortion is independent of that in depth map coding. Preliminary experiments reveal that the virtual view distortion in depth map coding is also influenced by the reconstructed color video distortion. Therefore, we proposed a novel distortion criterion of depth map coding in which the reconstructed color video distortion is modeled and joins into the virtual view distortion calculation. Correspondingly, the associated Lagrange multiplier is also proposed. Experimental results demonstrate that the method by integrating the proposed distortion criterion into RDO for depth map coding can achieve an average 12.72% bitrate saving compared with SSD based RDO method and can also lead a bitrate reduction (0.64%) compared with the existing distortion estimation method in the current 3D-HEVC reference software. With the associated Lagrange multiplier, the proposed distortion criterion can achieve 12.98% bitrate saving compared with SSD based RDO method on average. [ABSTRACT FROM AUTHOR]

Bit starvation resulted from inefficient rate control of 3D video coding deteriorates visual quality of synthesized views. Most region based rate control schemes allocate higher bitrate to regions of interest (ROIs), but bits may be still consumed by early coded units. This paper avoids reducing coding bits in high-cost regions of 3D-HEVC. Instead of ROIs that are determined by humans, high-cost regions sensitive to bit starvation are detected. Region-level bit allocation is achieved by curve fitting of coding statistics, and recursive Taylor expansion (RTE) based bit allocation is adopted to optimally estimates the target bitrate and the QP related λ of LCUs in each region. Based on retained earnings in economics, bits are reserved for LCUs that are sensitive to distortions. Experimental results show that the proposed scheme outperforms both the R-λ model and RTE model in bitrate accuracy with similar R-D performance. [ABSTRACT FROM AUTHOR]

The popularity of multi-view panoramic video has been considerably increased for producing Virtual Reality (VR) content, due to its immersive visual experience. We argue in this paper that PSNR is less effective in assessing objective visual quality of compressed panoramic video than Sphere-based PSNR (S-PNSR), in which sphere-to-plain mapping of panoramic videos is considered. We also argue that S-PSNR is less effective in assessing perceptual visual quality compared with Perceptual PSNR (P-PSNR), which considers the front-center-bias prior of human viewing direction. The conventional Rate Control (RC) schemes of 2-Dimensional (2D) video coding are optimized on PSNR, and thus they are not suitable for panoramic video coding. To optimize S-PSNR and P-PSNR, two novel RC schemes are proposed for panoramic video coding. In particular, we develop objective and perceptual RC formulations, corresponding to optimization on S-PSNR and P-PSNR, respectively. Then, solutions to these two formulations are provided, such that bits can be allocated to each coding block for achieving optimal S-PSNR or P-PSNR in panoramic video coding. Finally, the experiment results validate the effectiveness of the proposed RC schemes in improving S-PSNR and P-PSNR of panoramic video coding. [ABSTRACT FROM AUTHOR]

Fast intra mode decision strategies are proposed to overcome the brute force mode decision for the coding unit (CU) in High Efficiency Video Coding (HEVC). The proposed work improves the rough-mode-decision (RMD) by initializing the candidate intra mode list using the fusion of the Hadamard-cost and the statistical-inference formed using spatial/ temporal correlations. Then an early termination is predicted using optimal stopping theory that addresses early decision for a generic class of decision problems. Subsequently, a novel RD-cost prediction model is developed for early termination that is based on the RD-cost variation in the neighboring CUs with-respect-to their co-located CUs. Experimental results demonstrate that the RMD module of HEVC and the state-of-the-art fast intra mode prediction published method are outperformed by saving up to 0.61% and 0.91% Bjøntegaard delta bit rate (BDBR) on average, respectively. [ABSTRACT FROM AUTHOR]

Adaptive quantization (AQ) proves to be an effective coding tool to improve the performance of video coding. This paper presents a perceptually temporal AQ method to improve the subjective coding performance for High Efficiency Video Coding (HEVC). We first put forward a perceptual quality oriented motion estimation algorithm, which is conducted with a spatial-temporal just noticeable distortion (JND) model. Then one perceptual feature in temporal domain is proposed to develop our AQ method, which can generate different quantization parameter (QP) offsets for each coding unit (CU). The proposed method fully utilizes the temporal and perceptual characteristics of each CU, which can produce more visual-friendly QP offsets distribution. Experiments are conducted on HM16.0 (HEVC reference software), and with SSIM (Structure Similarity Index Metric) as the distortion metric, more than 8.08% and 7.95% rate savings can be obtained for Low-Delay-P (LDP) and Low-Delay-B (LDB) configurations on average, respectively. The subjective quality evaluation demonstrates that the proposed AQ method can achieve comparable visual quality as the HM16.0 while the proposed method can yield remarkable bitrate reductions. [ABSTRACT FROM AUTHOR]

The 3D extension of High Efficiency Video Coding (3D-HEVC) is a new international video coding standard developed by the Joint Collaborative Team on 3D Video Coding Extensions (JCT-3V) in order to support coding of multiple views and its associated depth data. 3D-HEVC aims at improving the coding efficiency of 3D and multi-view videos by introducing new coding tools to utilize the correlations between views and between texture and depth components. In this paper, an inter-view motion prediction (inter-view merge candidate) and an inter-component motion prediction (texture merge candidate) are proposed to explore the inter-view and the inter-component redundancies for texture and depth components, respectively. Moreover, a new coding mode termed as single depth mode which simply reconstructs a coding block with a single depth value based on block merging scheme under the HEVC quad-tree based block partitioning is also introduced. All the proposed schemes are adopted in 3D-HEVC. The experimental results evaluated under the common test conditions (CTC) for developing 3D-HEVC show that the proposed inter-view merge candidate, texture merge candidate, and single depth mode achieve significant BD-rate reductions of 19.5% for dependent texture views and 8.3% for the synthesized texture views. [ABSTRACT FROM AUTHOR]

Cubic-spline interpolation (CSI) scheme is known to be designed to resample the discrete image data based on the least-square method in conjunction with the cubic convolution interpolation (CCI) function. It is superior in performance and can be used together with the discrete cosine transform (DCT)-based image or video codec to improve the coding performance for a variety of high compression ratios. In this paper, we firstly make some comments on the direct computation algorithm for CSI scheme developed by Lin et al. Moreover, a low-complexity direct computation algorithm for CSI scheme is developed to further improve the computational efficiency. The mathematical derivations and simulation results indicate that such simplified CSI scheme using the proposed low-complexity direct computation algorithm can achieve almost the same objective and subjective performance with much fewer arithmetic operations in comparison with the CSI scheme using the direct computation algorithm. [ABSTRACT FROM AUTHOR]