Search

It has been recognized that videos have to be encoded in a rate-distortion optimized manner for high coding performance. Therefore, operational coding methods have been developed for conventional distortion metrics such as Sum of Squared Error (SSE). Nowadays, with the rapid development of machine learning, the state-of-the-art learning based metric Video Multimethod Assessment Fusion (VMAF) has been proven to outperform conventional ones in terms of the correlation with human perception, and thus deserves integration into the coding framework. However, unlike conventional metrics, VMAF has no specific computational formulas and may be frequently updated by new training data, which invalidates the existing coding methods and makes it highly desired to develop a rate-distortion optimized method for VMAF. Moreover, VMAF is designed to operate at the frame level, which leads to further difficulties in its application to today’s block based coding. In this paper, we propose a VMAF oriented perceptual coding method based on piecewise metric coupling. Firstly, we explore the correlation between VMAF and SSE in the neighborhood of a benchmark distortion. Then a rate-distortion optimization model is formulated based on the correlation, and an optimized block based coding method is presented for VMAF. Experimental results show that 3.61% and 2.67% bit saving on average can be achieved for VMAF under the low_delay_p and the random_access_main configurations of HEVC coding respectively.

We develop a technique for generating smooth and accurate 3D human pose and motion estimates from RGB video sequences. Our method, which we call Motion Estimation via Variational Autoencoder (MEVA), decomposes a temporal sequence of human motion into a smooth motion representation using auto-encoder-based motion compression and a residual representation learned through motion refinement. This two-step encoding of human motion captures human motion in two stages: a general human motion estimation step that captures the coarse overall motion, and a residual estimation that adds back person-specific motion details. Experiments show that our method produces both smooth and accurate 3D human pose and motion estimates.

Venetian blinds are widely used to improve the performance of window systems, while their energy-saving potentials vary with different materials. This paper aims to undertake a standardized evaluation of the annual energy-saving potential of venetian blinds in residential buildings, and identify the most influential parameters for design optimization. To achieve this goal, nine parameters were chosen to establish venetian blinds models in WINDOW. Then, 9072 shading models were coupled with building models for annual energy consumption simulation in EnergyPlus, where the shading schedule was derived from questionnaires. Further, an experiment was conducted to verify the reliability of the joint simulation between WINDOW and EnergyPlus. Finally, annual energy performance (AEP) was adopted for a standardized energy efficiency evaluation of various venetian blinds products, and the sensitivity analysis of AEP to various parameters was conducted to identify the key parameters. The results indicated that by adjusting the key parameters of the venetian blinds, cooling energy saving can reach 44% and the heating energy saving can be up to 65%. The significant energy saving potential of venetian blinds indicates the prospect and importance of their use in residential buildings.

The MnBi permanent magnetic alloys with high coercivity was prepared by a low-energy ball milling process. The coercivity of the MnBi alloys can be controlled by tuning the particle size and the size distribution of the MnBi alloys powders through the milling process. The size-dependent coercivity of the particles was expressed by a function of Hc = 106.17−0.56·log(D), where the particle size (D) was supposed to be lager than the single-domain size. With an optimized treatment, a room-temperature coercivity of ∼1.9 T was achieved in the MnBi powders. The magnetization curves and micromagnetic analyses indicated that the magnetization process of the MnBi alloys was governed by the coherent magnetization rotation. The hardening mechanism was corresponding to the nucleation process in the anisotropic MnBi powders with refined particle size.

Vessel behaviour analysis plays an important role in maritime situational awareness. However, available technology still provides only limited approaches to vessel behaviour analysis. In this paper, we propose a visual analytics framework to interactively explore the characteristics of vessel behaviour by means of integrating visualisation with data mining and a human-computer interaction controlling model, which combines human insight with the enormous storage and processing capacities of computers to gain insight into vessel behaviour. In addition, we provide multiple views for visually analysing vessel trajectories, densities and speeds. Case studies with 15 days' AIS data collected from the middle Hankou channel to Yangluo channel in the Yangtze River demonstrate the effectiveness of our approach.

The low temperature phase (LTP) compound of MnBi is a promising, rare-earth metal free, and permanent magnetic material. However, it is difficult to obtain pure LTP MnBi, which results from a peritectic reaction between Mn and Bi atoms. In this study, the microstructural evolution and phase transformation kinetics of MnxBi100-x (x = 50, 55, 60) alloys were systematically investigated. Choosing an appropriate Mn content (Mn55Bi45) was found to promote the formation of the LTP MnBi. The results of the phase transformation kinetics indicated that the nucleation, growth and soft impingement processes are involved in the phase transformation process. In addition, in the initial stage of the LTP phase transformation, it was shown that the diffusional transformation was governed by two- and three-dimensional nucleation and growth, as well as a single mechanism of diffusion-controlled growth, which describes the entire transformation. At the final stage of the phase transformation, diffusion controlled growth, with soft impingement effects, dominates the process. The phase transformation kinetics analysis was confirmed with the X-ray diffraction and scanning electron microscopy results. This work implies that the local activation energy, Ec(α), and local Avrami exponent, n(α), can be applied to characterize the phase transformation behaviors of the LTP MnBi alloy.

In the light of low costs and automatic assessment, objective visual quality metrics enjoy many important applications such as perceptual coding. Recently multiple metrics obtain further improvement by means of machine learning. However, due to the absence of specific formulas, it's often hard to incorporate learning based metrics into video coding. In this paper, taking the state-of-the-art learning based metric VMAF for example, we propose a method of perceptual coding in an inferential manner for learning based metrics. The rate distortion optimization is adapted during coding as well. Experimental results show that compared with conventional methods, the proposed method can achieve obvious bitrate saving under HEVC coding.

Internet of Things(IoT) devices, mobile phones, and robotic systems are often denied the power of deep learning algorithms due to their limited computing power. However, to provide time-critical services such as emergency response, home assistance, surveillance, etc, these devices often need real-time analysis of their camera data. This paper strives to offer a viable approach to integrate high-performance deep learning-based computer vision algorithms with low-resource and low-power devices by leveraging the computing power of the cloud. By offloading the computation work to the cloud, no dedicated hardware is needed to enable deep neural networks on existing low computing power devices. A Raspberry Pi based robot, Cloud Chaser, is built to demonstrate the power of using cloud computing to perform real-time vision tasks. Furthermore, to reduce latency and improve real-time performance, compression algorithms are proposed and evaluated for streaming real-time video frames to the cloud., Accepted to The 11th International Conference on Machine Vision (ICMV 2018). Project site: https://zhengyiluo.github.io/projects/cloudchaser/

Being developed by Joint Video Exploration Team (JVET), Future Video Coding (FVC) aims at higher resolutions and higher compression performance than the state-of-the-art HEVC standard, undoubtedly at the cost of further computing increases. As an efficient computing platform, Graphics Processing Unit (GPU) is often used to accelerate encoding. But with the adoption of instruction set acceleration in the reference software of FVC, previous methods often become less efficient or even lead to a lower speed. In this paper, based on the comparative analysis of the time consumption between HEVC and FVC, we propose a GPU based acceleration method for the most computation-intensive step - frame interpolation of FVC, where frame caching strategy and a multi-stream mechanism is designed to make the best of GPU resources. Experimental results show that compared with the instruction set accelerated reference software of FVC, our method could achieve average 67.12% speed-up gains on the interpolation module and average 6.35% speed-up gains on overall encoding with exactly the same performance as before.

To understand rural–urban differences in stressors, this study compared the cognitive and emotional language in geolocated Twitter posts in the United States against survey-reported county-level trends from the Gallup-Sharecare Well-Being Index. Mentions of stress on Twitter can predict population-level trends in stress in both rural (R2=31.6%) and urban (R2=26.7%) communities. While mentions of poor health are limited to only rural areas with low socioeconomic status, higher emotional expression is associated with higher stress across all rural communities. Controlling for socioeconomic status, urban communities reporting higher stress are also more likely to discuss relationships on Twitter. The findings contribute to an understanding of how language use on social media acts as a barometer of the social and cultural differences between regions. The data and stress topics developed in this paper are publicly available and can be accessed at https://osf.io/af8ce/ .

With the climate change and depletion of fossil energy, distributed energy systems (DESs) have attracted widespread attention. In this study, a DES driven by solar, geothermal, aerothermal, natural gas and power grid is constructed with energy conversion devices modeled based on part load performance. A novel operation strategy for the DES is presented considering the complementary characteristics of different energy sources. Besides, a multi-objective nonlinear optimization model for the device capacity is proposed with economic, environmental and energy objectives considered simultaneously. To solve the optimization model, an integrated solution method combining Non-dominated Sorting Genetic Algorithm-II, Technique for Order Preference by Similarity to an Ideal Solution and Shannon entropy approach is developed. A case study of an indoor swimming pool in Changsha city of China is undertaken. Optimal equipment capacity and corresponding energy management strategies of the case are obtained. The final number and capacity of air source heat pump (ASHP) are determined via improving its part load ratio. Additionally, three schemes are set to investigate the effects of constant efficiency/COP of energy conversion devices and operation strategies on the capacity optimization of DESs. Results indicate that constant efficiency/COP of equipment yields an 11.7% drop in annual total cost (ATC), a 10.4% increment in annual total CO2 emission (ATE) and a 12.5% reduction in coefficient of energy performance (CEP). ATC and ATE of the optimal solution acquired under a conventional operation strategy increase by 6.8% and 3.7%, while CEP decreases by 66.9%. This work provides a guidance for the future application of DESs.

As an open-source and computationally efficient High Efficiency Video Coding (HEVC) encoder, x265 has been gaining increasing popularity in video applications. x265 provides numerous encoding parameters in view of flexibility. However, proper and efficient setting of parameters often becomes a great challenge in practice. In this paper, we deeply investigate the influence of x265 parameters based on the Slow preset and pick out important parameters in terms of efficiency and complexity. Then a Markov Chain Monte Carlo (MCMC) based algorithm is proposed for efficient parameter adaptation at the target encoding time. This paper shows that carefully selected low-complexity encoding configurations can achieve the coding efficiency comparable to that of high-complexity ones. Specifically, average 26.72% encoding time reduction can be achieved while maintaining similar Rate Distortion (RD) performance to x265 presets using the proposed algorithm.

4-aminopyridine (4-AP), a voltage-gated potassium channel blocker, was revealed to possess pro‑apoptotic properties in various types of cancer cells. The present study aimed to explore the effect of 4‑AP on a cisplatin (DDP) resistant lung cancer cell line A549/CDDP and the underlying mechanism by which it had an effect. In the present study, an MTT assay and cell cycle analysis were used to determine that 4‑AP inhibited cell growth in vitro and a tumorigenesis assay in nude mice determined that 4‑AP also inhibited cell growth in vivo. 4‑AP induced cell apoptosis of A549/CDDP cells observed by electron microscopy and Annexin V‑APC/7‑ADD analysis. In addition, 4‑AP enhanced the sensitivity of A549/CDDP cells to DDP as revealed by an MTT assay. Mechanistically, 4‑AP upregulated the phosphatase and tensin homolog (PTEN) and modulated the phosphoinositide 3‑kinase/protein kinase B signaling pathway and its downstream cell cycle factors, including cyclin D1, cyclin‑dependent kinase 4 and p21, as well as apoptosis‑associated proteins B‑cell lymphoma 2, pro‑caspase 9, pro‑caspase 3, cleaved caspase 9 and cleaved caspase 3. The effects of 4‑AP on cell growth and apoptosis were reversed by PTEN silencing. In conclusion, the results indicated that 4‑AP inhibited cell growth, induced apoptosis and sensitized A549/CDDP cells to DDP via the upregulation of PTEN. 4‑AP may be a potential therapeutic agent for patients with DDP resistance.

This paper presents a masking effects based rate control scheme for high efficiency video coding (HEVC). Rate control is regarded as a very effective tool to improve the performance of video coding under the limited bandwidth. However, the state-of-the-art rate control algorithm based on R-X model ignores the characteristics of human visual system (HVS), which leads to poor performance in subjective quality. Moreover, some structural similarity (SSIM) or saliency based perceptual rate control algorithms only consider spatial characteristics. Since spatial and temporal visual masking effects can better reflect the characteristics of HVS, in this paper masking effects based perceptual factor for coding tree unit (CTU) is proposed, which takes both texture complexity and motion information into account. Then the proposed perceptual factor is utilized to guide bit allocation in CTU-level rate control. Experimental results show that the proposed scheme can effectively improve the coding performance compared with the R-λ algorithm.

Motion compensated prediction (MCP) is an effective technology for video coding to improve compression efficiency. Fractional sample precision prediction is utilized in HEVC to further remove temporal redundancy, and finite impulse response (FIR) filters designed using decomposition of the discrete cosine transform are applied to generate samples that do not fall on the integer positions. However, the coefficients of these DCT-based interpolation filters are fixed, which may not be able to adapt to varied video content. Inspired by the remarkable success of convolutional neural network (CNN) in the single image super-resolution task, we propose to learn a convolutional neural network for fractional interpolation in HEVC inter prediction. Compared with super-resolution, there is one big difference in fractional interpolation — fractional interpolation needs to maintain samples at integer positions while super-resolution generates a whole high-resolution image. Another difference is no real ground truth is available in fractional interpolation process. To overcome these two challenges, we introduce a constraint strategy to the training phase of the original super-resolution network as well as a specially designed preprocessing step which reuses the DCTIF interpolation process. Unlike other previous work, our proposed approach simultaneously generating the fractional positions from one network and experimental results show our proposed approach achieves 0.45% BD-Rate reduction under the low-delay-P configuration on average.

This paper describes a luminance based rate control (RC) model for high dynamic range (HDR) video. A novel mathematical relationship between luminance and bit allocation of a coding tree unit (CTU) is presented. By adjusting the existing RC algorithm through the proposed model, a better balance between dark and bright areas can be achieved and −4.4% gains can be obtained in terms of average BD-Rate (tPSNR-XZY). Moreover, subjective assessment also shows that, compared with the existing RC model, the proposed method can convey a wider range of perceptible shadow and highlight more details.

Video encoding is based on the DPCM framework where temporal prediction coding introduces Rate-Distortion (RD) dependence. The RD operating point of the current unit depends on the particular choices of RD points of its reference units. Unfortunately, common Lagrangian optimization method based Rate-Distortion Optimization (RDO) for video coding is based on an independence assumption which omits the RD dependences, and thus compromises the RD performance. In this paper, we revisit the Lagrangian optimization method based RDO for dependent video coding. A theoretical RD dependence decoupling method based on independent distortion decomposition is firstly presented. After the discussion of reasonability of the theoretical decoupling method, the practical One Step Ahead Decoupling Strategy (OSADS) is proposed. After implemented on the HEVC encoder, the strategy achieves average 2.1% BD-rate saving compared with the HM encoder under the same low-delay P configuration.

Rate control plays a key role in video coding, which has a significant effect on encoder performance. With parallel video coding frameworks more and more popular, rate control suitable for parallel coding is highly desired. However, most rate control algorithms only focus on the rate distortion performance but ignoring the data correlation in parallel coding. In this paper, based on the parallel framework of the x265 encoder, we propose a parallel-friendly rate control scheme for HEVC coding, which supports both frame level and slice level parallel. Experimental results show that the proposed algorithm can achieve not only highly accurate rate control but also excellent rate distortion performance under parallel coding.

Entropy coding, which acts as one of the most important compression tools in video coding standard, had been improved step by step for HEVC. There are also several advanced methods which provide better performance than current solutions of HEVC proposed during the standardization of HEVC. However, these methods are all tested in different conditions. Comprehensive evaluation of these advanced methods under a common scenario is desired to indicate where the potential improvement of entropy coding may come from for next generation video codec. In this paper, we first introduce several advanced entropy coding methods for DCT transform coefficients, which aim to improve CABAC performance from two aspects — context modeling and probability updating. Then some modifications based on these original ones are presented. Comprehensive comparison of these methods is conducted under common test conditions. Besides, some combined methods of these two aspects are also tested. Experimental results show that all individual approaches can achieve coding gain and two new combined methods can reduce the BD-Rate up to 1.7%, 1.2% and 1.0% on common test sequences and 1.4%, 1.0% and 1.1% on 4K sequences under all intra, random access and low delay configurations, respectively.

Raptor codes are state-of-the-art forward error correction (FEC) solutions for multimedia transmission, which have been applied to unequal error protection (UEP) of multi-layered media such as scalable video coding. In this paper, we address the problem of UEP for single-layered video over packet erasure channels. By exploiting the different priorities of video packets inside a group of pictures (GOP) and making full use of the good characteristics of standardized Raptor codes at large block length, we propose an optimized UEP framework for single-layered video and develop an efficient algorithm to solve it. Simulation results show that significant gains can be obtained by our method in case of packet losses.

In this paper, we propose a temporal dependent bit allocation scheme for rate control in state-of-the-art High Efficiency Video Coding (HEVC) standard, to improve its coding performance by utilizing the temporal correlation information. In typical rate control scheme, the bit is allocated to different Coding Tree Units (CTUs) according to their complexity, which reflect their texture and motion information in some degree. However, this scheme ignores the temporal correlation between different frames, which leads to the bit allocation is not the optimal for the whole sequence. Therefore, we refine the bit allocation by taking into account temporal information, especially different coding units' distortion influence on the future frames. The result shows that under the same rate control performance, proposed scheme can effectively improve the coding quality performance, especially the Structure Similarity (SSIM) quality performance, the subjective assesment also shows very well.

H.264/AVC adopts many directional spatial prediction models in block-based manner that neighboring pixels on the left and top sides yield prediction for the pixels in a data block to be encoded. However, such models may adapt poorly to the rich textures inside blocks of video signal. In this letter, a new lossless intra coding method based on pixel-wise interleave prediction is presented to enhance the compression performance of H.264/AVC. In our scheme, pixels are coded alternately with interleave prediction, which makes full use of reconstructed pixels to predict later ones in bidirectional or multidirectional manner. Extensive experiments demonstrate that compared to the H.264/AVC standard, our scheme has higher compression ratio, especially for sequences of high resolution. In addition, the scheme can be regarded as a frame-level coding mode and can be easily integrated into the H.264/AVC framework.

Subjective evaluation conducted by JCT-VC (Joint Collaborative Team on Video Coding) members shows that H.265/HEVC achieves about 50% rate saving over H.264/AVC without sacrificing subjective quality. In this paper, we study 13 objective image and video quality assessment (IQA/VQA) metrics, including the recently proposed ones — FSIM [1], GMSD [2] and IWSSIM [3], in terms of coding gain prediction of HEVC over H.264. Experimental results on HEVC Class B and Class C test sequences show that most of the metrics underestimate the rate saving. Surprisingly, a relatively old metric — Noise Quality Measure (NQM) [4] index consists well with subjective evaluation. To verify the universality of the phenomenon, we carried out further tests on another ten video sequences with different levels of spatial and temporal complexities. The experimental results show that NQM still predicts rate saving more accurately than the other metrics.

The H.265/MPEG-HEVC is the latest video coding standard, which achieves an increase of about 50% in coding efficiency compared to its predecessor H.264/MPEG-AVC. Ever since H.265/MPEG-HEVC was designed to replace almost all existing H.264/ MPEG-AVC codecs, high-resolution video coding beyond High Definition (4K, 8K, etc.) has drawn more attention. On the other hand, it's well known that reference implementation of HEVC codec, HM, acts an important role during standardization, particularly for evaluation of rate distortion performance of different tools. However, HM is far from a practical codec because of very slow coding speed even on modern multi-core computers. Up to now except for HM few comparisons are known about both the coding performance and the coding speed of practical HEVC encoders for high resolution video sequences. To address this issue, this paper conducts a comprehensive evaluation of latest high performance H.265/MPEG-HEVC encoders, including the open source encoder-x265 and the commercial encoder-DivX265, based on default parameters and a new open 4K video database. Furthermore, latest HM and x264 are also included for performance anchors. The experimental results show DivX265 provides average bit-rate savings of 4.79% relative to HM while x265 with default preset achieves an average reduction of 3.21% in terms of BD-BR saving. In addition, different presets of x265 make a good tradeoff between coding speed and R-D performance while DivX265 is almost as fast as x265 ultrafast preset. We believe such evaluation information could provide a more comprehensive picture of state-of-the-art H.265/MPEG-HEVC encoders.

Due to the best effort feature of many existing transmission channels, video streams often suffer from inevitable transmission errors. In this paper, we propose a scheme of robust video transmission based on the state-of-the-art Raptor codes, whose applications are in full swing now. And considering Region of Interest (ROI) often draws much attention in images, the scheme adopts a nested protection framework to show partialities to ROI areas for better protection. Different from many existing Raptor codes based UEP methods, our scheme is developed based on the easy-to-use standardized Raptor codes. Experimental results show that significant robustness can be obtained for the video streams, especially for the ROI areas.

Video coding has been widely adopted to achieve pleasant video quality at constrained bitrate. In this paper, adaptive frequency coefficient suppression directed by Human Visual System (HVS) is presented for H.264 video coding. Firstly, starting from Just Noticeable Distortion (JND) models for the classic DCT domain, we deduce a JND threshold for the H.264 transform domain with decent adaptation. Then the resultant threshold is used to adaptively suppress the transform coefficients of prediction residuals. It should be noted that our scheme is fully compatible with the H.264 standard. And experimental results show that compared to normal methods, significant bitrate reduction can be obtained by our scheme at similar subjective quality.

During the period of transmission, video data usually suffer from transmission errors inevitably. Intra update is a common approach to stop error propagation. However, damaged images cannot recover until next update in case of errors, which often leads to annoying effect. In this paper, we propose an enhanced leaky prediction approach that enables the Region-Of-Interest (ROI) of images to recover gently from the immediate succeeding frame of erroneous ones in favor of better human perception. Moreover, an optimized offset compensation technique is designed to improve coding performance. Experimental results show that the proposed scheme can achieve better image quality for ROI and the fluctuation of bitrate is greatly reduced, compared to the intra update method.

Unequal Error Protection (UEP), which provides important data with more protection, has been proven to be able to produce better quality in image communication. Previous UEP schemes are mostly proposed for single-image or single-program scenarios. Yet few are developed for multiple programs. Inspired by the MPEG-2 Transport Stream (TS), in this paper we transform the conventional TS packets to be length-variable, and propose a new UEP scheme, which is suitable for multiple-program scenarios. A theoretical model for this scheme is built in this paper, and experimental results also demonstrate the effectiveness of the scheme.

Video transmission over error-prone channels often suffers from inevitable transmission errors, which necessitates proper error concealment (EC) for acceptable image quality. Furthermore, the region of interest (ROI) in images usually draws much attention, and so the EC of the ROI receives special treatment during encoding and decoding. We explore a data hiding-based scheme to effectively improve the EC of the ROI in the case of erasures of large continuous regions, which becomes impractical for conventional EC methods. At the encoder side, motion vectors of the ROI are adaptively embedded in the background based on original quantized coefficients of background macroblocks. Considering the limited embedding capacity of the background, we further propose to assign priorities to each ROI macroblock based on a predefined metric of error propagation. Our scheme is applied with the state-of-the-art H.264/AVC standard in a packet loss scenario, and better video quality can be obtained. Experimental results show that the scheme can improve the EC of the ROI significantly without much loss of coding efficiency.