Search

This paper presents a new objective quality metric that was adapted to the complex characteristics of immersive video (IV) which is prone to errors caused by processing and compression of multiple input views and virtual view synthesis. The proposed metric, IV-PSNR, contains two techniques that allow for the evaluation of quality loss for typical immersive video distortions: corresponding pixel shift and global component difference. The performed experiments compared the proposal with 31 state-of-the-art quality metrics, showing their performance in the assessment of quality in immersive video coding and processing, and in other applications, using commonly used image quality assessment databases– TID2013 and CVIQ. As presented, IV-PSNR outperforms other metrics in immersive video applications and still can be efficiently used in the evaluation of different images and videos. Moreover, basing the metric on the calculation of PSNR allowed the computational complexity to remain low. Publicly available, efficient implementation of IV-PSNR software was provided by the authors of this paper and is used by ISO/IEC MPEG for evaluation and research on the upcoming MPEG Immersive video (MIV) coding standard. [ABSTRACT FROM AUTHOR]

In this paper, we find that the conventional convolution operation becomes the bottleneck for extremely efficient binary neural networks (BNNs). To address this issue, we open up a new direction by introducing a reshaped point-wise convolution (RPC) to replace the conventional one to build BNNs. Specifically, we conduct a point-wise convolution after rearranging the spatial information into depth, with which at least $2.25\times $ computation reduction can be achieved. Such an efficient RPC allows us to explore more powerful representational capacity of BNNs under a given computation complexity budget. Moreover, we propose to use a balanced activation (BA) to adjust the distribution of the scaled activations after binarization, which enables significant performance improvement of BNNs. After integrating RPC and BA, the proposed network, dubbed as RB-Net, strikes a good trade-off between accuracy and efficiency, achieving superior performance with lower computational cost against the state-of-the-art BNN methods. Specifically, our RB-Net achieves 66.8% Top-1 accuracy with ResNet-18 backbone on ImageNet, exceeding the state-of-the-art Real-to-Binary Net (65.4%) by 1.4% while achieving more than $3\times $ reduction (52M vs. 165M) in computational complexity. [ABSTRACT FROM AUTHOR]

Multi-view clustering has received widespread attention owing to its effectiveness by integrating multi-view data appropriately, but traditional algorithms have limited applicability to large-scale real-world data due to their high computational complexity and low robustness. Focusing on the aforementioned issues, we propose an efficient and robust multi-view clustering algorithm with anchor graph regularization (ERMC-AGR). In this work, a novel anchor graph regularization (ARG) is designed to improve the quality of the learned embedded anchor graph (EAG), and the obtained EAG is decomposed by nonnegative matrix factorization (NMF) under correntropy criterion to acquire clustering results directly. Different from the traditional graph regularization that needs to construct a large-scale Laplacian matrix pertaining to the all-sample graph, our lightweight AGR, constructed from the perspective of anchors, can reduce the computational complexity significantly while improving the EAG quality. Moreover, a factor matrix of NMF is constrained to be the cluster indicator matrix to omit additional k-means after optimization. Subsequently, correntropy is utilized to improve the effectiveness and robustness of ERMC-AGR owing to its promising performance to complex noises and outliers. Extensive experiments on real-world datasets and noisy datasets show that ERMC-ARG can improve the clustering efficiency and robustness while ensuring comparable or even better effectiveness. [ABSTRACT FROM AUTHOR]

The depth video contains large smooth contents with sharp edges. Since the deep learning-based color video orientated intra prediction methods pay no attention to the characteristics of depth video, they are unsuitable for optimizing the coding efficiency of depth video. In this paper, a multiple resolution prediction method with deep up-sampling is proposed to promote the coding efficiency of depth video. To efficiently encode the depth blocks of different complexity, the depth block is selectively encoded at different resolutions, including $\times 1$ , $\times 1$ /2, and $\times 1$ /4 resolutions. If the block is encoded with a low-resolution (LR), the resolution of reconstructed LR depth block is recovered by an up-sampling network. To constrain the quality of both reconstructed high-resolution depth block and its synthesized view, a view synthesis distortion guidance mechanism is proposed for the up-sampling network. In addition, a distillation-based lightweight up-sampling network is proposed to reduce the computational complexity. Experimental results demonstrate that the proposed multiple resolution prediction method obtains an average of 10.84% BD-rate saving in comparison with 3D-HEVC. [ABSTRACT FROM AUTHOR]

In the three-dimensional video system, the depth image-based rendering is a key technique for generating synthesized views, which provides audiences with depth perception and interactivity. However, the inaccuracy of depth information leads to geometrical rendering position errors, and the compression distortion of texture and depth videos degrades the quality of the synthesized views. Although existing quality enhancement methods can eliminate the distortions in the synthesized views, their huge computational complexity hinders their applications in real-time multimedia systems. To this end, a residual distillation enhanced network (RDEN)-guided lightweight synthesized view quality enhancement (SVQE) method is proposed to minimize holes and compression distortions in the synthesized views while reducing the model complexity. First, a rethinking on the deep-learning-based SVQE methods is performed. Then, a feature distillation attention block is proposed to effectively reduce the distortions in the synthesized views and make the model fulfill more real-time tasks, which is a lightweight and flexible feature extraction block using an information distillation mechanism and a lightweight multi-scale spatial attention mechanism. Third, a residual feature fusion block is proposed to improve the enhancement performance by using the feature fusion mechanism, which efficiently improves the feature extraction capability without introducing any additional parameters. Experimental results prove that the proposed RDEN efficiently improves the SVQE performance while consuming few computational complexities compared with the state-of-the-art SVQE methods. [ABSTRACT FROM AUTHOR]

The discrete cosine transform (DCT) is a relevant tool in signal processing applications, mainly known for its good decorrelation properties. Current image and video coding standards—such as JPEG and HEVC—adopt the DCT as a fundamental building block for compression. Recent works have introduced low-complexity approximations for the DCT, which become paramount in applications demanding real-time computation and low-power consumption. The design of DCT approximations involves a trade-off between computational complexity and performance. This paper introduces a new multiparametric transform class encompassing the round-off DCT (RDCT) and the modified RDCT (MRDCT), two relevant multiplierless 8-point approximate DCTs. The associated fast algorithm is provided. Four novel orthogonal low-complexity 8-point DCT approximations are obtained by solving a multicriteria optimization problem. The optimal 8-point transforms are scaled to lengths 16 and 32 while keeping the arithmetic complexity low. The proposed methods are assessed by proximity and coding measures with respect to the exact DCT. Image and video coding experiments and hardware realization are performed. The novel transforms perform close to or outperform the current state-of-the-art DCT approximations. [ABSTRACT FROM AUTHOR]

Video compression is an important procedure for digital applications. A commonly used format for inputs to video coders is YUV420, which is a subsampled result from YUV444, that is transformed and demosaicked from a color filter array (CFA) format. The process of chroma subsampling is a crucial step for the reconstructed image quality. A state-of-the-art method utilizes the solution of a prior method to compute the starting point, and devises a search method to improve the results. However, the search method does not consider the optimality condition of the problem, and the overall computational complexity is high because of the execution of the prior method for the starting point. In the proposed work, the cost function of the optimization problem is analyzed and decomposed into 4 subchannels based on the CFA format. The optimal line of each subterm is studied in 3-dimensional space. The problem is then mathematically reduced to a 3-subchannel problem. Based on the combinations of the optimal lines, a triangle search area is formed and proved to contain the optimal solution. And because the derived triangle search area is small on average, the search for the optimal value can be considerably fast. Experimental results demonstrate that the proposed approach can produce reconstructed images with qualities that are exactly the same as those generated using the state-of-the-art method. Relative to the state-of-the-art method, the proposed approach can reduce the time complexity by 84.08%–84.91%, and reduce the number of search points by 40.24%–85.25%, for the image datasets Kodak and IMAX with different demosaicking methods. [ABSTRACT FROM AUTHOR]

Edge-preserving image filtering is an essential task in computational photography and imaging. In this paper, we propose a simple yet effective global edge-preserving filter based on soft clustering, and we propose a novel soft clustering algorithm based on a restricted Gaussian mixture model. Given specified parameters, the soft clustering process is firstly performed on the image to derive the partition matrix, from which the affinity matrix is then constructed for filtering. The filtering output is calculated as the weighted average of the pixels in the local window, so the proposed filter could suppress the intensity shift artifacts that impede most global filters. Besides, the weights in the proposed filter are derived by clustering, which properly separates dissimilar pixels, so the proposed filter could handle the halo artifacts that haunt many local filters. Moreover, our filter provides flexible control over the amount of smoothing that is deficient in the deep learning-based filters. Besides the efficacy in smoothing, the proposed filter naturally has low computational complexity. Qualitative and quantitative results suggest that the proposed filter benefits various applications, including edge-preserving smoothing, image enhancing, flash/non-flash fusion, HDR tone mapping, and dehazing. [ABSTRACT FROM AUTHOR]

The rising demand for high quality display has ensued active research in high dynamic range (HDR) imaging, which has the potential to replace the standard dynamic range imaging. This is due to HDR’s features like accurate reproducibility of a scene with its entire spectrum of visible lighting and color depth. But this capability comes with expensive capture, display, storage and distribution resource requirements. Also, display of HDR images/video content on an ordinary display device with limited dynamic range requires some form of adaptation. Many adaptation algorithms, widely known as tone mapping (TM) operators, have been studied and proposed in the last few decades. In this article, we present a comprehensive survey of 60 TM algorithms that have been implemented on hardware for acceleration and real-time performance. In this state-of-the-art survey, we will discuss those TM algorithms which have been implemented on GPU, FPGA, and ASIC in terms of their hardware specifications and performance. Output image quality is an important metric for TM algorithms. From our literature survey we found that, various objective quality metrics have been used to demonstrate the quality of those algorithms hardware implementation. We have compiled those metrics used in this survey, and analyzed the relationship between hardware cost, image quality and computational efficiency. Currently, machine learning-based (ML) algorithms have become an important tool to solve many image processing tasks, and this article concludes with a discussion on the future research directions to realize ML-based TM operators on hardware. [ABSTRACT FROM AUTHOR]

Light Field (LF) video technology is a step towards offering a better immersive experience through on-demand refocusing and perspective viewing. However, the significant increase in captured data makes the need for efficient compression of paramount importance. In this paper, we proposed two prediction structures and coding orders that efficiently compress LF video content using the existing HEVC standard. This is achieved by utilizing horizontal and vertical correlation among the views for better inter-view prediction. To assess the performance of the schemes, ten publicly available and widely used LF video sequences were used. Our first method is highly suitable for applications demanding high compression efficiency. It outperforms the best pseudo-sequence-based compression technique to date by up to 17% in bitrate reduction while being scalable in the number of views. The second method is proficient for low computational and random-access complexity to any arbitrary view in the light field video. It offers 10% faster decoding and 20% lower random-access complexity compared to the best existing technique. [ABSTRACT FROM AUTHOR]

In video-based dynamic point cloud compression (V-PCC), 3D point clouds are projected into patches, and then the patches are padded into 2D images suitable for the video compression framework. However, the patch projection-based method produces a large number of empty pixels; the far and near components are projected to generate different 2D images (video frames), respectively. As a result, the generated video is with high resolutions and double frame rates, so the V-PCC has huge computational complexity. This paper proposes an occupancy map guided fast V-PCC method. Firstly, the relationship between the prediction coding and block complexity is studied based on a local linear image gradient model. Secondly, according to the V-PCC strategies of patch projection and block generation, we investigate the differences of rate-distortion characteristics between different types of blocks, and the temporal correlations between the far and near layers. Finally, by taking advantage of the fact that occupancy maps can explicitly indicate the block types, we propose an occupancy map guided fast coding method, in which coding is performed on the different types of blocks. Experiments have tested typical dynamic point clouds, and shown that the proposed method achieves an average 43.66% time-saving at the cost of only 0.27% and 0.16% Bjontegaard Delta (BD) rate increment under the geometry Point-to-Point (D1) error and attribute Luma Peak-Signal-Noise-Ratio (PSNR), respectively. [ABSTRACT FROM AUTHOR]

Traditional clustering algorithms are widely used for building bag-of-words (BOW) models to aggregate spatio-temporal feature points extracted from a video for human activity recognition problems. Their performances are restricted by the computational complexity which limits the number of feature points being used. In contrast, deep clustering yields good clustering performance without the limit of the number of feature points. Therefore, this work proposes a dual stacked autoencoders features embedded clustering (DSAFEC) and a BOW construction method based on the DSAFEC (B-DSAFEC) to reduce the computational complexity and to remove the selection restriction. The DSAFEC first transforms feature points extracted from a video to a learned feature space and then probabilities of cluster assignment of feature points are predicted to build BOWs for human activity recognition. A soft clustering is used by assigning each feature point to multiple clusters yielding the largest probabilities instead of only one in hard clustering. Experimental results on three benchmark human activity datasets show that the B-DSAFEC yields better performance compared to five reference methods which are developed based on either traditional clustering methods or deep clustering methods. [ABSTRACT FROM AUTHOR]

Flexible optimization on video coding computational complexity is imperative to adapt to diverse terminals in video streaming, especially with high definition videos and increasingly comprehensive encoders. Until now, few efforts have been contributed to the encoder of multiview videos, which complicates its encoding structure with duplex frame display. To address this issue, we propose a flexible complexity optimization framework in this paper. In the first place, the proposed algorithm flexibly reduces the encoder complexity at different external-defined complexity constraints; furthermore, it is also committed to compression efficiency optimization at different complexity levels. The framework is achieved by a hybrid approach: complexity allocation, which allocates the external-defined complexity constraint to all views, hierarchical layers and frames based on linear programming; and complexity regulation, which dynamically adjusts local candidate partitions to fulfil the targeted local complexity constraint, with a probability-driven Alternate Partition Cost (APC) minimization. The overall algorithm is implemented on the popular multiview encoder, Multiview High Efficiency Video Coding (MV-HEVC), with promising Rate-Distortion (RD) performances at different complexity constraints, which are also superior to two recent complexity optimization algorithms. [ABSTRACT FROM AUTHOR]

The classical chroma subsampling involves downsampling chrominance components (U and V) while maintaining the luminance component (Y). Recently, a study has attempted to change Y for the chroma-subsampling process, and improved results in the reconstructed image quality are obtained. However, computational complexity remains an issue because the study examines all the candidates in the determined range for minimal pixel distortion for the modified Y; the complexity can be high if the range is large. In this study, we reduced the candidate number to seven at most, which is mathematically optimized. Boundary points for in-the-range red, green, and blue (RGB) values are first decided, followed by the determination of the intervals that concatenate the entire curve (at most seven intervals). Each interval belongs to one of the seven sub-cases of different linear combinations of individual curves. The optimal solution (modified Y) for each sub-case is derived, which is in highly efficient form. Compared with the existing method, the proposed fast method ensures that the image quality is preserved, while the number of search candidates is reduced by 61.30%–69.19% on average, and the computational time of the process is reduced by 31.39%–58.97%, thereby demonstrating an efficient performance. [ABSTRACT FROM AUTHOR]

Recent learned image coding has emerged with superior efficiency to conventional methods. It, however, is criticized for its complexity-exhaustive deep neural network (DNN) architectures, especially on resource-constrained mobile platforms. Thus we devise a two-stage approach: first, a range pre-processing is applied to constrain the dynamic range of feature map activation by leveraging its sparsity nature with densely clustered distribution, then a layer-wise range-adaptive quantization for convolutional parameter (e.g., weight, bias), and simple yet efficient linear scaling and range-dependent normalization for activation are executed, leading to a fully fixed-point inference architecture. All arithmetic operations and associated data tensors are processed using low-bit-width fixed-point numbers, yielding significant reductions of the computational complexity, memory space, and the elimination of platform-dependent inconsistency induced by floating-point operations. We first exemplify such fixed-point inference in a DNN-based image decoder, showing the comparable coding efficiency with its native floating-point model, against the same anchor using the High-Efficiency Video Coding (HEVC)-based intra image coder. We also extend proposed approach to the super-resolution network for learned resolution scaling-based video streaming, and VGG network-based classification tasks, both of which present negligible performance loss. These evidence the generalization of our approach for efficient DNN processing of various tasks. All materials are made publicly accessible at http://njuvision.github.io/fixed-point/. [ABSTRACT FROM AUTHOR]

This paper presents an overview of the decoder-side motion vector refinement (DMVR) algorithm in the Versatile Video Coding (VVC) standard. The proposed DMVR algorithm aims to increase the prediction accuracy of the blocks coded in merge mode using the bilateral matching-based refinement method. Compared with previous decoder-side motion vector derivation approaches, the proposed method significantly increases the coding efficiency without signaling additional side information. Furthermore, the hardware implementation considerations of the DMVR design are particularly focused in this study. This paper details and analyzes the novel features of DMVR contributing to the increase in coding efficiency and the reduction in computational complexity and implementation difficulty. Experimental results based on the VVC test model version 8.0 demonstrate that average Bjøntegaard Delta rate savings of 0.80 % and 2.81 % are achieved for the “tool-off” and “tool-on” test configurations, respectively. Moreover, 4 % additional decoding time and negligible additional external memory bandwidth requirements of DMVR based on the common test conditions for VVC are reported. [ABSTRACT FROM AUTHOR]

Motion-compensated prediction adopts fractional-pixel interpolation to obtain the best motion vector. Traditional fixed interpolation filters cannot handle various content and structures well, and existing convolutional neural network based methods cannot fully exploit the distortion characteristics for fractional interpolation. Therefore, this paper proposes a distortion-aware multi-task learning framework (DA-MLF) to perform fractional interpolation. First, a multi-task training framework is proposed to provide the distortion characteristics as complementary information for improving the performance of subsequent interpolation. Then, a uniform interpolation sub-network is proposed to accomplish fractional interpolation, which utilizes the feature fusion module to fuse abundant local features, and the distortion awareness module to capture the multi-scale information of compression artifacts. Furthermore, DA-MLF is integrated into High Efficiency Video Coding (HEVC) test model, and multiple experiments are performed to evaluate the effectiveness of our method. On HEVC testing sequences, DA-MLF achieves 5.0%, 4.0% and 1.7% BD-rate reduction on average compared to the HEVC baseline, under low-delay P, low-delay B and random-access configurations, respectively. The experimental results validate that our framework not only achieves the best interpolation performance but also has the lowest computational complexity compared with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Currently, video streaming services put more emphasis on user feeling or satisfaction than before. How to design suitable model and algorithm to effectively and efficiently evaluate user quality of experience (QoE) has become a significant technical challenge. To get over this dilemma, this paper proposes the broad forest, a non-neural network style broad model for streaming video QoE evaluation. The design target of broad forest is to take advantage of representation potential of forest and low complexity characteristic of broad learning system. Specifically, we first give the construction of broad forest. Then, the associated incremental learning algorithm for efficiently supporting the added structure and inputs is designed. Finally, we apply the broad forest to streaming video QoE evaluation. Experimental results show that the broad forest can not only guarantee accuracy, but also decrease training time. When subjective features are considered, it can further promote performance of QoE evaluation. [ABSTRACT FROM AUTHOR]

In this paper, we present a simple, efficient, and highly modularized network architecture for single-image snow-removal. To address the challenging snow-removal problem in terms of network interpretability and computational complexity, we employ a pyramidal hierarchical design with lateral connections across different resolutions. This design enables us to incorporate high-level semantic features with other feature maps at different scales to enrich location information and reduce computational time. In addition, a refinement stage based on recently introduced generative adversarial networks (GANs) is proposed to further improve the visual quality of the resulting snow-removed images and make a refined image and a clean image indistinguishable by a computer vision algorithm to avoid the potential perturbations of machine interpretation. Finally, atrous spatial pyramid pooling (ASPP) is adopted to probe features at multiple scales and further boost the performance. The proposed DesnowGAN (DS-GAN) performs significantly better than state-of-the-art methods quantitatively and qualitatively on the Snow100K dataset. [ABSTRACT FROM AUTHOR]

Video encoding draws high research interest, due to the enormous demand for video traffic and real-time encoding for transmission. In video encoding standards such as HEVC (High-Efficiency Video Coding), the final step of the encoding stage is the CABAC (Context-Adaptive Binary Arithmetic Coding). The coding efficiency of the CABAC comes at the cost of increased computational complexity, especially for parallelization purposes, being the BAE (Binary Arithmetic Encoder) the critical part of CABAC. Thus, an important goal is to balance the real-time throughput requirements and the power/energy consumption in the design of BAE dedicated hardware. This work introduces a novel configurable high-throughput BAE design, named ET-BAE, utilizing a combination of a new modified Multiple-Bypass Bins Scheme (MBBS) and a power-saving approach into a single ASIC design with two-mode configuration. Synthesis and power-analysis results show that the configurable BAE design, the first of its kind with this feature, is more energy-efficient and less area consuming than utilizing non-configurable versions. The ET-BAE is able to accomplish the same real-time requirements of its competitors. [ABSTRACT FROM AUTHOR]

JPEG is the most widely used lossy image compression standard. When using JPEG with high compression ratios, visual artifacts cannot be avoided. These artifacts not only degrade the user experience but also negatively affect many low-level image processing tasks. Recently, convolutional neural network (CNN)-based compression artifact removal approaches have achieved significant success, however, at the cost of high computational complexity due to an enormous number of parameters. To address this issue, we propose a dual-stream recursive residual network (STRRN) which consists of structure and texture streams for separately reducing the specific artifacts related to high-frequency or low-frequency image components. The outputs of these streams are combined and fed into an aggregation network to further enhance the restored images. By using parameter sharing, the proposed network reduces the total number of training parameters significantly. Moreover, experiments conducted on five commonly used datasets confirm that the proposed STRRN can efficiently reduce the compression artifacts, while using up to 4.6 times less training parameters and 5 times less running time compared to the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

This study proposes a video retargeting method using deep neural network-based object detection. First, the meaningful regions of the input video denoted by bounding boxes of the object detection are extracted. In this case, the area is defined considering the size and number of bounding boxes for objects detected. The bounding boxes of each frame image are considered as regions of interest (RoIs). Second, the Siamese object tracking network is used to address high computational complexity of the object detection network. By dividing the video into scenes, object detection is performed for the first frame image of each scene to obtain the first bounding box. Object tracking is performed for the next sequential frame image until a scene change is detected. Third, the image is resized in the horizontal direction to alter the aspect ratio of the image and obtain the 1D RoIs of the image by projecting bounding boxes in the vertical direction. Then, the proposed method computes the grid map from the 1D RoIs to calculate new coordinates of each column data of the image. Finally, the retargeted video is obtained by rearranging all retargeted frame images. Comparative experiments conducted with various benchmark methods show an average bidirectional similarity score of 1.92, which is higher than other conventional methods. The proposed method was stable and satisfied viewers without causing cognitive discomfort as conventional methods. [ABSTRACT FROM AUTHOR]

A wide range of face appearance variations can be modeled by using set-based recognition approaches effectively, but computational complexity of current methods is highly dependent on the set and class sizes. This paper introduces new video-based classification methods designed for reducing the required disk space of data samples and speed up the testing process in large-scale face recognition systems. In the proposed method, image sets collected from videos are approximated with kernelized convex hulls and it was shown that it is sufficient to use only the samples that participate in shaping the image set boundaries in this setting. The kernelized support vector data description (SVDD) is used to extract those important samples that form the image set boundaries. Moreover, we show that these kernelized hypersphere models can also be used to approximate image sets for classification purposes. Then, we propose a binary hierarchical decision tree approach to improve the speed of the classification system even more. At last, we introduce a new video database that includes 285 people with 8 videos of each person, since the most popular video data sets used for set-based recognition methods include either a few people, or small number of videos per person. The experimental results on varying sized databases show that the proposed methods greatly improve the testing times of the classification system (we obtained speed-ups to a factor of 20) without a significant drop in accuracies. [ABSTRACT FROM AUTHOR]

Semi-global matching (SGM) is favored for high accuracy real-time stereo matching design as it achieves a good trade-off between disparity image quality and computational complexity. Nevertheless, most of previous SGM designs so far are restricted to the real-time processing of small image resolution and disparity range, or achieve high throughput by simplifying the original algorithm at the penalty of significant disparity image quality degradation. We analyze that the major challenge to efficient SGM design is its memory architecture, including both on-chip memory cost and off-chip memory bandwidth. We address the memory architecture challenge by algorithm and architecture co-design. Based on two observed features of SGM algorithm, i.e. incompleteness and inaccuracy, this paper proposes several efficient techniques to reduce on-chip memory cost and compress off-chip memory bandwidth respectively. Moreover, we also design high throughput and pipelined architecture to implement the proposed techniques. The disparity image quality and hardware efficiency of the proposed SGM design are evaluated on both KITTI2015 and Middlebury V3 stereo datasets. Evaluation results demonstrate that, the throughput of the proposed circuit designs can easily achieve 1080P@30fps at the disparity range of 128, and can reduce the on-chip memory cost and off-chip memory bandwidth by up to $4\times $ and $2\times $ respectively while achieving better or the same disparity image quality, compared with the best reference design techniques. [ABSTRACT FROM AUTHOR]

Various kinds of features prove to be effective for content-based image retrieval. However, due to the diversity of image contents, a descriptor may achieve impressive performance on specific images while becoming invalid on others. Although some efforts have been made to combine features as complementary counterparts, proper weighting scheme is still a challenge for fast and accurate retrieval. In this paper, we propose an effective fusion method, termed as Topo-correlation (Topo), where the importance of each feature is measured by cross-view correlations on local affinity graphs. Specifically, the weights of similarities are node-sensitive as well as modality-sensitive, thus boosting the results of good cues while depressing adverse factors for individual images. By estimating the consensus of similarity scores with regard to a query-driven criterion, the weighted graphs are generated efficiently with low computational complexity. Extensive experimental results on four benchmarks demonstrate the superiority of the proposed approach over the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Reversible image watermarking is a technique that allows the cover image to remain unmodified after watermark extraction. Prediction error expansion-based schemes are currently the most efficient and widely used class of reversible image watermarking techniques. In this paper, first, we prove that the bounded capacity distortion minimization problem for prediction error expansion-based reversible watermarking schemes is NP-hard, and the corresponding decision version of the problem is NP-complete. Then, we prove that the dual problem of bounded distortion capacity maximization problem for prediction error expansion-based reversible watermarking schemes is NP-hard, and the corresponding decision problem is NP-complete. Furthermore, taking advantage of the integer linear programming formulations of the optimization problems, we find the optimal performance metric values for a given image, using concepts from the optimal linear prediction theory. Our technique allows the calculation of these performance metric limit without assuming any particular prediction scheme. The experimental results for several common benchmark images are consistent with the calculated performance limits validate our approach. [ABSTRACT FROM AUTHOR]

Deep learning provides a great potential for in-loop filtering to improve both coding efficiency and subjective quality in video coding. State-of-the-art work focuses on network structure design and employs a single powerful network to solve all problems. In contrast, this paper proposes a deep learning based systematic approach that includes an effective Convolutional Neural Network (CNN) structure, a hierarchical training strategy, and a video codec oriented switchable mechanism. First, we propose a novel CNN structure, i.e., Squeeze-and-Excitation Filtering CNN (SEFCNN), as an optional in-loop filter. To capture the non-linear interaction between channels, the SEFCNN is comprised of two subnets, i.e., Feature EXtracting (FEX) subnet and Feature ENhancing (FEN) subnet. Then, we develop a hierarchical model training strategy to adapt the two subnets to different coding scenarios. For high-rate videos with small artifacts, we train a single global model using the FEX for all types of frames, whereas for low-rate videos with large artifacts, different models are trained using both FEX and FEN for different types of frames. Finally, we propose an adaptive enhancing mechanism which is switchable between the CNN-based and the conventional methods. We selectively apply the CNN model to some frames or some regions in a frame. Experimental results show that the proposed scheme outperforms state-of-the-art work in coding efficiency, while the computational complexity is acceptable after GPU acceleration. [ABSTRACT FROM AUTHOR]

Screen content coding (SCC) is an extension of high efficiency video coding (HEVC), and it is developed to improve the coding efficiency of screen content videos by adopting two new coding modes: Intra Block Copy (IBC) and Palette (PLT). However, the flexible quadtree-based coding tree unit (CTU) partitioning structure and various mode candidates make the fast algorithms of the SCC extremely challenging. To efficiently reduce the computational complexity of SCC, we propose a deep learning-based fast prediction network DeepSCC that contains two parts: DeepSCC-I and DeepSCC-II. Before feeding to DeepSCC, incoming coding units (CUs) are divided into two categories: dynamic CTUs and stationary CTUs. For dynamic CTUs having different content as their collocated CTUs, DeepSCC-I takes raw sample values as the input to make fast predictions. For stationary CTUs having the same content as their collocated CTUs, DeepSCC-II additionally utilizes the optimal mode maps of the stationary CTU to further reduce the computational complexity. Compared with the HEVC-SCC reference software SCM-8.3, the proposed DeepSCC reduces the encoding time by 48.81% on average with a negligible Bjøntegaard delta bitrate increase of 1.18% under all-intra configuration. [ABSTRACT FROM AUTHOR]

Blind image deblurring is a challenging problem in computer vision, which aims to restore both the blur kernel and the latent sharp image from only a blurry observation. Inspired by the prevalent self-example prior in image super-resolution, in this paper, we observe that a coarse enough image down-sampled from a blurry observation is approximately a low-resolution version of the latent sharp image. We prove this phenomenon theoretically and define the coarse enough image as a latent structure prior of the unknown sharp image. Starting from this prior, we propose to restore sharp images from the coarsest scale to the finest scale on a blurry image pyramid and progressively update the prior image using the newly restored sharp image. These coarse-to-fine priors are referred to as multi-scale latent structures (MSLSs). Leveraging the MSLS prior, our algorithm comprises two phases: 1) we first preliminarily restore sharp images in the coarse scales and 2) we then apply a refinement process in the finest scale to obtain the final deblurred image. In each scale, to achieve lower computational complexity, we alternately perform a sharp image reconstruction with fast local self-example matching, an accelerated kernel estimation with error compensation, and a fast non-blind image deblurring, instead of computing any computationally expensive non-convex priors. We further extend the proposed algorithm to solve more challenging non-uniform blind image deblurring problem. The extensive experiments demonstrate that our algorithm achieves the competitive results against the state-of-the-art methods with much faster running speed. [ABSTRACT FROM AUTHOR]

This paper proposes a novel video error concealment method for the reconstruction of all unknown regions in frames. The proposed method first performs temporal error concealment (TEC) sequentially in forward direction with the propagation of source information for the unknown regions of frames in group-of-frames (GOF). Then, it performs the TEC sequentially in backward direction for the remaining unknown regions in the GOF. For the reconstruction of unknown regions in each target frame in forward and backward TEC stages, we propose an adaptive homography-based registration of the reference frame. The proposed method selects adaptively global or local homography-based registration for the reconstruction of each unknown region. After the backward direction, the TEC is performed in each frame and the spatial error concealment is applied to reconstruct completely the remaining unknown regions with relatively small sizes. The experimental results show that the proposed method gives significantly improved objective and subjective video quality compared with the traditional reconstruction methods. The proposed method also gives comparably good reconstruction results for large size unknown regions and is feasible for practical real-time applications. [ABSTRACT FROM AUTHOR]

Extensive efforts have been made to design hardware-based integer motion estimation (IME) that is much faster than software-based IME but suffers from the degradation in the coding efficiency. This is because the strategy for previous efforts was a simple algorithmic modification of the fast IME to facilitate the given hardware design at the expense of coding efficiency. This paper proposes a novel hardware design of the IME that not only offers real-time processing capability but also provides a flexible tradeoff between computational complexity and coding efficiency. First, a prediction unit (PU) loop unrolling scheme is proposed to solve the pipeline stall problem owing to the nature of fast IME algorithms such as the test zone search (TZS). It reduces idle cycles by 89.24%. Next, to further reduce the computational complexity of the TZS algorithm, a computational redundancy among PUs within a coding unit is reduced through a search step synchronization and search point sharing scheme. Thus, the computational complexity is reduced by 72.25%. The proposed schemes eliminate the inefficiency of hardware design; thus, they do not suffer from serious degradation in the coding efficiency. Consequently, the proposed hardware-based IME processes $7680\times4320$ videos at 30 frames per second while increasing the Bjøntegaard delta bitrate by only 0.90% on average. The hardware design is synthesized using a 65 nm general purpose CMOS technology, and its gate count is 268.5K at an operating clock frequency of 500 MHz. [ABSTRACT FROM AUTHOR]

Recovering low-rank and sparse components from missing observations is an essential problem in various fields. In this paper, we have proposed a method to address the missing low-rank and sparse decomposition problem. We have used the smoothed nuclear norm and the $L_{1}$ norm to impose the low-rankness and sparsity constraints on the components, respectively. Furthermore, we have suggested a linear modeling for the corrupted observations. The problem has been solved with the aid of alternating minimization. Moreover, some simplifications have been applied to the relations to reduce the computational complexity, which makes the algorithm suitable for large-scale problems. To evaluate the proposed method, different simulation scenarios have been devised. The superiority of the suggested scheme over its counterparts has been confirmed on both the recovery accuracy and the convergence speed in various applications. [ABSTRACT FROM AUTHOR]

Quadtree with nested multi-type tree (QTMT) partition structure is an efficient improvement in versatile video coding (VVC) over the quadtree (QT) structure in the advanced high-efficiency video coding (HEVC) standard. With the exception of the recursive QT partition structure, recursive multi-type tree partition is applied to each leaf node, which generates more flexible block sizes. Besides, intra prediction modes are extended from 35 to 67 so as to satisfy various texture patterns. These newly developed techniques achieve high coding efficiency but also result in very high computational complexity. To tackle this problem, we propose a fast intra-coding algorithm consisting of low-complexity coding tree units (CTU) structure decision and fast intra mode decision in this paper. The contributions of the proposed algorithm lie in the following aspects: 1) the new block size and coding mode distribution features are first explored for a reasonable fast coding scheme; 2) a novel fast QTMT partition decision framework is developed, which can determine the partition decision on both QT and multi-type tree with a novel cascade decision structure; and 3) fast intra mode decision with gradient descent search is introduced, while the best initial search point and search step are also investigated in this paper. The simulation results show that the complexity reduction of the proposed algorithm is up to 70% compared to VVC reference software (VTM), and averagely 63% encoding time saving is achieved with 1.93% BDBR increasing. Such results demonstrate that our method yields a superior performance in terms of computational complexity and compression quality compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The screen content coding (SCC) extension of high efficiency video coding (HEVC) improves coding gain for screen content videos by introducing two new coding modes, namely, intra block copy (IBC) and palette (PLT) modes. However, the coding gain is achieved at the increased cost of computational complexity. In this paper, we propose a decision tree-based framework for fast intra mode decision by investigating various features in the training sets. To avoid the exhaustive mode searching process, a sequential arrangement of decision trees is proposed to check each mode separately by inserting a classifier before checking a mode. As compared with the previous approaches where both IBC and PLT modes are checked for screen content blocks (SCBs), the proposed coding framework is more flexible which facilitates either the IBC or PLT mode to be checked for SCBs such that computational complexity is further reduced. To enhance the accuracy of decision trees, dynamic features are introduced, which reveal the unique intermediate coding information of a coding unit (CU). Then, if all the modes are decided to be skipped for a CU at the last depth level, at least one possible mode is assigned by a CU-type decision tree. Furthermore, a decision tree constraint technique is developed to reduce the rate–distortion performance loss. Compared with the HEVC-SCC reference software SCM-8.3, the proposed algorithm reduces computational complexity by 47.62% on average with a negligible Bjøntegaard delta bitrate (BDBR) increase of 1.42% under all-intra (AI) configurations, which outperforms all the state-of-the-art algorithms in the literature. [ABSTRACT FROM AUTHOR]

The edge-preserving bilateral filter (BF) is a widely used smoothing tool in many applications. However, its brute-force implementation depends on the size of the box window. The shortcoming causes BF time-consuming for the image processing task with a large window. To make the computational complexity irrelevant to the window size, sparse approximations of the filtering kernels are calculated on a learned cosine dictionary by two steps. First, all possible frequencies are learned (estimated) from the filtering kernel to compose a cosine dictionary. Then, the sparse approximation is conducted on the learned dictionary to seek the optimal cosine approximation for both the range and spatial kernels. By making use of the one-dimensional cosine approximation for the range kernel, the BF is transformed into spatial convolutions. Subsequently, by employing the two-dimensional cosine approximation for the spatial kernel, spatial convolutions are decomposed into box filters of which the computational complexity is $O(1)$. To the best of our knowledge, our approach is the first method that adaptively constructs a cosine dictionary according to the input kernel. This merit guarantees the best filtering accuracy and efficiency. These advantages are corroborated by several carefully designed experiments. [ABSTRACT FROM AUTHOR]

Hashing has been recognized as an efficient representation learning method to effectively handle big data due to its low computational complexity and memory cost. Most of the existing hashing methods focus on learning the low-dimensional vectorized binary features based on the high-dimensional raw vectorized features. However, the studies on how to obtain preferable binary codes from the original 2D image features for retrieval is very limited. This paper proposes a bilinear supervised discrete hashing (BSDH) method based on 2D image features which utilizes bilinear projections to binarize the image matrix features such that the intrinsic characteristics in the 2D image space are preserved in the learned binary codes. Meanwhile, the bilinear projection approximation and vectorization binary codes regression are seamlessly integrated together to formulate the final robust learning framework. Furthermore, a discrete optimization strategy is developed to alternatively update each variable for obtaining the high-quality binary codes. In addition, two 2D image features, traditional SURF-based FVLAD feature, and CNN-based AlexConv5 feature are designed for further improving the performance of the proposed BSDH method. The results of extensive experiments conducted on four benchmark datasets show that the proposed BSDH method almost outperforms all competing hashing methods with different input features by different evaluation protocols. [ABSTRACT FROM AUTHOR]

High-efficiency video coding (HEVC) poses a considerable challenge to hardware implementation due to its complexity. Mobile devices are powered by batteries that are limited in capacity. Therefore, reducing the power consumption arising from the implementation of sophisticated coding tools in HEVC is an especially important issue for mobile devices. In particular, motion estimation (ME) is the major contributor to the power consumption of the encoder and the calculation of the sum of absolute difference (SAD) for ME consumes more than 50% of the total ME power. In this paper, a low-power motion estimation VLSI architecture is proposed based on a novel method of calculating the SAD. By reusing the calculation, the computation complexity and, hence, the power consumption are reduced. A low-power systolic processing elements array and a novel memory hierarchy are developed, which enable real-time processing of 8K resolution video with only half of the power consumption when compared with the state-of-the-art design. [ABSTRACT FROM AUTHOR]

Although contrast is a major issue in overall quality assessment of an image, existing contrast evaluators with a reasonable performance are currently scarce. Here, we propose a learning-based blind/no-reference (NR) image quality assessment (IQA) model, a dubbed histogram eigen-feature-based contrast score (HEFCS) for evaluating image contrast. This research seeks for the inter-relationship between contrast degradation and relevant image histogram features. We introduce “eigen-histograms,” which are the eigenvectors of the set of image patches’ histograms. We found that the randomness of image eigen-histograms and the amplitude of corresponding eigenvalues can reliably reflect the changes in image contrast. Employing these characteristics leads to contrast-aware HEF vectors, which are used to compute the contrast score through a prediction model trained using support vector regression. Extensive analysis and cross validation are performed with five contrast relevant image databases, and the HEFCS performance results are compared with a collection of full-reference (FR), reduced-reference (RR), and NR measures. Despite its simplicity and low computational complexity, the HEFCS performs better than all competing NR-IQA models and also stands among the three best performers of FR and RR models. [ABSTRACT FROM AUTHOR]

Distributed compressive video sensing (DCVS) has great potential for signal acquisition and processing in source-limited communication, e.g., wireless video sensors networks, because it shifts complicated motion estimation and motion compensation from the encoder to the decoder. Known as a state-of-the-art technique in DCVS, multihypothesis (MH) prediction is widely used because of its acceptable performance and low computational complexity. However, this technique is restricted by inaccurate regularizations, which can cause susceptibility to inaccurate hypotheses. In this paper, we present an iterative reweighted Tikhonov-regularized scheme for MH prediction reconstruction. Specifically, to enhance robustness, this scheme proposes a reweighted Tikhonov regularization that synthetically considers three factors that affect the MH prediction performance—accuracy of the hypothesis set, number of hypotheses, and accuracy of regularizations—by utilizing the influence of each hypothesis. Furthermore, to avoid over-iteration in iterative MH prediction reconstruction, we propose a Bhattacharyya coefficient-based stopping criterion for use in the recovery of non-key frames, in which we exploit the similarity to an adjacent key frame rather than a previous iteration result. The simulation results show that the proposed scheme outperforms the state-of-the-art MH methods in terms of robustness to inaccurate hypotheses when there are a limited number of hypotheses. [ABSTRACT FROM AUTHOR]

The growing demand for high-performance ultra-high-definition video coding leads to H.265/high-efficiency video coding (HEVC), where the increased computational complexity and data/timing dependence hinder its coding throughput. To address these challenges, this paper presents four algorithm adaptations and a fully parallel hardware architecture for an H.265/HEVC intra encoder. To the best of our knowledge, this is the first fully parallel H.265/HEVC intra encoder. This design supports 35 prediction modes and all coding tree unit partitions. All PUs are independently processed in four prediction engines for high parallelism. An appropriate set of intra prediction modes, RDO candidates, and CABAC rate estimate instances is assigned to each prediction engine, where internal computational tasks are pipelined and scheduled to maximize the processing throughput. Compared with the HM-15.0 software, the proposed algorithm adaptations lead to a reduction of 27% in computational workload, while the average BD-rate and BD-PSNR are 4.39% and −0.21 dB, respectively. This BD-rate is lower than the existing designs with the same video resolution. FPGA implementation of the proposed design shows that it operates at 120 MHz and supports 45 fps of 1080P video sequences using 201-K logic elements and 120-KB on-chip SRAM. ASIC implementation of the proposed design in TSMC 90-nm technology shows that its clock frequency reaches 320 MHz with a hardware gate count of 2288 K, and that it supports real-time encoding of 30 fps of 4-K video sequences. Compared with the state-of-the-art designs, our proposed design demonstrates advantages in computational complexity, bit rate, video quality, throughput, reliability, and flexibility. [ABSTRACT FROM AUTHOR]

Video summarization is an economical way of representing video contents and is useful for effective and quick browsing of the relevant activity present in the video. Most current video summarization approaches do not provide the correct insight of the events occurring in the video, such as moving the objects in a nonchronological order and tampering with the background and size of the objects. In this paper, we present an approach to create a video summarization that is a precise representation of the video content. First, our approach finds out the salient activities that are taking place in the video using an optimization framework for static and dynamic scenes. Second, the frames with the salient activities are stitched using alpha matting to form a single frame. Third, the summarized frame over multiple video shots obtained by our approach gives superior retrieval performance with image queries while reducing retrieval latency and memory requirement. This paper proposes a single frame-based indexing of the video database instead of multi-frame indexing as it is an interesting and potentially promising approach. Finally, several experiments are carried out to evaluate the proposed approach. The experiments demonstrate that the video summarization produced by our approach gives a realistic view of the events in the video, while the reduction in computational complexity and memory requirement is high. [ABSTRACT FROM AUTHOR]

The coarse-to-fine shape searching (CFSS) framework is a recently developed algorithm that achieves relatively high accuracy in face alignment by alleviating the poor initialization problem facing traditional cascaded regression approaches. However, its high computational complexity and memory access demands make it difficult for CFSS to satisfy the requirements of real-time processing. To address this issue, a fast shape searching face alignment (F-SSFA) accelerator is presented based on the optimization of the CFSS algorithm and an efficient hardware implementation. First, the learning-based low-dimensional speeded-up robust features method, based on the correlations between the SURF features and the regression targets, is introduced to distill the feature set down to the only most distinct features to reduce the computing load. Second, the partial keypoints Euclidean distance and shape affine transformation are introduced to replace feature extraction and support vector machine classification, thereby accelerating the shape searching process. Compared with CFSS, F-SSFA achieves a $5.8\times $ speedup while achieving similar accuracy. Moreover, a VLSI architecture is proposed to realize the fixed-point F-SSFA algorithm. Multiple descriptors located in adjacent regions are simultaneously generated in a single access to the corresponding image data. Therefore, repeated memory access operations are avoided. The optimal parameter configuration for hardware implementation is also exploited based on a tradeoff between accuracy and hardware performance. Simulated with TSMC 65-nm 1P8M technology within a 3.6 mm2 area, a post-layout simulation shows that 700 fps can be achieved while consuming 300 mW at 200 MHz. [ABSTRACT FROM AUTHOR]

Preserving projection-based methods are good for finding the manifold structure embedded in data. As they use the Euclidean distance as a metric, which is sensitive to noise and outliers in data, nuclear norm-based 2D locality preserving projection (NN-2DLPP) is thus proposed to improve the robustness of 2DLPP. However, NN-2DLPP does not consider the discriminant ability of data. In order to improve the discriminant ability of preserving projection methods, in this paper, we use preserving projection learning with structurally incoherence of data and propose structurally incoherent low-rank 2DLPP (SILR-2DLPP) for image classification. This approach provides a discriminative representation of preserving projection learning by recovering the distinct different classes of the data. SILR-2DLPP searches the optimal subspace and low-rank representation simultaneously. We further extend SILR-2DLPP to a kernel case and propose kernel SILR-2DLPP (KSILR-2DLPP) to obtain a nonlinear representation. The theoretical analysis including the convergence and computational complexity of SILR-2DLPP are presented. To verify the performance of SILR-2DLPP and KSILR-2DLPP, six well-known image databases were used in the experiments. The experimental results show that the proposed methods are superior to the previous preserving projection methods for image classification. [ABSTRACT FROM AUTHOR]

We present a high-quality temporal frame interpolation (TFI) method that employs piecewise-smooth motion and handles disoccluded regions using the observation that motion discontinuities travel with the foreground object. We derive a “motion discontinuity” likelihood map from the divergence of a motion field between the input frames. Motion which is modeled at the reference frame is mapped to the target frame using a cellular-affine mapping strategy—a process during which regions of disocclusion are readily observed. This information is then used to guide the occlusion-aware, bidirectional FI process. Furthermore, we propose two computationally inexpensive texture optimizations that selectively improve the quality of the interpolated frames in regions around moving objects. The scheme produces very high-quality interpolated frames and outperforms current high-quality state-of-the-art TFI schemes by 2–2.5 dB; the method works with a very low-complexity motion estimation scheme and runs orders of magnitudes faster than its competitors. [ABSTRACT FROM AUTHOR]

High efficiency video coding (HEVC), which is the newest video coding standard currently, achieves the best coding efficiency compared with all the other existing video coding standards. However, the computational complexity of the typical HEVC encoder dramatically increases because of the recursive searching scheme for finding the best coding unit (CU) partitions. In this paper, an adaptive fast CU size decision algorithm for HEVC Intra prediction is proposed based on CU complexity classification (CC) by using machine learning (ML) technology. Firstly, certain image features are extracted to characterize the CU complexity, which has a strong relationship with CU partitions, and then, the support vector machine is employed to analyze and construct the classification model according to the CU complexity. Finally, the proposed adaptive fast CU size decision algorithm, named as CCML, is released based on the complexity classification. The experimental results show that the proposed algorithm could achieve around 60% encoding time reduction for various test video sequences on average with only 1.26% Bjontegaard delta bit rate increase compared with the reference test model HM15.0 of HEVC. [ABSTRACT FROM AUTHOR]

Defogging is an essential preprocessing technique for object detection in computer vision-based systems and has been widely used in outdoor surveillance system applications. This paper proposes an efficient defogging algorithm for both static images and videos. Considering real-time applications, the proposed defogging algorithm involves a low-cost hardware oriented design that is based on an atmospheric scattering model and dark channel prior. Compared with previous low-complexity techniques, simulation results indicated that the proposed design demonstrated superior performance in terms of quantitative and qualitative evaluations. A weighting technique and a contour preserving estimation approach are adopted alternately to refine the factors in the defogging process. Furthermore, in the atmospheric light estimation, an adjuster is applied to the video for preventing “flicker” which means that the brightness changes dramatically between two neighbor frames in the video. Hence, the proposed algorithm is suitable for video defogging applications, which have not been dealt with in previous approaches. To achieve the requirement of real-time applications for both static and dynamic images, an implementation of seven-stage very-large-scale integration architecture for the proposed algorithm is presented. By using TSMC 0.13- $\mu {\mathrm{ m}}$ technology, the design yielded a processing rate of approximately 200 Mpixels/s. [ABSTRACT FROM AUTHOR]

Perceptual hashing has been broadly used in the literature to identify similar contents for video copy detection. It has also been adopted to detect malicious manipulations for video authentication. However, targeting both applications with a single system using the same hash would be highly desirable as this saves the storage space and reduces the computational complexity. This paper proposes a perceptual video hashing system for content identification and authentication. The objective is to design a hash extraction technique that can withstand signal processing operations on one hand and detect malicious attacks on the other hand. The proposed system relies on a new signal calibration technique for extracting the hash using the discrete cosine transform (DCT) and the discrete sine transform (DST). This consists of determining the number of samples, called the normalizing shift, that is required for shifting a digital signal so that the shifted version matches a certain pattern according to DCT/DST coefficients. The rationale for the calibration idea is that the normalizing shift resists signal processing operations while it exhibits sensitivity to local tampering (i.e., replacing a small portion of the signal with a different one). While the same hash serves both applications, two different similarity measures have been proposed for video identification and authentication, respectively. Through intensive experiments with various types of video distortions and manipulations, the proposed system has been shown to outperform related state-of-the art video hashing techniques in terms of identification and authentication with the advantageous ability to locate tampered regions. [ABSTRACT FROM AUTHOR]

Although advanced motion vector prediction (AMVP) modes based on motion estimation (ME) are selected significantly less due to the merge mode newly adopted in the high-efficiency video coding (HEVC), integer ME (IME) still occupies a large amount of computation in HEVC because the HEVC supports a highly flexible block partitioning structure. Introduction of the merge mode in HEVC substantially affects the optimal search algorithm of IME. Therefore, this gives a chance to reduce the computational complexity of ME with marginal drop of the compression efficiency. This paper proposes a new IME algorithm that significantly reduces its search ranges. Computational complexity of the proposed algorithm is reduced without serious degradation in coding efficiency by obtaining additional accurate motion vector predictors (MVPs) in a bottom-up order and searching narrow regions around multiple MVPs. These bottom-up MVPs are obtained from the prediction units (PUs) in the coding units (CUs) in the hierarchically lower level, which share the pixel area either completely or partially with the current PU. Although the encoder should perform the proposed IME in a bottom-up order from the smallest CUs to the larger CUs, several other processes, such as fractional ME and merges mode are performed in a top-down order to exploit several fast algorithms already adopted in the HEVC test model encoder. To keep compatibility with HEVC standard, the bottom-up MVPs are utilized for only IME. The bitstream is generated using standard AMVP. According to the simulation results, it is confirmed that the proposed algorithm improves the accuracy of MVPs, which leads to the reduction of IME computational complexity up to 81.95% on average with the Bjøntegaard delta bitrate of 0.39%. [ABSTRACT FROM AUTHOR]

In this paper, we propose a fast H.264-to-HEVC transcoder composed of a motion propagation algorithm and a fast mode decision framework. The motion propagation algorithm creates a motion vector candidate list at the coding tree unit (CTU) level and, thereafter, selects the best candidate at the prediction unit level. This method eliminates computational redundancy by pre-computing the prediction error of each candidate at the CTU level and reusing the information for various partition sizes. The fast mode decision framework is based on a post-order traversal of the CTU and includes several mode reduction techniques. In particular, the framework permits the early termination of the rate distortion cost computation, a highly complex task, when a mode is unpromising. Moreover, a novel method exploits the data created by the motion propagation algorithm to determine whether a coding unit must be split. This allows the pruning of unpromising sub-partitions. Compared with a cascaded pixel-domain transcoding approach, the experimental results show that the proposed solution using one reference frame is on average 8.5 times faster, for an average Bjøntegaard delta-rate (BD-Rate) of 2.63%. For a configuration with four reference frames, the average speed-up is 11.77 times and the average BD-Rate is 3.82%. [ABSTRACT FROM AUTHOR]

In this paper, we propose an event-guided support vector machine (ESVM) for tracking high-speed moving objects. Tracking fast-moving objects with low frame rate cameras is always difficult due to motion blur and large displacements. The accuracy problem can be solved by using high frame rate cameras at the expense of tremendous computational cost. For this issue, our ESVM incorporates event-based guiding methods into the traditional structured support vector machine to improve the tracking accuracy at a relatively low-complexity level. The event-based guiding methods include two models, event position guided search localization and event intensity guided sample supplement, which are based on the event features of the CeleX motion sensor. The motion sensor continuously responds to intensity change, which is generally related to object motion. Once it has detected intensity change, the motion sensor outputs event packages, and each of them contains the pixel location, time stamp, and pixel illumination. The generated events are continuous in the temporal domain and thus record the motion trajectory of fast-moving objects, which cannot be fully captured by frame-based cameras. In this paper, we convert high-speed test sequences into sequences of spiking events recorded by the CeleX motion sensor. Our approach presents fairly high computational efficiency, and experiments over sequences from multiple tracking benchmarks demonstrate the superior accuracy and real-time performance of our method, compared to the state-of-the-art trackers. [ABSTRACT FROM AUTHOR]