Search

In scalable video coding (SVC) as an extension of H.264/advanced video coding (AVC), a computationally expensive exhaustive mode decision is employed to select the best coding mode for each macroblock (MB). In order to reduce computational complexity, we propose a fast mode decision algorithm for SVC which uses an all-zero block (AZB) detection. Based on the empirical analysis of the inter-layer correlation of the AZB, the MB at the enhancement layer (EL) is predicted whether to be the AZB. Then, only predicted MBs are examined by the AZB detection algorithm. Since the mode decision can be terminated by detecting the AZB, we determine the processing order of the candidate modes at the EL in order to terminate the mode decision in the early stage. The proposed algorithm can be combined with other conventional fast mode decision methods in order to further reduce the computational complexity of those methods. Experimental results show that the proposed algorithm can significantly speed up the encoding process, especially for low bit-rate video sequences, without deteriorating the coding efficiency of SVC. [ABSTRACT FROM AUTHOR]

Wyner-Ziv (WZ) coding of video utilizes simple encoders and highly complex decoders. A transcoder from a WZ codec to a traditional codec can potentially increase the range of applications for WZ codecs. We present a transcoder scheme from the most popular WZ codec architecture to a differential pulse code modulation/discrete cosine transform codec. As a proof of concept, we implemented this transcoder using a simple pixeldomain WZ codec and the standard H.263+. The transcoder design aims at reducing complexity as a large amount of computation is saved by reusing the motion estimation, calculated at the side information generation process, and the I-frame streams. New approaches are used to generate side information and to map motion vectors for the transcoder. Results are presented to demonstrate the transcoder performance. [ABSTRACT FROM AUTHOR]

This paper explains the development of a highly efficient progressive 3-D mesh geometry coder based on the region adaptive transform in the spectral mesh compression method. A hierarchical set partitioning technique, originally proposed for the efficient compression of wavelet transform coefficients in high performance wavelet-based image coding methods, is proposed for the efficient compression of the coefficients of this transform. Experiments confirm that the proposed coder employing such a region adaptive transform has a high compression performance rarely achieved by other state of the art 3-D mesh geometry compression algorithms. A new, high-performance fixed spectral basis method is also proposed for reducing the computational complexity of the transform. Many-to-one mappings are employed to relate the coded irregular mesh region to a regular mesh whose basis is used. To prevent loss of compression performance due to the low-pass nature of such mappings, transitions are made from transform-based coding to spatial coding on a per region basis at high coding rates. Experimental results show the performance advantage of the newly proposed fixed spectral basis method over the original fixed spectral basis method in the literature that employs one-to-one mappings. [ABSTRACT FROM AUTHOR]

The newly developed enhanced hexagonal-based search using point-oriented inner search (EHS-POIS) enormously speeds up hexagon-based search (HS). From a different perspective, an inherent correlation between distortion and spatial direction through statistical analysis is found. Based on the observed distortion distribution, a novel enhanced hexagonalbased search with direction-oriented inner search (EHS-DIOS) is proposed to avoid real distortion calculation and thus reduce high computation. Experimental results show that, the proposed algorithm is faster than EHS-POIS by achieving two times improvement in terms of inner search speed, and as compared with previous works, it makes a better tradeoff between speed and decoded image quality. [ABSTRACT FROM AUTHOR]

The motion estimation process and coding mode selection are responsible for a large portion of the computational effort in H.264-based video encoding systems optimized for rate-distortion (RD). This paper presents a rate sorting and truncation strategy that incorporates the RD optimization criterion in the decision to evaluate distortion for both motion vectors and coding modes. Experimental results confirm the effectiveness of the proposed approach, yielding up to a 90% reduction in the computational complexity. An additional saving can also be obtained, with insignificant RD performance loss, by using a quality threshold. [ABSTRACT FROM AUTHOR]

Rate-distortion cost estimation is useful for many H.264/advanced video coding (AVC) applications including rate-distortion optimization (RDO) for mode-decision and rate-control. In this paper, we propose a new rate-prediction model and an adaptive algorithm to provide more accurate estimation of the number of coding bits for encoding intra and inter-blocks compared to previously proposed methods. The rate estimation is modeled by a linear combination of existing coding parameters, which are more accurately related to entropy coding and transform coefficients. Based on the proposed model, a cost estimation function is also proposed to give a more accurate rate-distortion cost estimation. Furthermore, we propose a block classification approach to further improve the effectiveness of the proposed scheme. The proposed schemes can achieve better results in estimating the bit-rate and rate-distortion cost compared to previously proposed approaches. [ABSTRACT FROM AUTHOR]

Portable video communication devices operate on batteries with limited energy supply. Video compression is computationally intensive and energy-demanding. Therefore, one critical issue in portable video communication system design is to minimize the energy consumption of video encoding so as to prolong the operational lifetime of portable video devices. In this paper, we explore advanced methods in adaptive system control and develop an online complexity control and energy minimization scheme for real-time video encoding. More specifically, we introduce a set of parameters to control the computational complexity and energy consumption of the video encoder. We consider this video encoder as a nonlinear control system. Based on adaptive critic design, an advanced adaptive control method recently developed in control science, we design an online control scheme which is able to select the best configuration of complexity control parameters to minimize the energy consumption under rate-distortion constraints, or equivalently maximize video quality under rate and energy constraints. Our extensive experimental results demonstrate that the proposed scheme approaches the true optimum performance. The proposed online complexity control and energy minimization scheme will provide an important tool for energy minimization of portable video devices. [ABSTRACT FROM AUTHOR]

Stereo vision is a well-known ranging method because it resembles the basic mechanism of the human eye. However, the computational complexity and large amount of data access make real-time processing of stereo vision challenging because of the inherent instruction cycle delay within conventional computers. In order to solve this problem, the past 20 years of research have focused on the use of dedicated hardware architecture for stereo vision. This paper proposes a fully pipelined stereo vision system providing a dense disparity image with additional sub-pixel accuracy in real-time. The entire stereo vision process, such as rectification, stereo matching, and post-processing, is realized using a single field programmable gate array (FPGA) without the necessity of any external devices. The hardware implementation is more than 230 times faster when compared to a software program operating on a conventional computer, and shows stronger performance over previous hardware-related studies. [ABSTRACT FROM AUTHOR]

In scalable video coding (SVC), an exhaustive mode decision is performed to search for the best mode at each macroblock. Although this method achieves an optimal trade-off between rate and distortion, it introduces an extreme computational burden on the encoder. In this letter, we propose a fast mode decision algorithm that can reduce the computational load of the mode decision for SVC. We statistically derive the expectation of the rate-distortion cost (RDcost) increase caused by skipping each mode in the mode decision. In the proposed algorithm, the encoder performs the mode decision using a small number of modes that are determined based on the expected increase of the RDcost. Experimental results show that the proposed algorithm can reduce the computational complexity significantly with negligible video quality degradation and bitrate increment. [ABSTRACT FROM AUTHOR]

This paper proposes a novel approach to early determination of zero-quantized 8 x 8 discrete cosine transform (DCT) coefficients for fast video encoding. First, with the dynamic range analysis of DCT coefficients at different frequency positions, several sufficient conditions are derived to early determine whether a prediction error block (8 x 8) is an all-zero or a partial-zero block, i.e., the DCT coefficients within the block are all or partially zero-quantized. Being different from traditional methods that utilize the sum of absolute difference (SAD) of the entire prediction error block, the sufficient conditions are derived based on the SAD of each row of the prediction error block. For partial-zero blocks, fast DCTIIDCT algorithms are further developed by pruning conventional 8-point butterflybased DCTIIDCT algorithms. Experimental results exhibit that the proposed early determination algorithm greatly reduces computational complexity in terms of DCT/IDCT, quantization, and inverse quantization, as compared with existing algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel 2-D layered multiple description coding (2DL-MDC) for error-resilient video transmission over unreliable networks. The proposed 2DL-MDC scheme allocates multiple description sub-bitstreams of a 2-D scalable bitstream to two network paths with unequal loss rates. We formulate the 2-D scalable rate-distortion problem and derive the expected distortion for the proposed scheme. To minimize the end-to-end distortion given the total rate budget and packet loss probabilities, we need to optimally allocate source and channel rates for each hierarchical sublayer of the scalable bitstream. The conventional Lagrangian multiplier method can be utilized to solve this problem but with overwhelming computational complexity. Therefore, we consider the use of the genetic algorithm to solve the rate-distortion optimization problem. The simulation results verify that the proposed method is able to achieve significant performance gain as opposed to the conventional equal rate allocation method. [ABSTRACT FROM AUTHOR]

The design of embedded hardware/software systems is often subject to strict requirements concerning its various aspects, including real-time performance, power consumption, and die area. Especially for data intensive applications, the number of memory accesses is a dominant factor for these aspects. In order to meet the requirements and design a well- adapted system, the software parts need to be optimized and an adequate system and processor architecture needs to be designed. In this paper, we focus on finding an optimized memory hierarchy for bus-based architectures. Additionally, useful instruction set extensions for application-specific processor cores are explored. For complex applications, this design space exploration is difficult and requires in-depth analysis of the application and its implementation alternatives. Tools are required which aid the designer in the design, optimization, and scheduling of hardware and software. We present a profiling tool for fast and accurate performance, power, and memory access analysis of embedded systems. This paper shows how the tool can be applied for an efficient hardware/software co-exploration within the design flow of processor-centric architectures. This concept has been proven in the design of a mixed hardware/software system with multiple processing units for video decoding. [ABSTRACT FROM AUTHOR]

Rate control plays an important role in regulating the bit rate to meet the bandwidth and storage requirement. Most existing video encoders regulate the bit rate by adjusting the quantization step size. We propose to incorporate a new dimension: the quantization rounding offset into rate control. In this paper, we present a rate control algorithm with adaptive rounding offsets (ARO) that jointly adjusts the quantization step size and the rounding offset for high bit rate accuracy. Different from the quantization step size that has a limited number of choices, the rounding offset is a continuously adjustable variable that allows the rate control algorithm to reach any precision in principle. Our extensive experimental results show that the proposed ARO algorithm significantly improves the rate control accuracy at almost no extra computational complexity. Compared with the p-domain rate control, the ARO algorithm reduces the rate control errors from about 2% to 0.5% for INTRA frames, and 5% to 1.5% for INTER frames. Our experiments also demonstrate that ARO provides the extra benefit of smoother visual quality. [ABSTRACT FROM AUTHOR]

This letter tackles the problem of reducing the complexity of H.264 decoding. Since deblocking accounts for a significant percentage of H.264 decoding time, our focus is on the H.264 in-loop deblocking filter. Observing that branch operations are costly and that in the deblocking process there are events with significantly high probability of occurrence, we regroup and simplify the branch operations. We apply the idea of Huffman tree optimization to speed up the boundary strength derivation and the true-edge detection by taking advantage of the biased statistical distribution. Our analyses and experiments show that the proposed techniques can reduce the deblocking computation time typically by a factor of more than seven times, while maintaining the bit-exact output. [ABSTRACT FROM AUTHOR]

The variable block-size motion estimation (ME) process is the H.264/AVC encoder's most time-consuming function. This letter proposes to reduce the complexity of the ME process with an early termination algorithm that features an adaptive threshold based on the statistical characteristics of rate-distortion (RD) cost regarding current block and previously processed blocks and modes. In this method, most motion searches can be stopped early, with a large number of search points saved. A region-based search is also suggested to further reduce the computation required for full search ME. A search point reduction scheme for the fast motion estimation of H.264/AVC is also introduced, and the experimental results illustrate how the proposed method reduces ME times for full search and fast motion estimation by about 77 and 31%, respectively, despite the insignificant degradation of RD performance. [ABSTRACT FROM AUTHOR]

Video deinterlacing can be realized using time space interpolation filters. To improve video deinterlacing quality with respect to missing pixels on moving diagonal lines, we developed a fuzzy concept that utilizes deinterlacing methods. The proposed algorithm consists of three parts. The first part is fuzzy rule-assisted edge-preserving-based deinterlacing (FED), which has an edge-preserving unit that utilizes fuzzy theory to find the most accurate edge direction and interpolates the missing pixels. Using the introduced gradients in the interpolation, the vertical resolution in the deinterlaced image is subjectively concealed. The second part is weighted fuzzy-reasoning-assisted deinterlacing (WFD), which works in the spatio-temporal domain. In this part, the computed weights are considered and multiplied by the candidate deinterlaced pixels, which successively build approximations of the deinterlaced sequence. The third part of the proposed algorithm is weighted fuzzy switching filtering, which analyzes the suitability of each method on system performance and uses a switching algorithm between FED and WFD. The outcome of image interpolation can be adjusted continuously by varying the setting of the membership function for fuzzy inference. Compared with conventional image interpolation methods, the algorithm presented in this paper provides improved edge quality in the deinterlaced image without the introduction of evident artifacts. [ABSTRACT FROM AUTHOR]

Image sequences of smoke, fire, etc. are known as dynamic textures. Research is mostly limited to characterization of single dynamic textures. In this paper we address the problem of detecting the presence of multiple dynamic textures in an image sequence by establishing a correspondence between the feature space of dynamic textures and that of their mixture in an image sequence. Accuracy of our proposed technique is both analytically and empirically established with detection experiments yielding 92.5% average accuracy on a diverse set of dynamic texture mixtures in synthetically generated as well as real-world image sequences. [ABSTRACT FROM AUTHOR]

We investigate image and video denoising using adaptive dual-tree discrete wavelet packets (ADDWP), which is extended from the dual-tree discrete wavelet transform (DDWT). With ADDWP, DDWT subbands are further decomposed into wavelet packets with anisotropic decomposition, so that the resulting wavelets have elongated support regions and more orientations than DDWT wavelets. To determine the decomposition structure, we develop a greedy basis selection algorithm for ADDWP, which has significantly lower computational complexity than a previously developed optimal basis selection algorithm, with only slight performance loss. For denoising the ADDWP coefficients, a statistical model is used to exploit the dependency between the real and imaginary parts of the coefficients. The proposed denoising scheme gives better performance than several state-of-the-art DDWT-based schemes for images with rich directional features. Moreover, our scheme shows promising results without using motion estimation in video denoising. The visual quality of images and videos denoised by the proposed scheme is also superior. [ABSTRACT FROM AUTHOR]

Motion estimation (ME) plays an important role in modern video coders since it consumes approximately 60-80% of the entire encoder's computations. In this paper, three novel techniques are proposed to effectively speed up the ME process. First, a smart prediction technique for effectively deciding an initial search center is proposed. Second, a zero motion prejudgment technique is proposed to accurately decide whether the pre-estimated ISC can be considered as a best match motion vector (MV) and consequently save the required computations for the MV refinement process. Finally, a variable padding pixels ME technique is proposed to adaptively determine the number of padding pixels required for the search window for more computational cost savings. The three techniques are combined and applied to the block-based ME for a superior computational complexity savings in the ME process. The performance of the proposed techniques is tested in both the pixel domain ME and the frequency domain ME in terms of their quantitative visual quality (peak signal-to-noise ratio, PSNR), their computational complexity, and their bit rate. Experimental results demonstrate that the proposed fast ME technique is able to achieve approximately a 99.4% reduction in ME time compared to the conventional full search block-based ME (FSSBB-ME) with negligible degradation in both the PSNR and the bit rate. Additionally, the experimental results prove the effectiveness of the proposed techniques if they are combined with any blockbased ME technique such as the fast extended diamond enhanced predictive zonal search. Experimental results also demonstrate that there is at least an additional savings of 72% in ME time. using the conventional discrete cosine transform phase correlation ME (DCT-PC-ME) in the frequency domain compared to the conventional FSBB-ME technique in pixel domain. Compared to the conventional DCT-PC-ME, applying the proposed novel techniques to the DCT-PC-ME saves up to 89% in ME time. [ABSTRACT FROM AUTHOR]

In this paper, the one-dimensional (1-D) fast 8 x 8 inverse integer transform algorithm for Windows Media Video 9 (WMV-9/VC-1) is proposed. Based on the symmetric property of the integer transform matrix and the matrix operations, which denote the row/column permutations and the matrix decompositions, the efficient fast 1-D 8 x 8 inverse integer transform is developed. Therefore, the computational complexities of the proposed fast inverse transform are smaller than those of the direct method and the previous fast method. With low complexity, the proposed fast algorithm is suitable to accelerate the video coding computations. [ABSTRACT FROM AUTHOR]

The displayed gray level of a plasma display panel (PDP) is often different from the intended target gray level because the displayed level is affected by the load of the panel. The effect of the load on the displayed gray level is decomposed into three factors: total load; line load magnitude; and line load distribution. Previous attempts to reduce this difference (i.e., error) have achieved only limited success because they have not taken three factors into consideration altogether. This paper proposes a new method that compensates for the error caused by all three factors. The proposed method first attempts to compensate for the error caused by line load magnitude and distribution. To this end, the error generated by one sub-field of a PDP is propagated to and compensated for by the remaining sub-fields. The amount of the error is reduced for every incidence of error propagation and, consequently, the final amount of error is reduced to a negligible level. To reduce the computational complexity for the evaluation of the amount of the error caused by the line load distribution, an iterative method is proposed to derive the effect of line load distribution on each cell from its adjacent cell. When sub-field coding is completed for a whole cells on a panel, total load of each sub-field is obtained and the error caused by total load is compensated for by controlling the number of sustain pulses for each sub-field. A significant error reduction is achieved, which shown by simulations and experiments with a 42-in. PDP set, including 91.2% reduction of the mean absolute error obtained through simulations and 69.7% reduction of luminance variation range caused by the load variation in experiments. [ABSTRACT FROM AUTHOR]

Features of the JPEG2000 compression standard include coding efficiency at low bit rates. However, its compressed bit stream is sensitive to transmission error. This paper presents three techniques to reduce both the computational complexity and the memory requirement in the ternary MQ arithmetic decoding. Such coders introduce a controlled degree of redundancy during the encoding process, which can be exploited at the decoder side in order to detect and correct errors. Our proposed techniques result in a substantial saving of decoding time and memory usage, with no or little degradation in the PSNR metric. [ABSTRACT FROM AUTHOR]

One common way to extend the battery life of a portable device is to reduce the LCD backlight intensity. In contrast to previous approaches that minimize the power consumption by adjusting the backlight intensity frame by frame to reach a specified image quality, the proposed method optimizes the image quality for a given backlight intensity. Image is enhanced by performing brightness compensation and local contrast enhancement. For brightness compensation, global image statistics and backlight level are considered to maintain the overall brightness of the image. For contrast enhancement, the local contrast property of human visual system (HVS) is exploited to enhance the local image details. In addition, a brightness prediction scheme is proposed to speed up the algorithm for display of video sequences. Experimental results are presented to show the performance of the algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we present a parallel architecture for fast and robust face detection implemented on FPGA hardware. We propose the first implementation that meets both real-time requirements in an embedded context and face detection robustness within complex backgrounds. The chosen face detection method is the Convolutional Face Finder (CFF) algorithm, which consists of a pipeline of convolution and subsampling operations, followed by a multilayer perception. We present the design methodology of our face detection processor element (PE). This methodology was followed in order to optimize our implementation in terms of memory usage and parallelization efficiency. We then built a parallel architecture composed of a PE ring and an FIFO memory, resulting in a scalable system capable of processing images of different sizes. A ring of 25 PEs running at 80 MHz is able to process 127 QVGA images per second and performing real-time face detection on VGA images (35 images per second). [ABSTRACT FROM AUTHOR]

This paper presents a probabilistic approach for automatically segmenting foreground objects from a video sequence. In order to save computation time and be robust to noise effects, a region detection algorithm incorporating edge information is first proposed to identify the regions of interest, within which the spatial relationships are represented by a region adjacency graph. Next, we consider the motion of the foreground objects and, hence, utilize the temporal coherence property in the regions detected. Thus, the foreground segmentation problem is formulated as follows. Given two consecutive image frames and the segmentation result priorly obtained, we simultaneously estimate the motion vector field and the foreground segmentation mask in a mutually supporting manner by maximizing the conditional joint probability density function of these two elements. To represent the conditional joint probability density function in a compact form, a Bayesian network is adopted, which is derived to model the interdependency of these two elements. Experimental results for several video sequences are provided to demonstrate the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

This paper presents a bit-depth scalable coding solution that is compatible with the scalable extension of H.264/Advanced Video Coding (AVC), also referred to as scalable video coding (SVC). The proposed solution is capable of providing an 8-bit AVC main profile or high-profile base layer-coded bitstream multiplexed with a higher bit-depth-enhancement layer coded bitstream generated through macroblock level inter-layer bit-depth prediction. New decoding processes for inter-layer prediction are introduced to enable bit-depth scalability. Compatibility with other types of scalability in the SVC standard—temporal, spatial, and SNR scalability—is ensured. It also supports the single-loop decoding required in the SVC specification. Further- more, it supports adaptive inter-layer prediction to determine whether or not the inter-layer bit-depth prediction shall be invoked. This solution is implemented on the basis of the SVC reference software Joint Scalable Video Model version 8.12. Experimental results are presented on 8-bit to 10-bit bit-depth scalability and also combined bit-depth and spatial scalability. [ABSTRACT FROM AUTHOR]

Intra-mode and inter-mode predictions have been made available in H.264/AVC for effectively improving coding efficiency. However, exhaustively checking for all the prediction modes for identifying the best one (commonly referred to as exhaustive mode decision) greatly increases computational complexity. In this paper, a fast mode decision algorithm, called the motion activity-based mode decision (MAMD), is proposed to speed up the encoding process by reducing the number of modes required to be checked in a hierarchical manner, and is as follows. For each macroblock, the proposed MAMD algorithm always starts with checking the rate-distortion (RD) cost computed at the SKIP mode for a possible early termination, once the RD cost value is below a predetermined "low" threshold. On the other hand, if the RD cost exceeds another "high" threshold, then this indicates that only the intra modes are worthwhile to be checked. If the computed RD cost falls between the above-mentioned two thresholds, the remaining seven modes, which are classified into three motion activity classes in our work, will be examined, and only one of the three classes will be chosen for further mode checking. The above-mentioned motion activity can be quantitatively measured, which is equal to the maximum city- block length of the motion vector taken from a set of adjacent macroblocks (i.e., region of support, ROS). This measurement is then used to determine the most possible motion-activity class for the current macroblock. Experimental results have shown that, on average, the proposed MAMD algorithm reduces the computational complexity by 62.96%, while incurring only 0.059 dB loss in PSNR (peak signal-to-noise ratio) and 0.19% increment on the total bit rate compared to that of exhaustive mode decision, which is a default approach set in the JM reference software. [ABSTRACT FROM AUTHOR]

In this paper, a joint complexity-distortion optimization approach is proposed for real-time H.264 video encoding under the power-constrained environment. The power consumption is first translated to the encoding computation costs measured by the number of scaled computation units consumed by basic operations. The solved problem is then specified to be the allocation and utilization of the computational resources. A computation allocation model (CAM) with virtual computation buffers is proposed to optimally allocate the computational resources to each video frame. In particular, the proposed CAM and the traditional hypothetical reference decoder model have the same temporal phase in operations. Further, to fully utilize the allocated computational resources, complexity-configurable motion estimation (CAME) and complexity-configurable mode decision (CAMD) algorithms are proposed for H.264 video encoding. In particular, the CAME is performed to select the path of motion search at the frame level, and the CAMD is performed to select the order of mode search at the macroblock level. Based on the hierarchical adjusting approach, the adaptive allocation of computational resources and the fine scalability of complexity control can be achieved. [ABSTRACT FROM AUTHOR]

Video surveillance has been widely used in recent years to enhance public safety and privacy protection. A video surveillance system that deals with content analysis and activity monitoring needs efficient transmission and storage of the surveillance video data. Video compression techniques can be used to achieve this goal by reducing the size of the video with no or small quality loss. State-of-the-art video compression methods such as H.264/AVC often lead to high computational complexity at the encoder, which is generally implemented in a video camera in a surveillance system. This can significantly increase the cost of a surveillance system, especially when a mass deployment of end cameras is needed. In this paper, we discuss the specific considerations for surveillance video compression. We present a surveillance video compression system with low-complexity encoder based on Wyner-Ziv coding principles to address the tradeoff between computational complexity and coding efficiency. In addition, we propose a backward-channel aware Wyner-Ziv (BCAWZ) video coding approach to improve the coding efficiency while maintaining low complexity at the encoder. The experimental results show that for surveillance video contents, BCAWZ can achieve significantly higher coding efficiency than H.264/AVC INTRA coding as well as existing Wyner-Ziv video coding methods and is close to H.264/AVC INTER coding, while maintaining similar coding complexity with INTRA coding. This shows that the low motion characteristics of many surveillance video contents and the low-complexity encoding requirement make our scheme a particularly suitable candidate for surveillance video compression. We further propose an error resilience scheme for BCAWZ to address the concern of reliable transmission in the backward-channel, which is essential to the quality of video data for real-time and reliable object detection and event analysis. [ABSTRACT FROM AUTHOR]

In H.264/AVC video encoding, zero quantized discrete cosine transform (DCT) coefficients are quite common in low bit-rate video application. The computations of DCT and quantization can be remarkably reduced if zero quantized DCT coefficients are detected prior to DCT and quantization. Many methods have been applied to early detect most zero quantized DCT coefficients. But the detection ratios of zero quantized DCT coefficients need to be improved when quantization parameter (QP) is small. We present an adaptive method to detect zero quantized DCT coefficients. When QP exceeds a certain value, a new threshold is derived to efficiently detect the all-zero blocks (AZBs) without any video degradation. Otherwise, a concept of fourteen-zeros block (FZB), which means only two coefficients are non-zero in 16 DCT coefficients of a 4 x 4 block, is proposed. An innovative method to process FZBs is proposed to reduce the redundant computations. Experimental results show that the proposed adaptive method achieves approximately 8%-20% computational savings, compared with that of the existing methods. [ABSTRACT FROM AUTHOR]

In block matching, the matching criterion plays a pivotal role in both matching accuracy and computational complexity. Mean squared error (MSE) is one most widely accepted benchmark for matching accuracy. To avoid multiplication operations for simpler implementation, the sum of absolute difference (SAD) is normally taken as a substitute to approximate MSE. In this paper, we first examine statistically the quantitative deviation of SAD from MSE in terms of maximum and average deviations. To minimize the average deviation, a new measure, namely, weighted sum of absolute difference (WSAD), is proposed in a two-element case first, for which an optimal weight is obtained theoretically. The weight is approximately adapted to 1/2 so that the weighting operation can be simplified to a shift operation, and the resulting multiplication-free WSAD also shows better matching accuracy with a smaller deviation from MSE than SAD. The two-element WSAD is further extended to a multi-element case in a multilevel pyramid structure. The proposed WSAD is experimentally validated by applying it in block motion estimation for video coding, showing better rate-distortion performance than SAD. [ABSTRACT FROM AUTHOR]

The objective of dynamic voltage scaling (DVS) is to adapt the frequency and voltage for configurable platforms to obtain energy savings. DVS is especially attractive for video decoding systems due to their time-varying and highly complex workload and because the utility of decoding a frame is solely depending on the frame being decoded before its display deadline. Several DVS algorithms have been proposed for multimedia applications. However, the prior work did not take into account the video compression algorithm specifics, such as considering the temporal dependencies among frames and the required display buffer. Moreover, the effect of the passive (leakage) power when performing DVS for multimedia systems was not explicitly considered. In this paper, we determine the optimal scheduling of the active and passive states to minimize the total energy for video decoding systems. We pose our problem as a buffer-constrained optimization problem with a novel, compression-aware definition of processing jobs. We propose low-complexity algorithms to solve the optimization problem and show through simulations that significant improvements can be achieved over state-of-the art DVS algorithms that aim to minimize only the active power. [ABSTRACT FROM AUTHOR]

The mean shift (MS) algorithm for object tracking using color has recently received a significant amount of attention thanks to its effectiveness and efficiency. Most current work, unfortunately, failing to notice that object color is usually very compactly distributed, partitions uniformly the whole color space and thus leads to a large number of void bins and limited capability of representing object color distribution. Also, there lacks a systematic way to determine automatically the number of bins. Aiming to address these problems, this paper presents an adaptive binning color model for MS tracking. First, the object color is analyzed based on a clustering algorithm and, according to the clustering result, the color space of the object is partitioned into subspaces by orthonormal transformation. Then, a color model is defined by considering the weighted number of pixels as well as intra-cluster distribution based on independent component analysis (ICA), and a similarity measure is introduced to evaluate likeness between the reference and the candidate models. Finally, the MS algorithm is developed based on the proposed color model and its computational complexity is analyzed. Experiments show that the proposed algorithm has better tracking performance than the conventional MS algorithm at the cost of moderately increasing computational load. [ABSTRACT FROM AUTHOR]

In this paper, we introduce a low-complexity fine-granularity scalable (FGS) video encoder that refines both residue and motion information in the quality layers. The current scalable video coding (SVC) draft shows that significant gains can be achieved when each enhancement layer undergoes the motion estimation/motion compensation (ME/MC) process with its own motion vector field (MVF). However, given the high computational cost of MEIMC, a motion-refined FGS scheme can be expensive to implement. The proposed scheme controls the macroblock (MB) mode allowed in the base layer and channels computational resources to refine motion in enhancement layers. Through a proper selection of Lagrangian factor for the generation of the first MVF, it is possible to design a low-complexity FGS encoder that has good overall coding performance. A simplified motion-refinement scheme is also adopted for selected MBs in enhancement layers by exploiting the correlation of MB-type information between successive layers to further reduce the complexity of the FGS encoder. Meanwhile, a framework of rate-distortion-complexity optimization is proposed for the FGS by considering the interpolation complexity during the ME/MC in each layer. The FGS decoder can be simplified through the reduction in the number of interpolations. [ABSTRACT FROM AUTHOR]

The H.264 video coding standard supports efficient coding of video at the expense of high computational complexity. This work addresses the problem of maintaining acceptable video coding performance in a computation-constrained application scenario. A complexity management approach is proposed for an H.264 encoder running in a processor/power-constrained environment. We hypothesize that, in a power-constrained application such as mobile video telephony, good perceptual quality requires a balance between a high frame rate and acceptable image quality. Therefore, the objective of the complexity management approach is to maintain a smooth video frame rate whilst ensuring that the frame quality is not degraded unacceptably. A frame-level algorithm calculates a target coding time for each frame and drops frames when necessary to maintain acceptable image quality. A per-frame algorithm controls the coding complexity of each frame in order to achieve the target coding time. The performance of the approach is evaluated by carrying out subjective tests and comparing the managed complexity encoder with a reference encoder in a computation-constrained scenario. Subjective results show that the managed complexity encoder consistently achieves superior perceptual video quality ratings compared to the reference encoder. [ABSTRACT FROM AUTHOR]

In this paper, we address the problem of privacy protection in video surveillance. We introduce two efficient approaches to conceal regions of interest (ROIs) based on transform-domain or codestream-domain scrambling. In the first technique, the sign of selected transform coefficients is pseudorandomly flipped during encoding. In the second method, some bits of the codestream are pseudorandomly inverted. We address more specifically the cases of MPEG-4 as it is today the prevailing standard in video surveillance equipment. Simulations show that both techniques successfully hide private data in ROIs while the scene remains comprehensible. Additionally, the amount of noise introduced by the scrambling process can be adjusted. Finally, the impact on coding efficiency performance is small, and the required computational complexity is negligible. [ABSTRACT FROM AUTHOR]

Motion estimation is a multistep process that encompasses techniques such as motion vector prediction, determination of search range and search patterns, and identification of termination criteria. Each of these techniques has several diversions that may suit a particular set of video characteristics. It would be hard to conceive a universal algorithm that can perform well for all kinds of video contents. However, if important characteristics of a video sequence can be identified and utilized for adjusting various steps of motion estimation, one can design an adjustable algorithm that can tune its parameters to suit the video at hand. A multistage motion estimation algorithm that includes a pre-stage for analyzing the motion characteristics of a video sequence online is proposed. This stage predicts the motion vector field (MVF) from the previous coded frame, and clusters macroblocks into background and foreground regions based on the predicted MVF. The information from the pre-stage are passed on to the next stage, which includes a mathematical model for block distortion surface to estimate the distance from the current search point to the global optimal position. This allows the motion estimation stage to adjust its search strategies accordingly. The search is performed on a precise area according to the statistical properties of the motion vector prediction error, separately for foreground and background regions. The proposed algorithm, including the pre-stage, is fast and, therefore, is suitable for online and real-time encoding. Extensive simulation results obtained for several video sequences affirm the effectiveness of the proposed algorithm. The self-tuning property enables the algorithm to perform well for several types of benchmark sequences, yielding better video quality without exceeding the time complexity as compared to the other predictive motion estimation algorithms. [ABSTRACT FROM AUTHOR]

A novel semi-fuzzy (SF) rate control algorithm (RCA) for variable bit rate (VBR) video applications is proposed. The proposed RCA is optimized to provide high quality compressed video bit streams in a wide operating range from constant quality to nearly constant bit rate. Thanks to a low degree of computational complexity, it is suitable for real-time applications of VBR video. The proposed RCA operates under given buffer size, delay and quality constraints. It provides a VBR video bit stream by controlling the quantization parameter (QP) on a picture basis. The QP is mainly controlled by a fuzzy rate controller and a deterministic quality controller, which are optimized such that they minimize the variation of quality to provide encoded video with high and stable visual quality. The proposed RCA has been implemented in an H.264/AVC video codec and the experimental results show that it provides a high-level average quality for encoded video while strictly obeying the buffering delay and quality constraints. [ABSTRACT FROM AUTHOR]

A complexity reduction algorithm tailored for the H.264/AVC encoder is described. It aims to alleviate the computational burden imposed by Lagrangian rate distortion optimization in the inter-mode selection process. The proposed algorithm is described as a hierarchical structure comprising three levels. Each level targets different types of macroblocks according to the complexity of the search process. Early termination of mode selection is triggered at any of the levels to avoid a full cycle of Lagrangian examination. The algorithm is evaluated using a wide range of test sequences of different classes. The results demonstrate a reduction in encoding time of at least 40%, regardless of the class of sequence. Despite the reduction in computational complexity, picture quality is maintained at all bit rates. [ABSTRACT FROM AUTHOR]

H.264 video coding standard supports several interprediction coding modes that use macroblock (MB) partitions with variable block sizes. Rate-distortion (R-D) optimal selection of both the motion vectors (MVs) and the coding mode of each MB is essential for an H.264 encoder to achieve superior coding efficiency. Unfortunately, searching for optimal MVs of each possible subblock incurs a heavy computational cost. In this paper, in order to reduce the computational burden of integer-pel motion estimation (ME) without sacrificing from the coding performance, we propose a R-D and complexity joint optimization framework. Within this framework, we develop a simple method that determines for each MB which partitions are likely to be optimal. MV search is carried out for only the selected partitions, thus reducing the complexity of the ME step. The mode selection criteria is based on a measure of spatiotemporal activity within the MB. The procedure minimizes the coding loss at a given level of computational complexity either for the full video sequence or for each single frame. For the latter case, the algorithm provides a tight upper bound on the worst case complexity/execution time of the ME module. Simulation results show that the algorithm speeds up integer-pel ME by a factor of up to 40 with less than 0.2 dB loss in coding efficiency. [ABSTRACT FROM AUTHOR]

We propose a simple yet effective deblocking method for JPEG compressed image through postfiltering in shifted windows (PSW) of image blocks. The MSE is compared between the original image block and the image blocks in shifted windows, so as to decide whether these altered blocks are used in the smoothing procedure. Our research indicates that there exists strong correlation between the optimal mean squared error threshold and the image quality factor Q, which is selected in the encoding end and can be computed from the quantization table embedded in the JPEG file. Also we use the standard deviation of each original block to adjust the threshold locally so as to avoid the over-smoothing of image details. With various image and bit-rate conditions, the processed image exhibits both great visual effect improvement and significant peak signal-to-noise ratio gain with fairly low computational complexity. Extensive experiments and comparison with other deblocking methods are conducted to justify the effectiveness of the proposed PSW method in both objective and subjective measures. [ABSTRACT FROM AUTHOR]

In this letter, we investigate the error resilience properties of JPEG2000. We identify the dependencies among coding passes of a codeblock codestream, and determine the sections of the codestream that can be salvaged in the presence of errors. In our analysis, we consider the effects of mode variations provided by the standard. The proposed methods are derived using the existing dependency structure of coding passes and do not require a substantial increase in computational complexity of the decoder. Experimental results indicate that the proposed methods can improve the error resilience performance substantially. [ABSTRACT FROM AUTHOR]

In this paper, we propose a systematic approach that approximates a target dictionary to reduce the complexity of a matching pursuit encoder. We combine calculation of the inner products and maximum atom extraction of a matching pursuit video coding scheme based on eigendictionary approximation and tree-based vector quantization. The approach makes the codec design and optimization cleaner and more systematic than previous dictionary approximation methods. We vary the quality of approximation to demonstrate the tradeoff between computational complexity and coding efficiency. The experiment results show that our codec achieves speed-up factors of up to 100 with a performance loss of less than 0.1 dB. We use double-stimulus impairment scale scores to evaluate the perceptual quality of our approach for different levels of complexity. [ABSTRACT FROM AUTHOR]

Characterized by their distinctive motion patterns, temporal textures are natural phenomenon exhibiting spatio-temporal regularity with indeterminate spatial and temporal extent. This paper presents a real-time motion-based temporal texture characterization technique for the first time using block-based motion measures with very high classification accuracy against the popular opinion that such an accurate characterization is only possible using pixel-based measures. Finding an optimal weight ratio between space and time domain features where the accuracy of this block-based technique peaks has been the essence of this success. Computational complexity analyses and classification results clearly demonstrate the capability of the proposed technique in producing comprehensive classification results comparable to the best pixel-based technique with overwhelming reduction in computational complexity. [ABSTRACT FROM AUTHOR]

Within the framework of spatiotemporal attention detection, this paper proposes an efficient method for focus of attention (FOA) detection which involves combining adaptively the spatial and motion attention to form an overall attention map. Without computing motion explicitly, it detects motion attention using the rank deficiency of grayscale gradient tensors. We also propose an attention-driven shot matching method using primarily FOA. Experimental results demonstrate the effectiveness of the proposed FOA detection method and the proposed shot matching approach outperforms other published methods. [ABSTRACT FROM AUTHOR]

This work presents a new class of algorithms for super-resolution (SR) of image sequences. This class of algorithms estimates simultaneously all frames of a sequence by employing an iterative minimization of a regularized cost function. Similarly to other SR techniques, the proposed approach exploits the correlation among the frames of the sequence. This correlated information helps to improve the resolution of the captured images. By employing the motion information only in the prior term of the cost function, the proposed method achieves a better fidelity and more robust performance, when compared to other methods. This original approach provides a fidelity equivalent to the obtained by the simultaneous methods, while achieving a lower computational complexity. The isolation of the motion equations in the prior term allows an increased control over the motion errors, and as a consequence, provides an improved robustness. The performance of the proposed method is discussed and compared with other methods in the literature. In the comparative experiments, it is considered both the minimization over the Euclidean norm, which is employed to achieve low computational complexity, and the minimization over the Huber norm or ℓ1 norm, which is used to produce images with sharp edges and higher robustness to large motion errors. [ABSTRACT FROM AUTHOR]

In order to take advantage of the byte-type data parallelism in the existing single-instruction multiple-data (SIMD) technique, this paper introduces the concept of 8-bit partial sums, obtained by a 4-bit right-shift operation on the sum of the 16 luminance values in a column of a 16 × 16 block of a video frame. Since these partial sums are of only eight bits, eight of them can be processed concurrently in a single 64-bit SIMD register. A method of employing these partial sums in order to speed up a given block motion-estimation algorithm is then proposed. The notion of the 8-bit partial sums is extended to the four-level case. It is shown that there are 15 possible methods of utilizing these multilevel 8-bit partial sums to accelerate a block motion-estimation algorithm without any loss of accuracy of the algorithm. Each of these 15 methods is used in the full-search algorithm to determine the one that provides the lowest computational complexity. This method is adopted as the chosen scheme to accelerate various block motion-estimation algorithms. Extensive simulations are carried out on eight video sequences showing that substantial speed-up can be achieved when the chosen scheme is incorporated with the various motion-estimation algorithms. The simulation results also demonstrate that the implementation on SIMD architectures can further accelerate the execution of the proposed scheme by more than 93% percent. [ABSTRACT FROM AUTHOR]

This paper presents a novel B-picture coding based on block-wise adaptive motion accuracy (BAMA) with low-complexity motion compensation (MC). BAMA is able to adaptively select the motion accuracy for each inter-block in a B-picture depending on whether it is of bidirectional prediction, in which compared to the unidirectionally predicted block, lower motion accuracy is applied to the bidirectionally predicted block to reduce MC complexity. To further reduce MC complexity for the bidirectionally predicted luminance block with subpixel motion vectors in both directions, the forward and backward MC interpolations are merged according to the linear operation property of MC interpolation. In addition, a simplified MC interpolation method is also introduced for chrominance components for the bidirectionally predicted block. When integrating the proposed methods into H.264/AVC reference software, the experimental results demonstrate that they can significantly reduce MC complexity for a B-picture while yielding the comparable rate distortion performance in comparison with H.264/AVC although they no longer generate H.264/AVC compliant bitstream. [ABSTRACT FROM AUTHOR]

In this paper, we propose a generalized discrete cosine transform (DCT) decimation scheme for DCT-domain spatial downscaling which performs two-fold decimation on subframes of a flexible size larger than the traditional 8 × 8 block size to improve the visual quality. Efficient sparse-matrix representations are then derived to reduce the computation of the proposed DCT decimation method. We compare the antialiasing filtering performances and computational complexities of the proposed downscaling scheme with the existing DCT-domain and pixel-domain downscaling schemes. Our analysis shows that the proposed scheme can reduce the aliasing artifact compared to the pixel-domain downscaling schemes, whereas the computational complexity may be increased. Experimental results are reported to show the efficacy of the proposed approach. [ABSTRACT FROM AUTHOR]

The rate-distortion (R-D) optimization technique plays an important role in optimizing video encoders. Modeling the rate and distortion functions accurately in acceptable complexity helps to make the optimization more practical. In this paper, we propose a bit-rate estimation function and a distortion measure by modeling the transform coefficients with spatial-domain variance. Furthermore, with quantization-based thresholding to determine the number, the absolute sum, and the squared sum of transform coefficients, the simplified transform-domain R-D measurement is introduced. The proposed algorithms can reduce the computation complexity for the R-D optimized mode-decision by using the new cost function evolving from the simplified transform-domain R-D model. Based on the proposed estimations, a rate-control scheme in the macroblock layer is also proposed to improve the coding efficiency. [ABSTRACT FROM AUTHOR]