Search

Low-light image enhancement algorithms can improve the visual quality of low-light images and support the extraction of valuable information for some computer vision techniques. However, existing techniques inevitably introduce color and lightness distortions when enhancing the images. To lower the distortions, we propose a novel enhancement framework using the response characteristics of cameras. First, we discuss how to determine a reasonable camera response model and its parameters. Then, we use the illumination estimation techniques to estimate the exposure ratio for each pixel. Finally, the selected camera response model is used to adjust each pixel to the desired exposure according to the estimated exposure ratio map. Experiments show that our method can obtain enhancement results with fewer color and lightness distortions compared with the several state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Due to the rapid growth of technology and the dropping cost of cameras, multiview imaging applications have attracted many researchers in recent years. Free viewpoint and 3D Televisions are among these interesting applications. One of the problems that should be solved to realize such applications is rendering. In this paper, we propose an optical flow-assisted adaptive patch-based view synthesis algorithm. This patch-based scheme reduces the size and number of holes during reconstruction. The size of patch is determined in response to edge information for better reconstruction, especially near the boundaries. In the first stage of the algorithm, disparity is obtained using optical flow estimation. Then, a reconstructed version of the left and right views is generated using our adaptive patch-based algorithm. The mismatches between each view and its reconstructed version are obtained in the mismatch detection steps. This stage results in two masks as outputs, which help with the refinement of disparities and the selection of the best patches for final synthesis. Finally, the remaining holes are filled using our simple hole-filling scheme and the refined disparities. The objective and subjective performances of the proposed algorithm are compared with recent methods. The results show that the proposed algorithm achieves an improvement of 2.14 dB on average. [ABSTRACT FROM AUTHOR]

Generic human detectors perform poorly in application scenes in which conditions are significantly different from those of the benchmark data sets. Based on the assumption that only a limited number of labeled examples are available, we propose a variant semi-supervised boosting approach for improving scene adaptiveness by utilizing unlabeled data. Specifically, we train a max-margin-based model as an initial detector for new example collection, instead of using generic detectors, and then a better model is trained via boosting in which the newly obtained examples influence the training process through their similarities to the labeled examples. Since the widely used human descriptors are usually high dimensional and redundant, we employ a graph-based method to determine the weight representing the importance of each feature, such that the weighted similarity measurement leads to a performance gain. In the experiments, the effectiveness of the proposed approach “Variant SemiBoost” is demonstrated and state-of-the-art performance on challenging data sets is achieved. [ABSTRACT FROM AUTHOR]

Retrieving videos from large repositories using image queries is important for many applications, such as brand monitoring or content linking. We introduce a new retrieval architecture, in which the image query can be compared directly with database videos—significantly improving retrieval scalability compared with a baseline system that searches the database on a video frame level. Matching an image to a video is an inherently asymmetric problem. We propose an asymmetric comparison technique for Fisher vectors and systematically explore query or database items with varying amounts of clutter, showing the benefits of the proposed technique. We then propose novel video descriptors that can be compared directly with image descriptors. We start by constructing Fisher vectors for video segments, by exploring different aggregation techniques. For a database of lecture videos, such methods obtain a two orders of magnitude compression gain with respect to a frame-based scheme, with no loss in retrieval accuracy. Then, we consider the design of video descriptors, which combine Fisher embedding with hashing techniques, in a flexible framework based on Bloom filters. Large-scale experiments using three datasets show that this technique enables faster and more memory-efficient retrieval, compared with a frame-based method, with similar accuracy. The proposed techniques are further compared against pre-trained convolutional neural network features, outperforming them on three datasets by a substantial margin. [ABSTRACT FROM AUTHOR]

Video surveillance systems have become extremely important recently. It has been observed that information extracted from a single spectrum video is often insufficient in adverse conditions like low illumination, shadowing, smoke, dust, unstable background, and camouflage. Real-time video processing systems also need to be very fast where future frames are often unavailable at the time of processing the current frame. In this paper, we propose a superpixel-based causal multisensor video fusion algorithm suitable for real-time surveillance tasks. We develop new superpixel level spatial and temporal saliency models. Novel superpixel level multiple fusion rules are also designed to obtain the fused output. Comprehensive comparisons with several existing works clearly indicate the benefit of our solution. [ABSTRACT FROM AUTHOR]

Screen content coding plays an important role in many applications. Conventional reference picture management (RPM) strategies developed for natural content may not work well for screen content. This is because many regions in screen content remain static for a long time, causing a lot of repetitive contents to stay in the decoded picture buffer. The repetitive contents are not conducive to inter prediction, but still occupy valuable memory. This paper proposes a diversity-based RPM scheme for screen content coding. The concept of diversity is introduced for the reference picture set (RPS) to help formulate the RPM problem. By maximizing the diversity of RPS, more potentially better predictions are provided. Better compression performance can then be achieved. Meanwhile, the proposed scheme is nonnormative and compatible with existing video coding standards, such as High Efficiency Video Coding. The experimental results show that, for low delay screen content coding, the bit saving of the proposed scheme is 4.9% on average and up to 13.7%, without increasing encoding time. [ABSTRACT FROM AUTHOR]

Guided image filtering (GIF) is a promising edge-preserving filtering technique that has been applied in a variety of applications. Nevertheless, an efficient very-large-scale integration (VLSI) architecture design of GIF is still very challenging for the real-time processing of full-high definition videos. Previously proposed architectures are somewhat inefficient in terms of either on-chip memory usage or off-chip memory bandwidth. This paper aims to improve the balance between on-chip memory usage and off-chip memory bandwidth through architecture exploration. Three critical architectural tradeoffs in the VLSI design of GIF are explored, and two efficient VLSI architectures, namely sequential line-based and parallel line-based architectures, are proposed. Experimental results demonstrate that the proposed VLSI design only consumes 34.1-K logic gates, 25.4-KB on-chip memories, and 373-MB/s off-chip memory bandwidth while achieving a real-time video processing of 1080P@60Hz at the maximum clock frequency of 297-MHz. Moreover, the proposed VLSI circuits are fully pipelined and synchronized to the pixel clock of output video, so can be seamlessly integrated into diverse real-time video processing systems. [ABSTRACT FROM PUBLISHER]

The screen content images (SCIs) quality influences the user experience and the interactive performance of remote computing systems. With numerous approaches proposed to evaluate the quality of natural images, much less work has been dedicated to reduced-reference image quality assessment (RR-IQA) of SCIs. Here, we propose an RR-IQA method from the perspective of SCI visual perception. In particular, the quality of the distorted SCI is evaluated by comparing a set of extracted statistical features that consider both primary visual information and unpredictable uncertainty. A unique property that differentiates the proposed method from previous RR-IQA methods for natural images is the consideration of behaviors when human subjects view the screen content, which motivates us to establish the perceptual model according to the distinct properties of SCIs. Validations based on the screen content IQA database show that the proposed algorithm provides accurate predictions across a wide range of SCI distortions with negligible transmission overhead. [ABSTRACT FROM PUBLISHER]

High-quality and cost-efficient image upscaling design is very important for many real-time video processing applications, especially when the display panel resolution reaches ultrahigh definition. Compared with New Edge-Directed Interpolation (NEDI) based implicit edge directional upscaling, explicit methods require less computational resource and more easily reach real-time performance, especially when the required image definition and upscaling ratio are very high. Nevertheless, the investigation of applications of explicit methods in video processing systems remains largely missing arguably because it is commonly believed that explicit edge-directed interpolation tends to introduce unexpected artifacts because of inaccurate detection and hence its image quality is relatively poor. This paper proposes an explicit edge-directed adaptive interpolation method that leverages more sophisticated edge detection and orientation estimation algorithms to avoid misinterpolation, thereby providing similar or even better image quality than those with implicit methods. Targeting the real-time 4K video display system, the proposed edge-directed image upscaling algorithm is further implemented with an efficient very-large-scale integration (VLSI) architecture. The experimental results demonstrate that the proposed interpolation algorithm outperforms previous explicit and implicit edge-directed methods in both objective and subjective tests. The presented VLSI implementation further demonstrates that the maximum output video sequence of the proposed interpolation method can reach 4\mathrm k\times 2\textk @60 Hz with a reasonable hardware cost. [ABSTRACT FROM AUTHOR]

Presents the table of contents for this issue of the publication. [ABSTRACT FROM AUTHOR]

Screen content images are originally captured in a full-chroma format. The chroma downsampling, which is commonly applied to the chroma component in screen content image representation and processing (e.g., YUV4:2:0 compression), will significantly degrade the image quality and create annoying artifacts such as blur and color shifting. To tackle this problem, in this paper we propose luma aware chroma downsampling and upsampling algorithms to jointly improve the quality of the chroma image reconstruction. Guided by the luma information, the chroma upsampling algorithm is proposed with the utilization of major color and index map representation. The geometric information-based linear mapping is developed to transfer the structure of luma to the interpolated chroma. Subsequently, the error sensitivity of the upsampling method is analyzed, and content dependent downsampling algorithm is presented to minimize the error sensitivity function. We further explore the applicability of the proposed scheme in the scenario of screen content compression, targeting at improving the decoded chroma image quality for display. Extensive experimental results demonstrate the viability and efficiency of the proposed scheme. [ABSTRACT FROM AUTHOR]

Centralized and multilevel implementations of the Panoptic omnidirectional multiaperture visual system were previously presented by us, relying on the transmission of all camera outputs to a single central processing node for omnidirectional image and video reconstruction. In this paper, a novel distributed and parallel implementation of the omnidirectional vision reconstruction algorithm of the Panoptic system is presented. The parallel approach aims to overcome the scalability problems and memory bandwidth limitations of the centralized approach. The real-time hardware implementation is presented for camera modules with image processing, memory, and interconnectivity features. A methodology is introduced for the arrangement of camera modules with interconnectivity feature into a target interconnection network topology. A unique custom-made multiple-field-programmable gate array hardware platform is introduced for the implementation of an interconnected network of 49 camera prototype Panoptic system. A hardware architecture based on presented hardware platform enabling the real-time implementation of the blending algorithms is presented, along with the imaging results and resource utilization. The real-time implementation results of the implemented omnivision application on the mentioned prototype are demonstrated. [ABSTRACT FROM PUBLISHER]

The autoregressive (AR) model has been widely used in signal processing for its effective estimation, especially in image processing. Many dedicated $2\times $ interpolation algorithms adopt the AR model to describe the strong correlation between low-resolution (LR) pixels and high-resolution (HR) pixels. However, these AR model-based methods closely depend on the fixed relative position between LR pixels and HR pixels that are nonexistent in the general scale interpolation. In this paper, we present an adaptive general scale interpolation algorithm that is capable of arbitrary scaling factors considering the nonstationarity of natural images. Different from other dedicated $2\times $ interpolation methods, the proposed AR terms are modeled by pixels with their adjacent unknown HR neighbors. To compensate for the information loss caused by mismatches of AR models, we consider a weighting scheme suitable for general scale situations based on the pixel similarity to increase accuracy of the estimation. Comprehensive experiments demonstrate the effectiveness of the proposed method on general scaling factors. The maximum gain of peak signal-to-noise ratio is 2.07 dB compared with segment adaptive gradient angle in $1.5 \times $ enlargements. To evaluate the performance in resolution adaptive video coding, we have also tested our method on Joint Scalable Video Model codec and obtained better subjective quality and rate-distortion performance. [ABSTRACT FROM PUBLISHER]

In this paper, a 2-D noncausal Markov model is proposed for passive digital image-splicing detection. Different from the traditional Markov model, the proposed approach models an image as a 2-D noncausal signal and captures the underlying dependencies between the current node and its neighbors. The model parameters are treated as the discriminative features to differentiate the spliced images from the natural ones. We apply the model in the block discrete cosine transformation domain and the discrete Meyer wavelet transform domain, and the cross-domain features are treated as the final discriminative features for classification. The support vector machine which is the most popular classifier used in the image-splicing detection is exploited in our paper for classification. To evaluate the performance of the proposed method, all the experiments are conducted on public image-splicing detection evaluation data sets, and the experimental results have shown that the proposed approach outperforms some state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

Homography estimation is a fundamental problem in the field of computer vision. For estimating the homography between two images, one of the key issues is to match keypoints in the reference image to the keypoints in the moving image. To match keypoints in real time, a binary image descriptor, due to its low matching and storage costs, emerges as a more and more popular tool. Upon achieving the low costs, the binary descriptor sacrifices the discriminative power of using floating points. In this paper, we present BB-Homography, a new approach that fuses fast binary descriptor matching and bipartite graph for homography estimation. Starting with binary descriptor matching, BB-Homography uses bipartite graph matching (GM) algorithm to refine the matching results, which are finally passed over to estimate homography. On realizing the correlation between keypoint correspondence and homography estimation, BB-Homography iteratively performs the GM and the homography estimation such that they can refine each other at each iteration. In particular, based on spectral graph, a fast bipartite GM algorithm is developed for lowering the time cost of BB-Homography. BB-Homography is extensively evaluated on both public benchmarks and live-captured video streams that consistently shows that BB-Homography outperforms conventional methods for homography estimation. [ABSTRACT FROM PUBLISHER]

Every year, a large number of wildfires all over the world burn forested lands, causing adverse ecological, economic, and social impacts. Beyond taking precautionary measures, early warning and immediate response are the only ways to avoid great losses. To this end, in this paper we propose a computer vision approach for fire-flame detection to be used by an early-warning fire monitoring system. Initially, candidate fire regions in a frame are defined using background subtraction and color analysis based on a nonparametric model. Subsequently, the fire behavior is modeled by employing various spatio-temporal features, such as color probability, flickering, spatial, and spatio-temporal energy, while dynamic texture analysis is applied in each candidate region using linear dynamical systems and a bag-of-systems approach. To increase the robustness of the algorithm, the spatio-temporal consistency energy of each candidate fire region is estimated by exploiting prior knowledge about the possible existence of fire in neighboring blocks from the current and previous video frames. As a final step, a two-class support vector machine classifier is used to classify the candidate regions. Experimental results have shown that the proposed method outperforms existing state-of-the-art algorithms. [ABSTRACT FROM PUBLISHER]

This paper addresses high performance depth coding in 3D video by making good use of its coded texture video counterpart. The relationship between the depth and its associated texture video in terms of coding mode and motion vector is carefully examined. Our statistical study suggests that the skip-coding mode and its associated motion vectors in the coded texture can be shared for depth coding by saving bit rate at the cost of little increase of distortion, which subsequently results in a nonsequential coding of the depth map. In this sense, coding/prediction of a block can be performed using the skip-coded blocks below and right, which are not available in the conventional sequential coding, thus producing the so-called omnidirectional blocks predicted in the intra-coding by making the best use of (at most) four neighboring blocks. Moreover, in view of the depth-texture structure similarity, a depth-texture cooperative clustering-based prediction method is proposed for cluster-based depth prediction in the intra-coding, which exploits the structure similarity for the current coding block and its neighboring pixels around the block. On the other hand, some large prediction errors may be present for the depth-texture misaligned pixels, which may greatly compromise the coding performance. To deal with these large residuals induced by the depth-texture misalignment, a simple yet effective detection and rectification approach is incorporated in the proposed depth coding scheme. Experimental results show that our proposed depth coding scheme achieves superior rate-distortion performance compared with other relevant coding methods. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a novel energy optimization framework to accurately estimate surface normal and reflectance of an object from an input image sequence. Input images are captured from a fixed viewpoint under varying lighting conditions. In the proposed approach we combine photometric stereo and Retinex constraints into our energy function. To formulate inter-image constraints, shading information is added to the Lambertian model to account for shadows. For intra-image constraints, we moderate the strength of shading smoothness according to shadow mask and normal variations. By minimizing this energy function we are able to recover accurate surface normals and reflectance. Experimental results show that our approach yields more realistic normal map and accurate albedo map than the state-of-the-art uncalibrated photometric stereo algorithms. As for intrinsic image decomposition, results on the real and synthetic scenes show that the proposed approach outperforms previous ones. [ABSTRACT FROM PUBLISHER]

Visible light cameras commonly used for surveillance applications usually have a limited field of view (FoV). To acquire a broader FoV, distributed aperture systems (DASs) combine views from multiple cameras. Although some commercial or proprietary systems already exist, the open literature in this field reports solely the performance and/or hardware architecture of the respective systems, and omits the required details for a reimplementation. In this paper we present a low-cost, personal-computer-based 360° DAS, with a full description of the hardware architecture and the software implementation details. In particular, we describe in detail two problems, the offline estimation of the orientations of the cameras of the proposed DAS and the online synthesis of a virtual view in real time. For the first problem, we propose an area-based bundle adjuster by combining the forward additive Lucas–Kanade algorithm with a bundle adjustment strategy. The color virtual view of up to $2048\times 1024$ pixels is synthesized online at 39 frames/s. A large majority of the workload of the online phase is implemented on the graphic processing unit. The techniques and methods proposed in this paper are generic and independent on the arrangement of cameras. [ABSTRACT FROM PUBLISHER]

An area efficient row-parallel architecture is proposed for the real-time implementation of bivariate algebraic integer (AI) encoded 2-D discrete cosine transform (DCT) for image and video processing. The proposed architecture computes 8\,\times\,8 2-D DCT transform based on the Arai DCT algorithm. An improved fast algorithm for AI-based 1-D DCT computation is proposed along with a single channel 2-D DCT architecture. The design improves on the four-channel AI DCT architecture that was published recently by reducing the number of integer channels to one and the number of eight-point 1-D DCT cores from five down to two. The architecture offers exact computation of 8\,\times\,8 blocks of the 2-D DCT coefficients up to the FRS, which converts the coefficients from the AI representation to fixed-point format using the method of expansion factors. Prototype circuits corresponding to FRS blocks based on two expansion factors are realized, tested, and verified on FPGA-chip, using a Xilinx Virtex-6 XC6VLX240T device. Post place-and-route results show a 20% reduction in terms of area compared to the 2-D DCT architecture requiring five 1-D AI cores. The area-time and area\-time^2 complexity metrics are also reduced by 23% and 22% respectively for designs with eight-bit input word length. The digital realizations are simulated up to place and route for ASICs using 45 nm CMOS standard cells. The maximum estimated clock rate is 951 MHz for the CMOS realizations indicating 7.608\cdot 10^9~pixels/s and a 8\,\times\,8 block rate of 118.875 MHz. [ABSTRACT FROM AUTHOR]

In projection-based augmented reality (AR) alleviating visual distraction of patterns has been a great challenge. As a representative one, a method of embedding patterns and their complements (hereafter, we call the pairs complementary patterns) into AR images alternately has been proposed recently. This paper presents subjective evaluation results and their statistical analysis on the visual perceptibility of embedding complementary patterns in different ways in a standard hardware environment. Then, we explore the constraints for embedded complementary patterns to be less perceptible. As expected, high projector refresh rate and low pattern strength were the general conditions for a decrease in the perception of embedded complementary patterns. However, reducing pattern size and projecting complementary patterns with an interval were also among the factors affecting the results. Detailed constraints are given in the experimental results. Also, we present which constraint is more dominant for pattern perceptibility. [ABSTRACT FROM AUTHOR]

Embedded block coding with optimized truncation (EBCOT) employed in the JPEG2000 standard accounts for the majority of the processing time, because the EBCOT is full of bit operations that cannot be implemented efficiently in software. The block coder consists of a bit-plane coder (BPC) followed by a binary arithmetic coder (BAC), where the most up-to-date BPC architectures are capable of producing symbols at a much higher rate than the conventional BACs can handle. This letter proposes a novel pipelined BAC architecture that can encode input symbols at a much higher rate than the conventional BAC architectures. The proposed architecture can significantly reduce the critical path delay and can achieve a throughput of 400Msymbols/s. The critical path delay synthesized with 0.18μm CMOS technology is 2.42 ns, which is almost half of the delay taken in conventional BAC architectures. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel motion representation and apply it to abnormality detection and behavior recognition. At first, pointwise correspondences for the foreground in two consecutive video frames are established by performing a salient- region-based pointwise matching algorithm. Then, based on the established pointwise correspondences, a pointwise motion image (PMI) for each frame is built up to represent the motion status of the foreground. The PMI is more suitable for video analysis as it encapsulates a variety of motion information such as pointwise motion speed, pointwise motion orientation, pointwise motion duration, as well as the global shape of the foreground. In addition, it represents all of these pieces of information by a color image in the HSV space, by which many popular techniques in the image processing field can be straightforwardly adopted. By combining the PMI and AdaBoost, a method for abnormality detection and behavior recognition is proposed. The proposed method is shown to possess a high discriminative ability and is capable of dealing with local motion, global motion, and similar motions with different speeds. Experiments including a comparison with two existing methods demonstrate the effectiveness of the proposed representation in abnormality detection and behavior recognition. [ABSTRACT FROM AUTHOR]

Integrating video coding and denoising is a novel processing paradigm, bringing mutual benefits to both video processing tools. In this paper, we propose a novel video denoising approach of which the main idea is reusing motion estimation resources from the video coding module for video denoising. In most cases, the motion fields produced by real-time video codecs cannot be directly employed in video denoising, since they, as opposed to noise filters, tolerate errors in the motion field. In order to solve this problem, we propose a novel motion-field filtering step that refines the accuracy of the motion estimates to a degree that is required for denoising. Additionally, a novel temporal filter is proposed that is robust against errors in the estimated motion field. Numerical results demonstrate that the proposed denoising scheme is of low-complexity and compares favorably to the state-of-the-art video denoising methods. [ABSTRACT FROM AUTHOR]

Occlusion reasoning is one of the most challenging issues in visual surveillance. In this letter, we propose a new approach for reasoning about occlusions between multiple people. In our approach, occlusion relationships between people are explicitly defined and deduction of the occlusion relationships is integrated into the whole tracking framework. The prior knowledge is supplied by a set of models which include a 2-D elliptical shape model, a spatial-color mixture of Gaussians appearance model, and a motion model with constant velocity. An observation likelihood function is constructed based on the similarity between the observations and the object appearance models with given states. The occlusion relationships are deduced from the current states of the objects and the current observations, using the observation likelihood function. The previous occlusion relationships are not required for deducing the current occlusion relationships. The problem of tracking and occlusion reasoning for more than two people is formulated mathematically, and a solution is proposed based on particle filtering. Experimental results on several real video sequences from indoor and outdoor scenes show the effectiveness of our approach. [ABSTRACT FROM AUTHOR]

This paper introduces a head pose estimation system that automatically localizes the nose-tips of the faces and estimates head poses in images simultaneously. In the training stage, the nose-tips of the faces are first manually labeled. The appearance variations caused by head pose changes are then characterized by a Tensorposes model. Given an image with unknown head pose and nose-tip location, the nose-tip of the face is automatically localized in a coarse-to-fine fashion after the skin color segmentation. The head pose is also estimated simultaneously. The performance of our system is evaluated on the Pointing'04 head pose image data set. We first evaluate the classification performance of the Tensorposes models with image patches of the faces cropped according to the manually labeled nose-tip locations of the faces in the Pointing '04 data set. By leaving-one-person-out evaluation strategy, we obtain the optimal parameters of the Tensorposes model, and evaluate the discriminative power of the Tensorposes model built based on high order singular value decomposition (HOSVD) and multilinear independent component analysis (MICA), and naive principal component analysis (PCA) subspace models. It is shown Tensorposes model by HOSVD and MICA decomposition performs similarly good but much better than naive PCA subspace models. The Tensorposes model is then utilized to automatically localize nose-tip location in the testing image and to simultaneously estimate the head pose. The nose-tip localization and pose estimation accuracy of the proposed system are evaluated against the ground truth. Finally, cross-database evaluation of the performance of our system is carried out on Pointing'04 database, a selected subset of CMU PIE database, and some pictures from CLEAR'07 head pose evaluation database. The experiments show that our system generalizes reasonably well to the real-world scenarios. [ABSTRACT FROM AUTHOR]

We explore predictive coding for the JPEG baseline sequential algorithm in this article. Our objective is to reduce inter-block redundancy of discrete cosine transform (DCT) blocks. We first present an implementation of image interpolation approach and develop a method for predicting ac coefficients of a block in terms of sum of pixels (or dc coefficients) of some surrounding patches. If we use 2-D quadratic polynomials for interpolation over 24 × 24 pixel area, our method derives the same equations, which JPEG developed for reducing blocking artifacts. Next, we show that nonpolynomial functions such as arctan perform better for ac prediction. We develop additional prediction equations considering different configurations of the interpolation area. Finally, we present performance results, which show that for images compressed at the default quality factor of 75, our equations reduce the ac energy of DCT blocks by about 22% and the final code bit size by about 5.5% over the JPEG sequential algorithm. At the quality factor of 90, the code size reduction is 10.8%. We also present an adaptive dc coding method by considering the prediction algorithm of JPEG-LS. It reduces the average JPEG dc residual by over 20% and the dc code bits by 7.75%. [ABSTRACT FROM AUTHOR]

We present a novel approach for the compensation of temporal brightness variations (commonly referred to as flicker) in archived film sequences. The proposed method is motivated by fundamental principles of photographic image registration and provides a substantial level of adaptation to temporal but also spatial variations of picture brightness. Additionally, our scheme provides an efficient mechanism for the adaptive estimation of flicker compensation profile, which makes it suitable for the compensation of long duration film sequences while it addresses problems arising from scene motion and illumination using a novel motion-compensated graylevel tracing approach. We present experimental evidence which suggests that our method offers high levels of performance and compares favorably with competing state-of-the-art techniques for flicker compensation. [ABSTRACT FROM AUTHOR]

The characterization of a video segment by a digital signature is a fundamental task in video processing. It is necessary for video indexing and retrieval, copyright protection, and other tasks. Semantic video signatures are those that are based on high-level content information rather than on low-level features of the video stream. The major advantage of such signatures is that they are highly invariant to nearly all types of distortion. A major semantic feature of a video is the appearance of specific persons in specific video frames. Because of the great amount of research that has been performed on the subject of face detection and recognition, the extraction of such information is generally tractable, or will be in the near future. We have developed a method that uses the pre-extracted output of face detection and recognition to perform fast semantic query-by-example retrieval of video segments. We also give the results of the experimental evaluation of our method on a database of real video. One advantage of our approach is that the evaluation of similarity is convolution-based, and is thus resistant to perturbations in the signature and independent of the exact boundaries of the query segment. [ABSTRACT FROM AUTHOR]

One of the most challenging problems for image retrieval applications is to find the optimal mapping between high-level semantic concept and low-level features. Traditional approaches often assume that images with same semantic label share strong visual similarities and should be clustered together to facilitate modeling and classification. Our research indicates this assumption is inappropriate in many cases. Instead we model the images as lying on nonlinear image subspaces embedded in the high-dimensional feature space and find that multiple subspaces may correspond to one semantic concept. By intelligently utilizing the similarity and dissimilarity information in semantic and geometric (image) domains, we propose an optimal semantic subspace projection (SSP) that captures the most important properties of the subspaces with respect to classification. Theoretical analysis proves that the well-known linear discriminant analysis (LDA) could be formulated as a special case of SSP. To capture the semantic concept dynamically, SSP can integrate relevance feedback efficiently through incremental learning. Kernel SSP is further proposed to handle nonlinearly separable data. Extensive experiments have been designed and conducted to compare our proposed method to the state-of-the-art techniques such as LDA, locality preservation projection (LPP), local linear embedding (LLE), local discriminant embedding (LDE) and their variants. The results show the superior performance of SSP. [ABSTRACT FROM AUTHOR]

Image moments are used extensively in various applications. Depending on the task at hand, the type as well as the order of moments to be implemented varies greatly. Simple geometric characteristics require the first few geometrical moments, while robust image watermarking employs high-order orthogonal moments. In this paper, we present an architecture which can be easily modified to implement various types of moments, including geometric, continuous as well as discrete orthogonal, in real-time. Although this architecture is optimized for hardware, software implementation is feasible and advantageous in some cases. [ABSTRACT FROM AUTHOR]

Three-dimensional animating meshes have been widely used in the computer graphics and video game industries. Reducing the animating mesh complexity is a common way of overcoming the rendering limitation or network bandwidth. Thus, we present a compact representation for animating meshes based on novel key-frames extraction and animating mesh simplification approaches. In contrast to the general simplification and key-frames extraction approaches which are driven by geometry metrics, the proposed methods are based on a deformation analysis of animating mesh to preserve both the geometric features and motion characteristics. These two approaches can produce a very compact animation representation in spatial and temporal domains, and therefore they can be beneficial in many applications such as progressive animation transmission and animation segmentation and transferring. [ABSTRACT FROM AUTHOR]

Many existing techniques for image restoration can be expressed in terms of minimizing a particular cost function. Iterative regularization methods are a novel variation on this theme where the cost function is not fixed, but rather refined iteratively at each step. This provides an unprecedented degree of control over the tradeoff between the bias and variance of the image estimate, which can result in improved overall estimation error. This useful property, along with the provable convergence properties of the sequence of estimates produced by these iterative regularization methods lend themselves to a variety of useful applications. In this paper, we introduce a general set of iterative regularization methods, discuss some of their properties and applications, and include examples to illustrate them. [ABSTRACT FROM AUTHOR]

In this paper, we propose a tracking algorithm based on an adaptive multifeature statistical target model. The features are combined in a single particle filter by weighting their contributions using a novel reliability measure derived from the particle distribution in the state space. This measure estimates the reliability of the information by measuring the spatial uncertainty of features. A modified resampling strategy is also devised to account for the needs of the feature reliability estimation. We demonstrate the algorithm using color and orientation features. Color is described with partwise normalized histograms. Orientation is described with histograms of the gradient directions that represent the shape and the internal edges of a target. A feedback from the state estimation is used to align the orientation histograms as well as to adapt the scales of the filters to compute the gradient. Experimental results over a set of real-world sequences show that the proposed feature weighting procedure outperforms state-of-the-art solutions and that the proposed adaptive multifeature tracker improves the reliability of the target estimate while eliminating the need of manually selecting each feature's relevance. [ABSTRACT FROM AUTHOR]

The motion compensated temporal filtering (MCTF) technique, which is extensively used in 3-D wavelet video coding schemes nowadays, leads to signal coupling among various spatial subbands because motion alignment is introduced in the temporal filtering. Using all spatial subbands as a reference enables MCTF to fully take advantage of temporal correlation across frames but inevitably brings drifting problem in supporting spatial scalability. This paper first analyzes the signal coupling phenomenon and then proposes a quantitative model to describe signal propagation across spatial subbands during the MCTF process. The signal propagation is modeled for a single MC step based on the shifting effect of wavelet synthesis filters and then it is extended to multi- level MCTF. This model is called subband coupling aware signal propagation (SCASP) model in this paper. Based on the model, we further propose a subband coupling aware rate allocation scheme as one possible solution to the above dilemma in supporting spatial scalability. To find the optimal rate allocation among all subbands for a specified reconstruction resolution, the SCASP model is used to approximate the reconstruction process and derive the synthesis gain of each subband with regard to that reconstruction. Experimental results have fully demonstrated the advantages of our proposed rate allocation scheme in improving both objective and subjective qualities of reconstructed low-resolution video, especially at middle bit rates and high bit rates. [ABSTRACT FROM AUTHOR]

This work presents a new class of algorithms for super-resolution (SR) of image sequences. This class of algorithms estimates simultaneously all frames of a sequence by employing an iterative minimization of a regularized cost function. Similarly to other SR techniques, the proposed approach exploits the correlation among the frames of the sequence. This correlated information helps to improve the resolution of the captured images. By employing the motion information only in the prior term of the cost function, the proposed method achieves a better fidelity and more robust performance, when compared to other methods. This original approach provides a fidelity equivalent to the obtained by the simultaneous methods, while achieving a lower computational complexity. The isolation of the motion equations in the prior term allows an increased control over the motion errors, and as a consequence, provides an improved robustness. The performance of the proposed method is discussed and compared with other methods in the literature. In the comparative experiments, it is considered both the minimization over the Euclidean norm, which is employed to achieve low computational complexity, and the minimization over the Huber norm or ℓ1 norm, which is used to produce images with sharp edges and higher robustness to large motion errors. [ABSTRACT FROM AUTHOR]

This paper presents a novel B-picture coding based on block-wise adaptive motion accuracy (BAMA) with low-complexity motion compensation (MC). BAMA is able to adaptively select the motion accuracy for each inter-block in a B-picture depending on whether it is of bidirectional prediction, in which compared to the unidirectionally predicted block, lower motion accuracy is applied to the bidirectionally predicted block to reduce MC complexity. To further reduce MC complexity for the bidirectionally predicted luminance block with subpixel motion vectors in both directions, the forward and backward MC interpolations are merged according to the linear operation property of MC interpolation. In addition, a simplified MC interpolation method is also introduced for chrominance components for the bidirectionally predicted block. When integrating the proposed methods into H.264/AVC reference software, the experimental results demonstrate that they can significantly reduce MC complexity for a B-picture while yielding the comparable rate distortion performance in comparison with H.264/AVC although they no longer generate H.264/AVC compliant bitstream. [ABSTRACT FROM AUTHOR]

We present an efficient content-based image coding called Locally Adaptive Resolution (LAR) offering advanced scalability at different semantic levels, i.e., pixel, block, and region. A local analysis of image activity leads to a nonuniform block representation supporting two layers of image description. The first layer provides global information encoded in the spatial domain enabling a low bit rate while preserving contours. The second layer holds texture information encoded in the spectral domain, enabling scalable bitstream in accordance with the required quality. This basic LAR coding leads to an efficient progressive compression, evaluated through subjective quality tests. Its nonuniform block representation also allows a hierarchical region representation providing higher semantic functionalities. More precisely, the segmentation process can be simultaneously performed at both the coder and the decoder from only the luminance component highly compressed by the first coding layer. This solution provides a representation at a region level while avoiding any contour encoding overhead. Region enhancement can then be realized through the second layer. Furthermore, very high compression of the chromatic components is achieved thanks to this region representation. In this scheme, a low-cost chromatic control, which was first introduced during the segmentation process, increases the consistency of region representation in terms of color. [ABSTRACT FROM AUTHOR]

A novel CMOS active pixel sensor imager, called pyramidal readout architecture, is presented in which the 1-D row sampling of a classical imager is replaced by a 2-D ring sampling and diagonal output buses. The first characteristic resulted in enhancement of the readout speed of the imager, while the second characteristic is shown here to make fixed pattern noise (FPN) less perceptible to the human visual system (HVS). This observation exploits the relatively poor sensitivity of the HVS to obliquely distributed patterns, known as the oblique effect. The effect was modeled using a spatial filter derived from measured characteristics of the HVS, and applied to the obliquely distributed FPN-dominated images of the pyramidal CMOS imager. [ABSTRACT FROM AUTHOR]

Wavelet-based coding is emerging as a promising framework for efficient and scalable compression of video. Nevertheless, a number of basic tools currently employed in this field have been conceived for hybrid block-based transform coding. This is the case of motion estimation, which generally aims to minimize the energy or the absolute sum of prediction error. However, as wavelet video coders do not employ predictive coding, this is no longer an optimal approach. In this paper we study the problem of the theoretical optimal criterion for wavelet-based video coders, using coding gain as merit figure. A simple solution has been found for a peculiar but useful class of temporal filters. Experiments confirm that the optimally estimated vectors increase the coding gain as well as the performance of a complete video coder, but at the cost of an augmented complexity. [ABSTRACT FROM AUTHOR]

Disparity estimation belongs to the most important, but difficult, problems in image processing and computer vision. Its importance stems from a wide range of applications, while its difficulty is related to ill-posedness. To date, numerous disparity estimation algorithms have been developed. In this paper, we consider a particular case of disparity estimation based on two views and a known alpha channel partitioning each view into foreground and background. The main idea is to use this partitioning in order to enhance disparity estimation in the foreground object close to its boundary. We propose a block-based disparity model with two alpha-channel constraints: a photometric one, disabling invalid intensity/color matches, and a geometric one, preventing disparity smoothing between foreground and background. We incorporate these constraints into a Bayesian framework using the maximum a posteriori probability criterion. We experimentally demonstrate improvements in the estimated disparities at foreground object boundaries, and show examples of image relighting using these disparities. [ABSTRACT FROM AUTHOR]

Effectively coding multiview visual content is an indispensable research topic because multiview image and video that provide greatly enhanced viewing experiences often contain huge amounts of data. Generally, conventional hybrid predictive-coding methodologies are adopted to address the compression by exploiting the temporal and interviewpoint redundancy existing in a multiview image or video sequences. However, their key yet time-consuming component, motion estimation (ME), is usually not efficient in interviewpoint prediction or disparity estimation (DE), because interviewpoint disparity is completely different from temporal motion existing in the conventional video. Targeting a generic fast DE framework for interviewpoint prediction, we propose a novel DE technique in this paper to accelerate the disparity search by employing epipolar geometry. Theoretical analysis, optimal disparity vector distribution histograms, and experimental results show that the proposed epipolar geometry-based DE can greatly reduce search region and effectively track large and irregular disparity, which is typical in convergent multiview camera setups. Compared with the existing state-of-the-art fast ME approaches, our proposed DE can obtain a similar coding efficiency while achieving a significant speedup for interviewpoint prediction and coding. Moreover, a robustness study shows that the proposed DE algorithm is insensitive to the epipolar geometry estimation noise. Hence, its wide application for multiview image and video coding is promising. [ABSTRACT FROM AUTHOR]

Prominences are clouds of relatively cool and dense gas in the solar atmosphere. In this paper, we present a new method to detect and characterize the prominence eruptions. The input is a sequence of consecutive Hα solar images, and the output is a list of prominence eruption events detected. We extract the limb events and measure their associated properties by applying image processing techniques. First, we perform image normalization and noise removal. Then, we isolate the limb objects and identify the prominence features. Finally, we apply pattern recognition techniques to classify the eruptive prominences. The characteristics of prominence eruptions, such as brightness, angular width, radial height and velocity are measured. The method presented can lead to automatic monitoring and characterization of solar events. [ABSTRACT FROM AUTHOR]

A fundamental element of stereoscopic image production is to geometrically analyze the conversion from real space to stereoscopic images by binocular parallax under various shooting and viewing conditions. This paper reports on this analysis, particularly on the setting of the optical axes of three-dimensional (3-D) cameras, which has received little attention in the past. First, we identified the conditions for setting the optical axes that maintain linearity during the conversion from real space to stereoscopic images. We then clarified, in geometrical terms, the shooting and viewing conditions and also conditions under which the puppet-theater effect and cardboard effect occur. The results showed that the parallel camera configuration, by which optical axes are kept parallel to each other, does not produce the puppet-theater effect as the apparent magnification (lateral magnification) of a shooting target is not dependent on the shooting distance. However, the toed-in camera configuration, where the apparent magnification of a shooting target is dependent on the shooting distance, may produce this effect. The cardboard effect is shown to be likely to occur for both camera configurations by defining this phenomenon by the ratio of depthwise reproduction magnification (depth magnification) and apparent reproduction magnification (lateral magnification). Lastly, the paper reports on the relationship between the results of this analysis and those of subjective evaluation experiments. The results need a closer examination by using many more images. [ABSTRACT FROM AUTHOR]

In this paper, a novel architecture for two-dimensional (2-D) inverse discrete cosine transform (IDCT) is implemented using 90-nm CMOS technology. By exploiting the sparseness property of a 2-D discrete cosine transform (DCT) coefficient matrix and the even and odd symmetry properties of the basis vectors of the one-dimensional (1-D) DCT, the proposed architecture reduces computational complexity and increases a throughput rate effectively. First, we derive a recursion equation from the definition of the 2-D IDCT algorithm and use it to design an efficient 2-D IDCT architecture. The proposed architecture consisting of processing elements is suitable for very-large-scale-integration implementation due to its highly regular and scalable structure for 2-D N x N IDCT computations. Based on the derived recursion equation, we show how data flow for the outer products scaled by only nonzero 2-D DCT coefficients performs 2-D IDCT naturally with low complexities of the controller and the interconnection. The proposed architecture based on the recursion equation provides optimum balance in both hardware complexity and throughput rate when compared with other 2-D IDCT architectures. [ABSTRACT FROM AUTHOR]

In this paper, we investigate an adaptive channel error-protection scheme for streaming stored MPEG-4 fine granular scalability (FGS) bitstreams over error-prone environments. The human eye is sensitive to the variations in reconstructed image quality. Therefore, instead of using only the minimal average frame distortion as the optimization criterion, we propose a rate-distortion (R-D)-based bit-allocation method to determine the source coding rate and the channel coding rate under a given channel condition so as to minimize both the average image distortion and quality variation. Based on our proposed piecewise model of the MPEG-4 FGS enhancement layer, a balance between the average frame quality and quality variation will be obtained. Experimental results show that, compared with other existing methods, our proposed method can achieve less quality variation and comparable average frame distortion under various channel conditions. [ABSTRACT FROM AUTHOR]

In this paper, we propose a fast coarse-to-fine video retrieval scheme using shot-level spatio-temporal statistics. The scheme consists of a two-step coarse search followed by a fine search. In the coarse search stage, the shot-level motion and color distribution is computed as spatio-temporal features for shot matching. The first-step coarse search uses the shot-level global statistics to reduce the size of the search space drastically. By adding an adjacent shot of the first query shot, the second-step coarse search introduces a ‘causality’ relation between two consecutive shots to improve the search accuracy. Finally, the fine-search step refines the search result by using the local color features extracted from the key frames of the query shots. Our experimental results show that the proposed method achieves good retrieval performance with a much reduced complexity compared to single-pass methods. [ABSTRACT FROM AUTHOR]

In this paper, we present two new methods for efficient rate control and entropy coding in lossy image compression using JPEG-2000. These two methods enable significant improvements in computation complexity and power consumption over the traditional JPEG-2000 algorithms. First, we propose a greedy heap-based rate-control algorithm (GHRaC), which achieves efficient postcompression rate control by implementing a greedy marginal analysis method using the heap sort algorithm. Second, we propose an integrated rate-control and entropy-coding (IREC) algorithm that reduces the computation complexity of entropy coding by selectively entropy coding only the image data that is likely to be included in the final bitstream, as opposed to entropy coding all image data. Together, these two methods enable significant savings in computation time and power consumption. For example, the GHRaC method demonstrates 16x speedup for rate control when encoding the Lena color image using a target compression ratio of 128:1, one quality layer, and code blocks of 32 x 32 pixels. The IREC method expands upon GHRaC to perform entropy coding in conjunction with rate control. Using an enhanced version of IREC, these two methods jointly achieve a speedup in execution time of 14 x over traditional rate control and entropy coding, which first entropy codes all image coefficients and then separately performs postcompression rate control using the generalized Lagrange multiplier method to select which data are included in the final bitstream. Both theoretical analysis and empirical results are presented in validating the advantages of the proposed methods. [ABSTRACT FROM AUTHOR]

The performance of various wavelets, including those known to be well suited for the coding of still images, has been evaluated for the multiresolution motion estimation of video sequences. The multiresolution motion estimation scheme proposed by Zhang and Zafar, which has been widely cited in the literature, is used as the simulation scheme in this study. In our study, the prediction mean square error in the wavelet transform coefficient domain is used as the measure for prediction performance. In order to show the overall rate distortion performance, the number of bits needed to encode the motion vectors is also calculated. Simulation results show that the 7/9 biorthogonal wavelet, one of the best wavelets for the coding of still images, is the best wavelet for the task of multiresolution motion estimation among the wavelets evaluated in this study. [ABSTRACT FROM AUTHOR]

Content with rapid, moderate, and slow motion is frequently mixed together in real video sequences. Until now, no fast block-matching algorithm (FBMA), including the well-known three-step search (TSS), the block-based gradient descent search (BBGDS), and the diamond search (DS), can efficiently remove the temporal redundancy of sequences with wide range motion content. This paper proposes an adaptive FBMA, called A-TDB, to solve this problem. Based on the characteristics of a proposed predicted profit list, the A-TDB can adaptively switch search patterns among the TSS, DS, and BBGDS, according to the motion content. Experimental results reveal that the A-TDB successfully adopts the search patterns to remove the temporal redundancy of sequences with slow, moderate and rapid motion content. [ABSTRACT FROM AUTHOR]