Search

Video halftoning is a key technology for use in electronic paper (e-paper) or smart paper, which is an emerging display device that has received considerable attention recently. In this paper, a temporal frequency of flickering-distortion optimized video halftoning method is proposed. We first uncover three visual defects that conventional neighboring frame referencing-based video halftoning methods, due to their sequential changes of reference frames, will encounter. To deal with the problem, we then propose a reference frame update per GOP-based error diffusion video halftoning method based on a flickering sensitivity-based human visual model. To efficiently compromise between average temporal frequency of flickering (ATFoF) and visual quality, temporal frequency of flickering-distortion (TFoFD) is presented as a metric for video halftoning performance evaluation. Based on the proposed probability model of video halftoning, the TFoFD curve can be accurately estimated to optimize the tradeoff between quality and ATFoF before the video is halftoned. Our temporal frequency of flickering-distortion optimization strategy can also be applied to other video halftoning schemes for performance improvement. Experimental results and comparisons with known methods demonstrate the effectiveness of our video halftoning method. [ABSTRACT FROM PUBLISHER]

Tubular structure tracking is a crucial task in the fields of computer vision and medical image analysis. The minimal paths-based approaches have exhibited their strong ability in tracing tubular structures, by which a tubular structure can be naturally modeled as a minimal geodesic path computed with a suitable geodesic metric. However, existing minimal paths-based tracing approaches still suffer from difficulties such as the shortcuts and short branches combination problems, especially when dealing with the images involving complicated tubular tree structures or background. In this paper, we introduce a new minimal paths-based model for minimally interactive tubular structure centerline extraction in conjunction with a perceptual grouping scheme. Basically, we take into account the prescribed tubular trajectories and curvature-penalized geodesic paths to seek suitable shortest paths. The proposed approach can benefit from the local smoothness prior on tubular structures and the global optimality of the used graph-based path searching scheme. Experimental results on both synthetic and real images prove that the proposed model indeed obtains outperformance comparing with the state-of-the-art minimal paths-based tubular structure tracing algorithms. [ABSTRACT FROM AUTHOR]

How people look at visual information reveals fundamental information about themselves, their interests and their state of mind. While previous visual attention models output static 2D saliency maps, saccadic models aim to predict not only where observers look at but also how they move their eyes to explore the scene. In this paper, we demonstrate that saccadic models are a flexible framework that can be tailored to emulate observer’s viewing tendencies. More specifically, we use fixation data from 101 observers split into five age groups (adults, 8–10 y.o., 6–8 y.o., 4–6 y.o., and 2 y.o.) to train our saccadic model for different stages of the development of human visual system. We show that the joint distribution of saccade amplitude and orientation is a visual signature specific to each age group, and can be used to generate age-dependent scan paths. Our age-dependent saccadic model does not only output human-like, age-specific visual scan paths, but also significantly outperforms other state-of-the-art saliency models. We demonstrate that the computational modeling of visual attention, through the use of saccadic model, can be efficiently adapted to emulate the gaze behavior of a specific group of observers. [ABSTRACT FROM PUBLISHER]

This paper proposes a new method of calculating a matching metric for motion estimation. The proposed method splits the information in the source images into multiple scale and orientation subbands, reduces the subband values to a binary representation via an adaptive thresholding algorithm, and uses mutual information to model the similarity of corresponding square windows in each image. A moving window strategy is applied to recover a dense estimated motion field whose properties are explored. The proposed matching metric is a sum of mutual information scores across space, scale, and orientation. This facilitates the exploitation of information diversity in the source images. Experimental comparisons are performed amongst several related approaches, revealing that the proposed matching metric is better able to exploit information diversity, generating more accurate motion fields. [ABSTRACT FROM AUTHOR]

Image hazing aims to render a hazy image from a given clean one, which could be applied to a variety of practical applications such as gaming, filming, photographic filtering, and image dehazing. To generate plausible haze, we study two less-touched but challenging problems in hazy image rendering, namely, i) how to estimate the transmission map from a single image without auxiliary information, and ii) how to adaptively learn the airlight from exemplars, i.e., unpaired real hazy images. To this end, we propose a neural rendering method for image hazing, dubbed as HazeGEN. To be specific, HazeGEN is a knowledge-driven neural network which estimates the transmission map by leveraging a new prior, i.e., there exists the structure similarity (e.g., contour and luminance) between the transmission map and the input clean image. To adaptively learn the airlight, we build a neural module based on another new prior, i.e., the rendered hazy image and the exemplar are similar in the airlight distribution. To the best of our knowledge, this could be the first attempt to deeply render hazy images in an unsupervised fashion. Compared with existing haze generation methods, HazeGEN renders the hazy images in an unsupervised, learnable, and controllable manner, thus avoiding the labor-intensive efforts in paired data collection and the domain-shift issue in haze generation. Extensive experiments show the promising performance of our method comparing with some baselines in both qualitative and quantitative comparisons. The code is available at https://github.com/XLearning-SCU. [ABSTRACT FROM AUTHOR]

In this work, we propose a non-linear hyperbolic-parabolic coupled Partial Differential Equation (PDE) based model for image despeckling. Here, a separate equation is used to calculate the edge variable, which improves the quality of edge information in the despeckled images. The existence of the weak solution of the present system is achieved via Schauder fixed point theorem. We used a generalized weighted average finite-difference scheme and the Gauss-Seidel iterative technique to solve the coupled system. Numerical studies are reported to show the effectiveness of the proposed approach with respect to standard PDE-based and nonlocal methods available in the literature. Numerical experiments are performed over gray-level images degraded by artificial speckle noise. Additionally, we investigate the noise removal efficiency of the proposed algorithm when applied to real synthetic aperture radar (SAR) and Ultrasound images. Overall, our study confirms that in most cases, the present model performs better than the other PDE-based models and shows competitive performance with the nonlocal technique. To the best of our knowledge, the proposed despeckling approach is the first work that utilizes the advantage of the non-linear coupled hyperbolic-parabolic PDEs for image despeckling. [ABSTRACT FROM AUTHOR]

Level set methods are widely used for image segmentation because of their convenient shape representation for numerical computations and capability to handle topological changes. However, in spite of the numerous works in the literature, the use of level set methods in image segmentation still has several drawbacks. These shortcomings include formation of irregularities of the signed distance function, sensitivity to initialization, lack of locality, and expensive computational cost, which increases dramatically as the number of objects to be simultaneously segmented grows. In this paper, we propose a novel parametric level set method called disjunctive normal level set (DNLS), and apply it to both two-phase (single object) and multiphase (multiobject) image segmentations. DNLS is a differentiable model formed by the union of polytopes, which themselves are created by intersections of half-spaces. We formulate the segmentation algorithm in a Bayesian framework and use a variational approach to minimize the energy with respect to the parameters of the model. The proposed DNLS can be considered as an open framework that allows the use of different appearance models and shape priors. Compared with the conventional level sets available in the literature, the proposed DNLS has the following major advantages: it requires significantly less computational time and memory, it naturally keeps the level set function regular during the evolution, it is more suitable for multiphase and local region-based image segmentations, and it is less sensitive to noise and initialization. The experimental results show the potential of the proposed method. [ABSTRACT FROM AUTHOR]

Scene background initialization is the process by which a method tries to recover the background image of a video without foreground objects in it. Having a clear understanding about which approach is more robust and/or more suited to a given scenario is of great interest to many end users or practitioners. The aim of this paper is to provide an extensive survey of scene background initialization methods as well as a novel benchmarking framework. The proposed framework involves several evaluation metrics and state-of-the-art methods, as well as the largest video data set ever made for this purpose. The data set consists of several camera-captured videos that: 1) span categories focused on various background initialization challenges; 2) are obtained with different cameras of different lengths, frame rates, spatial resolutions, lighting conditions, and levels of compression; and 3) contain indoor and outdoor scenes. The wide variety of our data set prevents our analysis from favoring a certain family of background initialization methods over others. Our evaluation framework allows us to quantitatively identify solved and unsolved issues related to scene background initialization. We also identify scenarios for which state-of-the-art methods systematically fail. [ABSTRACT FROM PUBLISHER]

We introduce a novel edge tracing algorithm using Gaussian process regression. Our edge-based segmentation algorithm models an edge of interest using Gaussian process regression and iteratively searches the image for edge pixels in a recursive Bayesian scheme. This procedure combines local edge information from the image gradient and global structural information from posterior curves, sampled from the model’s posterior predictive distribution, to sequentially build and refine an observation set of edge pixels. This accumulation of pixels converges the distribution to the edge of interest. Hyperparameters can be tuned by the user at initialisation and optimised given the refined observation set. This tunable approach does not require any prior training and is not restricted to any particular type of imaging domain. Due to the model’s uncertainty quantification, the algorithm is robust to artefacts and occlusions which degrade the quality and continuity of edges in images. Our approach also has the ability to efficiently trace edges in image sequences by using previous-image edge traces as a priori information for consecutive images. Various applications to medical imaging and satellite imaging are used to validate the technique and comparisons are made with two commonly used edge tracing algorithms. [ABSTRACT FROM AUTHOR]

Face images convey rich information which can be perceived as a superposition of low-complexity components associated with attributes, such as facial identity, expressions, and activation of facial action units (AUs). For instance, low-rank components characterizing neutral facial images are associated with identity, while sparse components capturing non-rigid deformations occurring in certain face regions reveal expressions and AU activations. In this paper, the discriminant incoherent component analysis (DICA) is proposed in order to extract low-complexity components, corresponding to facial attributes, which are mutually incoherent among different classes (e.g., identity, expression, and AU activation) from training data, even in the presence of gross sparse errors. To this end, a suitable optimization problem, involving the minimization of nuclear-and \ell 1 -norm, is solved. Having found an ensemble of class-specific incoherent components by the DICA, an unseen (test) image is expressed as a group-sparse linear combination of these components, where the non-zero coefficients reveal the class(es) of the respective facial attribute(s) that it belongs to. The performance of the DICA is experimentally assessed on both synthetic and real-world data. Emphasis is placed on face analysis tasks, namely, joint face and expression recognition, face recognition under varying percentages of training data corruption, subject-independent expression recognition, and AU detection by conducting experiments on four data sets. The proposed method outperforms all the methods that are compared with all the tasks and experimental settings. [ABSTRACT FROM AUTHOR]

In this paper, we propose an extremely efficient algorithm for visual re-ranking. By considering the original pairwise distance in the contextual space, we develop a feature vector called sparse contextual activation (SCA) that encodes the local distribution of an image. Hence, re-ranking task can be simply accomplished by vector comparison under the generalized Jaccard metric, which has its theoretical meaning in the fuzzy set theory. In order to improve the time efficiency of re-ranking procedure, inverted index is successfully introduced to speed up the computation of generalized Jaccard metric. As a result, the average time cost of re-ranking for a certain query can be controlled within 1 ms. Furthermore, inspired by query expansion, we also develop an additional method called local consistency enhancement on the proposed SCA to improve the retrieval performance in an unsupervised manner. On the other hand, the retrieval performance using a single feature may not be satisfactory enough, which inspires us to fuse multiple complementary features for accurate retrieval. Based on SCA, a robust feature fusion algorithm is exploited that also preserves the characteristic of high time efficiency. We assess our proposed method in various visual re-ranking tasks. Experimental results on Princeton shape benchmark (3D object), WM-SRHEC07 (3D competition), YAEL data set B (face), MPEG-7 data set (shape), and Ukbench data set (image) manifest the effectiveness and efficiency of SCA. [ABSTRACT FROM AUTHOR]

This paper presents a simple distance estimation for implicit polynomial fitting. It is computed as the height of a simplex built between the point and the surface (i.e., a triangle in 2-D or a tetrahedron in 3-D), which is used as a coarse but reliable estimation of the orthogonal distance. The proposed distance can be described as a function of the coefficients of the implicit polynomial. Moreover, it is differentiable and has a smooth behavior . Hence, it can be used in any gradient-based optimization. In this paper, its use in a Levenberg–Marquardt framework is shown, which is particularly devoted for nonlinear least squares problems. The proposed estimation is a generalization of the gradient-based distance estimation, which is widely used in the literature. Experimental results, both in 2-D and 3-D data sets, are provided. Comparisons with state-of-the-art techniques are presented, showing the advantages of the proposed approach. [ABSTRACT FROM AUTHOR]

In this paper, an original observation model for multiresolution optical flow estimation is introduced. Multiresolution frameworks, often based on coarse-to-fine warping strategies, are widely used by state-of-the-art optical flow methods. They allow the recovery of large motions by successive estimations of the flow field at several resolution levels. Although such approaches perform very efficiently and usually lead to faster minimizations, they generally consider independent problems at each resolution levels and do not exploit the existing interactions between scales (especially the influences of fine scales on larger ones). In this paper, we tackle this issue by proposing a flexible framework, inspired from fluid mechanics, able to partly counter these limitations. For each resolution level, our process filters the equations of interest and decomposes the key variables into resolved (i.e., at a given resolution) and unresolved (i.e., at finer resolutions) components. This enables to derive a new data term that takes into account, at coarse resolutions, the influence of their unresolved parts. From this new term, we propose two different estimation strategies, depending on whether we explicitly know the type of relations between the different scales (as for physical processes) or not. In order to test the efficiency of this new observation model, we have embedded it in a simple multiresolution Lucas-Kanade estimator. Comparing the usual optical flow constraint equation with this new term in the same motion estimation procedure, it clearly appears that the proposed term leads to more consistent estimates and prevents from errors propagation apparition during the estimation. In all situations (synthetic, real, physical images or not), our new term is able to greatly improve the results compared with usual conservation constraints. [ABSTRACT FROM PUBLISHER]

We introduce a novel edge tracing algorithm using Gaussian process regression. Our edge-based segmentation algorithm models an edge of interest using Gaussian process regression and iteratively searches the image for edge pixels in a recursive Bayesian scheme. This procedure combines local edge information from the image gradient and global structural information from posterior curves, sampled from the model’s posterior predictive distribution, to sequentially build and refine an observation set of edge pixels. This accumulation of pixels converges the distribution to the edge of interest. Hyperparameters can be tuned by the user at initialisation and optimised given the refined observation set. This tunable approach does not require any prior training and is not restricted to any particular type of imaging domain. Due to the model’s uncertainty quantification, the algorithm is robust to artefacts and occlusions which degrade the quality and continuity of edges in images. Our approach also has the ability to efficiently trace edges in image sequences by using previous-image edge traces as a priori information for consecutive images. Various applications to medical imaging and satellite imaging are used to validate the technique and comparisons are made with two commonly used edge tracing algorithms. [ABSTRACT FROM AUTHOR]

It is known that local filtering-based edge-preserving smoothing techniques suffer from halo artifacts. In this paper, a weighted guided image filter (WGIF) is introduced by incorporating an edge-aware weighting into an existing guided image filter (GIF) to address the problem. The WGIF inherits advantages of both global and local smoothing filters in the sense that: 1) the complexity of the WGIF is $O(N)$ for an image with $N$ pixels, which is same as the GIF and 2) the WGIF can avoid halo artifacts like the existing global smoothing filters. The WGIF is applied for single image detail enhancement, single image haze removal, and fusion of differently exposed images. Experimental results show that the resultant algorithms produce images with better visual quality and at the same time halo artifacts can be reduced/avoided from appearing in the final images with negligible increment on running times. [ABSTRACT FROM AUTHOR]

In this paper, we extend the gradient vector flow field for robust variational segmentation of vector-valued images. Rather than using scalar edge information, we define a vectorial edge map derived from a weighted local structure tensor of the image that enables the diffusion of the gradient vectors in accurate directions through the 4D gradient vector flow equation. To reduce the contribution of noise in the structure tensor, image channels are weighted according to a blind estimator of contrast. The method is applied to biological volume delineation in dynamic PET imaging, and validated on realistic Monte Carlo simulations of numerical phantoms as well as on real images. [ABSTRACT FROM AUTHOR]

Fast and accurate motion estimation takes an important place in many fields of computer vision and image processing. Using Radon transform to compute projections of the images along specified directions is an effective way to show the relationship between the 2D image object and its projections and estimate motions between the images. All existing projection-based motion estimation methods without the use of iteration have a severe defect that only five of the six affine parameters can be estimated. There are some other methods that can estimate the six parameters, but most of them are usually based on a certain iterative framework, which is computationally intensive and sensitively dependent on the initial values. In this paper, a novel method based on Radon transform is proposed to estimate all the six affine parameters directly. The relationship in the projection domain between a pair of images connected by an affine motion is studied and a linear model is established, by which all the six affine parameters can be directively found. The employment of a hierarchical framework can produce more accurate results. The experimental results reveal that the proposed method has a much better performance than the state-of-the-art methods in this field. [ABSTRACT FROM AUTHOR]

Many computational models of visual attention performing well in predicting salient areas of 2D images have been proposed in the literature. The emerging applications of stereoscopic 3D display bring an additional depth of information affecting the human viewing behavior, and require extensions of the efforts made in 2D visual modeling. In this paper, we propose a new computational model of visual attention for stereoscopic 3D still images. Apart from detecting salient areas based on 2D visual features, the proposed model takes depth as an additional visual dimension. The measure of depth saliency is derived from the eye movement data obtained from an eye-tracking experiment using synthetic stimuli. Two different ways of integrating depth information in the modeling of 3D visual attention are then proposed and examined. For the performance evaluation of 3D visual attention models, we have created an eye-tracking database, which contains stereoscopic images of natural content and is publicly available, along with this paper. The proposed model gives a good performance, compared to that of state-of-the-art 2D models on 2D images. The results also suggest that a better performance is obtained when depth information is taken into account through the creation of a depth saliency map, rather than when it is integrated by a weighting method. [ABSTRACT FROM AUTHOR]

This paper studies a nonlinear mixing model for hyperspectral image unmixing and nonlinearity detection. The proposed model assumes that the pixel reflectances are nonlinear functions of pure spectral components contaminated by an additive white Gaussian noise. These nonlinear functions are approximated by polynomials leading to a polynomial post-nonlinear mixing model. We have shown in a previous paper that the parameters involved in the resulting model can be estimated using least squares methods. A generalized likelihood ratio test based on the estimator of the nonlinearity parameter is proposed to decide whether a pixel of the image results from the commonly used linear mixing model or from a more general nonlinear mixing model. To compute the test statistic associated with the nonlinearity detection, we propose to approximate the variance of the estimated nonlinearity parameter by its constrained Cramér–Rao bound. The performance of the detection strategy is evaluated via simulations conducted on synthetic and real data. More precisely, synthetic data have been generated according to the standard linear mixing model and three nonlinear models from the literature. The real data investigated in this study are extracted from the Cuprite image, which shows that some minerals seem to be nonlinearly mixed in this image. Finally, it is interesting to note that the estimated abundance maps obtained with the post-nonlinear mixing model are in good agreement with results obtained in previous studies. [ABSTRACT FROM PUBLISHER]

Hyperspectral data processing typically demands enormous computational resources in terms of storage, computation, and input/output throughputs, particularly when real-time processing is desired. In this paper, a proof-of-concept study is conducted on compressive sensing (CS) and unmixing for hyperspectral imaging. Specifically, we investigate a low-complexity scheme for hyperspectral data compression and reconstruction. In this scheme, compressed hyperspectral data are acquired directly by a device similar to the single-pixel camera based on the principle of CS. To decode the compressed data, we propose a numerical procedure to compute directly the unmixed abundance fractions of given endmembers, completely bypassing high-complexity tasks involving the hyperspectral data cube itself. The reconstruction model is to minimize the total variation of the abundance fractions subject to a preprocessed fidelity equation with a significantly reduced size and other side constraints. An augmented Lagrangian-type algorithm is developed to solve this model. We conduct extensive numerical experiments to demonstrate the feasibility and efficiency of the proposed approach, using both synthetic data and hardware-measured data. Experimental and computational evidences obtained from this paper indicate that the proposed scheme has a high potential in real-world applications. [ABSTRACT FROM AUTHOR]

In this paper, we present a postprocessing method to tackle the single-image refocusing-and-defocusing problem. The proposed method can accomplish the tasks of focus-map estimation and image refocusing and defocusing. Given an image with a mixture of focused and defocused objects, we first detect the edges and then estimate the focus map based on the edge blurriness, which is depicted explicitly by a parametric model. The image refocusing problem is addressed in a blind deconvolution framework, where the image prior is modeled by using both global and local constraints. In particular, we correct the defocused blurry edges to sharp ones with the aid of the parametric edge model and then render this cue as a local prior to ensure the sharpness of the refocused image. Experimental results demonstrate that the proposed method performs well in producing visually plausible images with different focus effects from a single input. [ABSTRACT FROM AUTHOR]

This paper proposes a spectral-graph-based algorithm for face image repairing, which can improve the recognition performance on occluded faces. The face completion algorithm proposed in this paper includes three main procedures: 1) sparse representation for partially occluded face classification; 2) image-based data mining; and 3) graph Laplace (GL) for face image completion. The novel part of the proposed framework is GL, as named from graphical models and the Laplace equation, and can achieve a high-quality repairing of damaged or occluded faces. The relationship between the GL and the traditional Poisson equation is proven. We apply our face repairing algorithm to produce completed faces, and use face recognition to evaluate the performance of the algorithm. Experimental results verify the effectiveness of the GL method for occluded face completion. [ABSTRACT FROM PUBLISHER]

Liquid crystal display (LCD) devices are well known for their slow responses due to the physical limitations of liquid crystals. Therefore, fast moving objects in a scene are often perceived as blurred. This effect is known as the LCD motion blur. In order to reduce LCD motion blur, an accurate LCD model and an efficient deblurring algorithm are needed. However, existing LCD motion blur models are insufficient to reflect the limitation of human-eye-tracking system. Also, the spatiotemporal equivalence in LCD motion blur models has not been proven directly in the discrete 2-D spatial domain, although it is widely used. There are three main contributions of this paper: modeling, analysis, and algorithm. First, a comprehensive LCD motion blur model is presented, in which human-eye-tracking limits are taken into consideration. Second, a complete analysis of spatiotemporal equivalence is provided and verified using real video sequences. Third, an LCD motion blur reduction algorithm is proposed. The proposed algorithm solves an l_1-norm regularized least-squares minimization problem using a subgradient projection method. Numerical results show that the proposed algorithm gives higher peak SNR, lower temporal error, and lower spatial error than motion-compensated inverse filtering and Lucy–Richardson deconvolution algorithm, which are two state-of-the-art LCD deblurring algorithms. [ABSTRACT FROM PUBLISHER]

This paper describes a new, physically interpretable, fully automatic algorithm for removal of tissue autofluorescence (AF) from fluorescence microscopy images, by non-negative matrix factorization. Measurement of signal intensities from the concentration of certain fluorescent reporter molecules at each location within a sample of biological tissue is confounded by fluorescence produced by the tissue itself (autofluorescence). Spectral mixing models use mixing coefficients to specify how much fluorescence from each source is present and unmixing algorithms separate the two fluorescent sources. Current spectral unmixing methods for AF removal often require a priori knowledge of mixing coefficients. Those which do not, such as principal component analysis, generate negative mixing coefficients that are not physically meaningful. Non-negative matrix factorization constrains mixing coefficients to be non-negative, and has been used for spectral unmixing, but not AF removal. This paper describes a novel non-negative matrix factorization algorithm which separates fluorescent images into true signal and AF components utilizing an estimate of the dark current. We also present a test-bed, based on fluorescent beads, to compare the performance of different AF removal algorithms. Our algorithm out-performed previous state of the art on validation images. [ABSTRACT FROM PUBLISHER]

A missing intensity interpolation method using a kernel principal component analysis (PCA)-based projection onto convex sets (POCS) algorithm and its applications are presented in this paper. In order to interpolate missing intensities within a target image, the proposed method reconstructs local textures containing the missing pixels by using the POCS algorithm. In this reconstruction process, a nonlinear eigenspace is constructed from each kind of texture, and the optimal subspace for the target local texture is introduced into the constraint of the POCS algorithm. In the proposed method, the optimal subspace can be selected by monitoring errors converged in the reconstruction process. This approach provides a solution to the problem in conventional methods of not being able to effectively perform adaptive reconstruction of the target textures due to missing intensities, and successful interpolation of the missing intensities by the proposed method can be realized. Furthermore, since our method can restore any images including arbitrary-shaped missing areas, its potential in two image reconstruction tasks, image enlargement and missing area restoration, is also shown in this paper. [ABSTRACT FROM AUTHOR]

This paper addresses object tracking in ultrasound images using a robust multiple model tracker. The proposed tracker has the following features: 1) it uses multiple dynamic models to track the evolution of the object boundary, and 2) it models invalid observations (outliers), reducing their influence on the shape estimates. The problem considered in this paper is the tracking of the left ventricle which is known to be a challenging problem. The heart motion presents two phases (diastole and systole) with different dynamics, the multiple models used in this tracker try to solve this difficulty. In addition, ultrasound images are corrupted by strong multiplicative noise which prevents the use of standard deformable models. Robust estimation techniques are used to address this difficulty. The multiple model data association (MMDA) tracker proposed in this paper is based on a bank of nonlinear filters, organized in a tree structure. The algorithm determines which model is active at each instant of time and updates its state by propagating the probability distribution, using robust estimation techniques. [ABSTRACT FROM AUTHOR]

Cubic convolution is a popular method for image interpolation. Traditionally, the piecewise-cubic kernel has been derived in one dimension with one parameter and applied to two-dimensional (2-D) images in a separable fashion. However, images typically are statistically nonseparable, which motivates this investigation of nonseparable cubic convolution. This paper derives two new nonseparable, 2-D cubic-convolution kernels. The first kernel, with three parameters (designated 2D-3PCC), is the most general 2-D, piecewise-cubic interpolator defined on [-2, 2] × [-2, 2] with constraints for biaxial symmetry, diagonal (or 90° rotational) symmetry, continuity, and smoothness. The second kernel, with five parameters (designated 2D-SPCC), relaxes the constraint of diagonal symmetry, based on the observation that many images have rotationally asymmetric statistical properties. This paper also develops a closed-form solution for determining the optimal parameter values for parametric cubic-convolution kernels with respect to ensembles of scenes characterized by autocorrelation (or power spectrum). This solution establishes a practical foundation for adaptive interpolation based on local autocorrelation estimates. Quantitative fidelity analyses and visual experiments indicate that these new methods can outperform several popular interpolation methods. An analysis of the error budgets for reconstruction error associated with blurring and aliasing illustrates that the methods improve interpolation fidelity for images with aliased components. For images with little or no aliasing, the methods yield results similar to other popular methods. Both 2D-3PCC and 2D-5PCC are low-order polynomials with small spatial support and so are easy to implement and efficient to apply. [ABSTRACT FROM AUTHOR]

Recently, a tensor nuclear norm (TNN) based method was proposed to solve the tensor completion problem, which has achieved state-of-the-art performance on image and video inpainting tasks. However, it requires computing tensor singular value decomposition (t-SVD), which costs much computation and thus cannot efficiently handle tensor data, due to its natural large scale. Motivated by TNN, we propose a novel low-rank tensor factorization method for efficiently solving the 3-way tensor completion problem. Our method preserves the low-rank structure of a tensor by factorizing it into the product of two tensors of smaller sizes. In the optimization process, our method only needs to update two smaller tensors, which can be more efficiently conducted than computing t-SVD. Furthermore, we prove that the proposed alternating minimization algorithm can converge to a Karush–Kuhn–Tucker point. Experimental results on the synthetic data recovery, image and video inpainting tasks clearly demonstrate the superior performance and efficiency of our developed method over state-of-the-arts including the TNN and matricization methods. [ABSTRACT FROM PUBLISHER]

We propose multi-task and multivariate methods for multi-modal recognition based on low-rank and joint sparse representations. Our formulations can be viewed as generalized versions of multivariate low-rank and sparse regression, where sparse and low-rank representations across all modalities are imposed. One of our methods simultaneously couples information within different modalities by enforcing the common low-rank and joint sparse constraints among multi-modal observations. We also modify our formulations by including an occlusion term that is assumed to be sparse. The alternating direction method of multipliers is proposed to efficiently solve the resulting optimization problems. Extensive experiments on three publicly available multi-modal biometrics and object recognition data sets show that our methods compare favorably with other feature-level fusion methods. [ABSTRACT FROM PUBLISHER]

The tracking and recognition of facial activities from images or videos have attracted great attention in computer vision field. Facial activities are characterized by three levels. First, in the bottom level, facial feature points around each facial component, i.e., eyebrow, mouth, etc., capture the detailed face shape information. Second, in the middle level, facial action units, defined in the facial action coding system, represent the contraction of a specific set of facial muscles, i.e., lid tightener, eyebrow raiser, etc. Finally, in the top level, six prototypical facial expressions represent the global facial muscle movement and are commonly used to describe the human emotion states. In contrast to the mainstream approaches, which usually only focus on one or two levels of facial activities, and track (or recognize) them separately, this paper introduces a unified probabilistic framework based on the dynamic Bayesian network to simultaneously and coherently represent the facial evolvement in different levels, their interactions and their observations. Advanced machine learning methods are introduced to learn the model based on both training data and subjective prior knowledge. Given the model and the measurements of facial motions, all three levels of facial activities are simultaneously recognized through a probabilistic inference. Extensive experiments are performed to illustrate the feasibility and effectiveness of the proposed model on all three level facial activities. [ABSTRACT FROM PUBLISHER]

In view-based 3D object retrieval and recognition, each object is described by multiple views. A central problem is how to estimate the distance between two objects. Most conventional methods integrate the distances of view pairs across two objects as an estimation of their distance. In this paper, we propose a discriminative probabilistic object modeling approach. It builds probabilistic models for each object based on the distribution of its views, and the distance between two objects is defined as the upper bound of the Kullback–Leibler divergence of the corresponding probabilistic models. 3D object retrieval and recognition is accomplished based on the distance measures. We first learn models for each object by the adaptation from a set of global models with a maximum likelihood principle. A further adaption step is then performed to enhance the discriminative ability of the models. We conduct experiments on the ETH 3D object dataset, the National Taiwan University 3D model dataset, and the Princeton Shape Benchmark. We compare our approach with different methods, and experimental results demonstrate the superiority of our approach. [ABSTRACT FROM PUBLISHER]

Sparse representation is proven to be a promising approach to image super-resolution, where the low-resolution (LR) image is usually modeled as the down-sampled version of its high-resolution (HR) counterpart after blurring. When the blurring kernel is the Dirac delta function, i.e., the LR image is directly down-sampled from its HR counterpart without blurring, the super-resolution problem becomes an image interpolation problem. In such cases, however, the conventional sparse representation models (SRM) become less effective, because the data fidelity term fails to constrain the image local structures. In natural images, fortunately, many nonlocal similar patches to a given patch could provide nonlocal constraint to the local structure. In this paper, we incorporate the image nonlocal self-similarity into SRM for image interpolation. More specifically, a nonlocal autoregressive model (NARM) is proposed and taken as the data fidelity term in SRM. We show that the NARM-induced sampling matrix is less coherent with the representation dictionary, and consequently makes SRM more effective for image interpolation. Our extensive experimental results demonstrate that the proposed NARM-based image interpolation method can effectively reconstruct the edge structures and suppress the jaggy/ringing artifacts, achieving the best image interpolation results so far in terms of PSNR as well as perceptual quality metrics such as SSIM and FSIM. [ABSTRACT FROM PUBLISHER]

Determining the intrinsic dimension of a hyperspectral image is an important step in the spectral unmixing process and under- or overestimation of this number may lead to incorrect unmixing in unsupervised methods. In this paper, we discuss a new method for determining the intrinsic dimension using recent advances in random matrix theory. This method is entirely unsupervised, free from any user-determined parameters and allows spectrally correlated noise in the data. Robustness tests are run on synthetic data, to determine how the results were affected by noise levels, noise variability, noise approximation, and spectral characteristics of the endmembers. Success rates are determined for many different synthetic images, and the method is tested on two pairs of real images, namely a Cuprite scene taken from Airborne Visible InfraRed Imaging Spectrometer (AVIRIS) and SpecTIR sensors, and a Lunar Lakes scene taken from AVIRIS and Hyperion, with good results. [ABSTRACT FROM PUBLISHER]

This paper proposes an efficient method to modify histograms and enhance contrast in digital images. Enhancement plays a significant role in digital image processing, computer vision, and pattern recognition. We present an automatic transformation technique that improves the brightness of dimmed images via the gamma correction and probability distribution of luminance pixels. To enhance video, the proposed image-enhancement method uses temporal information regarding the differences between each frame to reduce computational complexity. Experimental results demonstrate that the proposed method produces enhanced images of comparable or higher quality than those produced using previous state-of-the-art methods. [ABSTRACT FROM AUTHOR]

When a catadioptric imaging system suffers from limited depth of field, a single image cannot capture all the objects with clear focus. To solve this problem, a set of multifocal images can be used to extend the depth of the field by fusing the best-focused image regions to an overall well-focused composite image. In this paper, we propose a novel model-based method that avoids the computational cost problem of previous extended-depth-of-field algorithms. On the basis of the special optical geometry properties of catadioptric systems, the proposed model describes the shapes of the best-focused image regions in multifocal images by a series of neighboring concentric annuluses. Then we propose a method to estimate the model parameters. On the basis of this model, an overall well-focused image is obtained by combining the best-focused regions with a fast and reliable online operation. Experiments on catadioptric images of a variety of different scenes and camera settings verify the validity of the model and the robust performance of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, an algorithm is proposed to extract two illuminant invariant chromaticity features from three image sensor responses. The algorithm extracts these chromaticity features at pixel level and therefore can perform well in scenes illuminated with nonuniform illuminant. An approach is proposed to use the algorithm with cameras of unknown sensitivity. The algorithm was tested for separability of perceptually similar colors under the International Commission on Illumination standard illuminants and obtained a good performance. It was also tested for color-based object recognition by illuminating objects with typical indoor illuminants and obtained a better performance compared to other existing algorithms investigated in this paper. Finally, the algorithm was tested for skin detection invariant to illuminant, ethnic background and imaging device. In this investigation, daylight scenes under different weather conditions and scenes illuminated by typical indoor illuminants were used. The proposed algorithm gives a better skin detection performance compared to widely used standard color spaces. Based on the results presented, the proposed illuminant invariant chromaticity space can be used for machine vision applications including illuminant invariant color-based object recognition and skin detection. [ABSTRACT FROM AUTHOR]

The goal of no-reference objective image quality assessment (NR-IQA) is to develop a computational model that can predict the human-perceived quality of distorted images accurately and automatically without any prior knowledge of reference images. Most existing NR-IQA approaches are distortion specific and are typically limited to one or two specific types of distortions. In most practical applications, however, information about the distortion type is not really available. In this paper, we propose a general-purpose NR-IQA approach based on visual codebooks. A visual codebook consisting of Gabor-filter-based local features extracted from local image patches is used to capture complex statistics of a natural image. The codebook encodes statistics by quantizing the feature space and accumulating histograms of patch appearances. This method does not assume any specific types of distortions; however, when evaluating images with a particular type of distortion, it does require examples with the same or similar distortion for training. Experimental results demonstrate that the predicted quality score using our method is consistent with human-perceived image quality. The proposed method is comparable to state-of-the-art general-purpose NR-IQA methods and outperforms the full-reference image quality metrics, peak signal-to-noise ratio and structural similarity index on the Laboratory for Image and Video Engineering IQA database. [ABSTRACT FROM PUBLISHER]

Recent years have witnessed a surge of interest in graph-based transductive image classification. Existing simple graph-based transductive learning methods only model the pairwise relationship of images, however, and they are sensitive to the radius parameter used in similarity calculation. Hypergraph learning has been investigated to solve both difficulties. It models the high-order relationship of samples by using a hyperedge to link multiple samples. Nevertheless, the existing hypergraph learning methods face two problems, i.e., how to generate hyperedges and how to handle a large set of hyperedges. This paper proposes an adaptive hypergraph learning method for transductive image classification. In our method, we generate hyperedges by linking images and their nearest neighbors. By varying the size of the neighborhood, we are able to generate a set of hyperedges for each image and its visual neighbors. Our method simultaneously learns the labels of unlabeled images and the weights of hyperedges. In this way, we can automatically modulate the effects of different hyperedges. Thorough empirical studies show the effectiveness of our approach when compared with representative baselines. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a novel tracking algorithm, i.e., the discriminative ranking list-based tracker (DRLTracker). The DRLTracker models the target object and its local background by using ranking lists of patches of different scales within object bounding boxes. The ranking list of each of such patches is its K nearest neighbors. Patches of the same scale with ranking lists of high purity values (meaning high probabilities to be on the target object) and some confusable background patches constitute the object model under that scale. A pair of object models of two different scales collaborate to determine which patches may belong to the target object in the next frame. The DRLTracker can effectively alleviate the distraction problem, and its superior ability over several representative and state-of-the-art trackers is demonstrated through extensive experiments. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a novel method based on a strategic combination of the active appearance model (AAM), live wire (LW), and graph cuts (GCs) for abdominal 3-D organ segmentation. The proposed method consists of three main parts: model building, object recognition, and delineation. In the model building part, we construct the AAM and train the LW cost function and GC parameters. In the recognition part, a novel algorithm is proposed for improving the conventional AAM matching method, which effectively combines the AAM and LW methods, resulting in the oriented AAM (OAAM). A multiobject strategy is utilized to help in object initialization. We employ a pseudo-3-D initialization strategy and segment the organs slice by slice via a multiobject OAAM method. For the object delineation part, a 3-D shape-constrained GC method is proposed. The object shape generated from the initialization step is integrated into the GC cost computation, and an iterative GC–OAAM method is used for object delineation. The proposed method was tested in segmenting the liver, kidneys, and spleen on a clinical CT data set and also on the MICCAI 2007 Grand Challenge liver data set. The results show the following: 1) The overall segmentation accuracy of true positive volume fraction \TPVF > 94.3\% and false positive volume fraction \FPVF < 0.2\% can be achieved; 2) the initialization performance can be improved by combining the AAM and LW; 3) the multiobject strategy greatly facilitates initialization; 4) compared with the traditional 3-D AAM method, the pseudo-3-D OAAM method achieves comparable performance while running 12 times faster; and 5) the performance of the proposed method is comparable to state-of-the-art liver segmentation algorithm. The executable version of the 3-D shape-constrained GC method with a user interface can be downloaded from xlink:href="http://xinjianchen.wordpress.com/research/" xlink:type="simple" xmlns:xlink="http://www.w3.org/1999/xlink"http://xinjianchen.wordpress.com/research/. [ABSTRACT FROM AUTHOR]

Action recognition is very important for many applications such as video surveillance, human–computer interaction, and so on; view-invariant action recognition is hot and difficult as well in this field. In this paper, a new discriminative model is proposed for video-based view-invariant action recognition. In the discriminative model, motion pattern and view invariants are perfectly fused together to make a better combination of invariance and distinctiveness. We address a series of issues, including interest point detection in image sequence, motion feature extraction and description, and view-invariant calculation. First, motion detection is used to extract motion information from videos, which is much more efficient than traditional background modeling and tracking-based methods. Second, as for feature representation, we exact variety of statistical information from motion and view-invariant feature based on cross ratio. Last, in the action modeling, we apply a discriminative probabilistic model-hidden conditional random field to model motion patterns and view invariants, by which we could fuse the statistics of motion and projective invariability of cross ratio in one framework. Experimental results demonstrate that our method can improve the ability to distinguish different categories of actions with high robustness to view change in real circumstances. [ABSTRACT FROM AUTHOR]

Segmenting semantically meaningful whole objects from images is a challenging problem, and it becomes especially so without higher level common sense reasoning. In this paper, we present an interactive segmentation framework that integrates image appearance and boundary constraints in a principled way to address this problem. In particular, we assume that small sets of pixels, which are referred to as seed pixels, are labeled as the object and background. The seed pixels are used to estimate the labels of the unlabeled pixels using Dirichlet process multiple-view learning, which leverages 1) multiple-view learning that integrates appearance and boundary constraints and 2) Dirichlet process mixture-based nonlinear classification that simultaneously models image features and discriminates between the object and background classes. With the proposed learning and inference algorithms, our segmentation framework is experimentally shown to produce both quantitatively and qualitatively promising results on a standard dataset of images. In particular, our proposed framework is able to segment whole objects from images given insufficient seeds. [ABSTRACT FROM AUTHOR]

Adaptive sampling schemes choose different sampling masks for different images. Blind adaptive sampling schemes use the measurements that they obtain (without any additional or direct knowledge about the image) to wisely choose the next sample mask. In this paper, we present and discuss two blind adaptive sampling schemes. The first is a general scheme not restricted to a specific class of sampling functions. It is based on an underlying statistical model for the image, which is updated according to the available measurements. A second less general but more practical method uses the wavelet decomposition of an image. It estimates the magnitude of the unsampled wavelet coefficients and samples those with larger estimated magnitude first. Experimental results show the benefits of the proposed blind sampling schemes. [ABSTRACT FROM AUTHOR]

Image formation algorithms in a variety of applications have explicit or implicit dependence on a mathematical model of the observation process. Inaccuracies in the observation model may cause various degradations and artifacts in the reconstructed images. The application of interest in this paper is synthetic aperture radar (SAR) imaging, which particularly suffers from motion-induced model errors. These types of errors result in phase errors in SAR data, which cause defocusing of the reconstructed images. Particularly focusing on imaging of fields that admit a sparse representation, we propose a sparsity-driven method for joint SAR imaging and phase error correction. Phase error correction is performed during the image formation process. The problem is set up as an optimization problem in a nonquadratic regularization-based framework. The method involves an iterative algorithm, where each iteration of which consists of consecutive steps of image formation and model error correction. Experimental results show the effectiveness of the approach for various types of phase errors, as well as the improvements that it provides over existing techniques for model error compensation in SAR. [ABSTRACT FROM AUTHOR]

This paper proposes the use of an adaptive locally oriented coordinate frame when calculating an optical flow field. The coordinate frame is aligned with the least curvature direction in a local window about each pixel. This has advantages to both fitting the flow field to the image data and in imposing smoothness constraints between neighboring pixels. In terms of fitting, robustness is obtained to a wider variety of image motions due to the extra invariance provided by the coordinate frame. Smoothness constraints are naturally propagated along image boundaries which often correspond to motion boundaries. In addition, moving objects can be efficiently segmented in the least curvature direction. We show experimentally the benefits of the method and demonstrate robustness to fast rotational motion, such as what often occurs in human motion. [ABSTRACT FROM AUTHOR]

Most existing nonblind image deblurring methods assume that the blur kernel is free of error. However, it is often unavoidable in practice that the input blur kernel is erroneous to some extent. Sometimes, the error could be severe, e.g., for images degraded by nonuniform motion blurring. When an inaccurate blur kernel is used as the input, significant distortions will appear in the image recovered by existing methods. In this paper, we present a novel convex minimization model that explicitly takes account of error in the blur kernel. The resulting minimization problem can be efficiently solved by the so-called accelerated proximal gradient method. In addition, a new boundary extension scheme is incorporated in the proposed model to further improve the results. The experiments on both synthesized and real images showed the efficiency and robustness of our algorithm to both the image noise and the model error in the blur kernel. [ABSTRACT FROM AUTHOR]

This paper proposes a new approach to the image blind super-resolution (BSR) problem in the case of affine interframe motion. Although the tasks of image registration, blur identification, and high-resolution (HR) image reconstruction are coupled in the imaging process, when dealing with nonisometric interframe motion or without the exact knowledge of the blurring process, classic SR techniques generally have to tackle them (maybe in some combinations) separately. The main difficulty is that state-of-the-art deconvolution methods cannot be straightforwardly generalized to cope with the space-variant motion. We prove that the operators of affine warping and blur commute with some additional transforms and derive an equivalent form of the BSR observation model. Using this equivalent form, we develop an iterative algorithm to jointly estimate the triple-coupled variables, i.e., the motion parameters, blur kernels, and HR image. Experiments on synthetic and real-life images illustrate the performance of the proposed technique in modeling the space-variant degradation process and restoring local textures. [ABSTRACT FROM AUTHOR]

This paper addresses statistical-based texture modeling using wavelets. We propose a new approach to represent the marginal distribution of the wavelet coefficients using finite mixtures of generalized Gaussian (MoGG) distributions. The MoGG captures a wide range of histogram shapes, which provides better description and discrimination of texture than using single probability density functions (pdf's), as proposed by recent state-of-the-art approaches. Moreover, we propose a model similarity measure based on Kullback–Leibler divergence (KLD) approximation using Monte Carlo sampling methods. Through experiments on two popular texture data sets, we show that our approach yields significant performance improvements for texture discrimination and retrieval, as compared with recent methods of statistical-based wavelet modeling. [ABSTRACT FROM AUTHOR]

Recently, extensive research efforts have been dedicated to view-based methods for 3-D object retrieval due to the highly discriminative property of multiviews for 3-D object representation. However, most of state-of-the-art approaches highly depend on their own camera array settings for capturing views of 3-D objects. In order to move toward a general framework for 3-D object retrieval without the limitation of camera array restriction, a camera constraint-free view-based (CCFV) 3-D object retrieval algorithm is proposed in this paper. In this framework, each object is represented by a free set of views, which means that these views can be captured from any direction without camera constraint. For each query object, we first cluster all query views to generate the view clusters, which are then used to build the query models. For a more accurate 3-D object comparison, a positive matching model and a negative matching model are individually trained using positive and negative matched samples, respectively. The CCFV model is generated on the basis of the query Gaussian models by combining the positive matching model and the negative matching model. The CCFV removes the constraint of static camera array settings for view capturing and can be applied to any view-based 3-D object database. We conduct experiments on the National Taiwan University 3-D model database and the ETH 3-D object database. Experimental results show that the proposed scheme can achieve better performance than state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This paper proposes a novel approach to jointly optimize spatial prediction and the choice of the subsequent transform in video and image compression. Under the assumption of a separable first-order Gauss-Markov model for the image signal, it is shown that the optimal Karhunen-Loeve Transform, given available partial boundary information, is well approximated by a close relative of the discrete sine transform (DST), with basis vectors that tend to vanish at the known boundary and maximize energy at the unknown boundary. The overall intraframe coding scheme thus switches between this variant of the DST named asymmetric DST (ADST), and traditional discrete cosine transform (DCT), depending on prediction direction and boundary information. The ADST is first compared with DCT in terms of coding gain under ideal model conditions and is demonstrated to provide significantly improved compression efficiency. The proposed adaptive prediction and transform scheme is then implemented within the H.264/AVC intra-mode framework and is experimentally shown to significantly outperform the standard intra coding mode. As an added benefit, it achieves substantial reduction in blocking artifacts due to the fact that the transform now adapts to the statistics of block edges. An integer version of this ADST is also proposed. [ABSTRACT FROM AUTHOR]