Search

This paper proposes a new method of calculating a matching metric for motion estimation. The proposed method splits the information in the source images into multiple scale and orientation subbands, reduces the subband values to a binary representation via an adaptive thresholding algorithm, and uses mutual information to model the similarity of corresponding square windows in each image. A moving window strategy is applied to recover a dense estimated motion field whose properties are explored. The proposed matching metric is a sum of mutual information scores across space, scale, and orientation. This facilitates the exploitation of information diversity in the source images. Experimental comparisons are performed amongst several related approaches, revealing that the proposed matching metric is better able to exploit information diversity, generating more accurate motion fields. [ABSTRACT FROM AUTHOR]

The great variations of videographic skills in videography, camera designs, compression and processing protocols, communication and bandwidth environments, and displays leads to an enormous variety of video impairments. Current no-reference (NR) video quality models are unable to handle this diversity of distortions. This is true in part because available video quality assessment databases contain very limited content, fixed resolutions, were captured using a small number of camera devices by a few videographers and have been subjected to a modest number of distortions. As such, these databases fail to adequately represent real world videos, which contain very different kinds of content obtained under highly diverse imaging conditions and are subject to authentic, complex, and often commingled distortions that are difficult or impossible to simulate. As a result, NR video quality predictors tested on real-world video data often perform poorly. Toward advancing NR video quality prediction, we have constructed a large-scale video quality assessment database containing 585 videos of unique content, captured by a large number of users, with wide ranges of levels of complex, authentic distortions. We collected a large number of subjective video quality scores via crowdsourcing. A total of 4776 unique participants took part in the study, yielding over 205 000 opinion scores, resulting in an average of 240 recorded human opinions per video. We demonstrate the value of the new resource, which we call the live video quality challenge database (LIVE-VQC), by conducting a comparison with leading NR video quality predictors on it. This paper is the largest video quality assessment study ever conducted along several key dimensions: number of unique contents, capture devices, distortion types and combinations of distortions, study participants, and recorded subjective scores. The database is available for download on this link: http://live.ece.utexas.edu/research/LIVEVQC/index.html. [ABSTRACT FROM AUTHOR]

Colorant-based direct binary search (CB-DBS) halftoning proposed in [27] provides an image quality benchmark for dispersed-dot halftoning algorithms. The objective of this paper is to further push the image quality limit. An algorithm called hierarchical colorant-based direct binary search (HCB-DBS) is developed in this paper. By appropriately integrating yellow colorant into dot-overlapping and dot-positioning controls, it is demonstrated that HCB-DBS can achieve better halftone texture of both individual and joint dot-color planes, without compromising the dot distribution of more visible halftone of cyan and magenta colorants. The input color specification is first converted from colorant space to dot-color space with minimum brightness variation principle for full dot-overlapping control. The dot-colors are then split into groups based upon dot visibility. Hierarchical monochrome DBS halftoning is applied to make dot-positioning decision for each group, constrained on the already generated halftone of the groups with higher priority. And dot-coloring is decided recursively with joint monochrome DBS halftoning constrained on the related total dot distribution. Experiments show HCB-DBS improves halftone texture for both individual and joint dot-color planes. And it reduces the halftone graininess and free of color mottle artifacts, comparing to CB-DBS. [ABSTRACT FROM AUTHOR]

A popular way to restore images comprising edges is to minimize a cost function combining a quadratic data-fidelity term and an edge-preserving (possibly nonconvex) regularization term. Mainly because of the latter term, the calculation of the solution is slow and cumbersome. Half-quadratic (HQ) minimization (multiplicative form) was pioneered by Geman and Reynolds (1992) in order to alleviate the computational task in the context of image reconstruction with nonconvex regularization. By promoting the idea of locally homogeneous image models with a continuous-valued line process, they reformulated the optimization problem in terms of an augmented cost function which is quadratic with respect to the image and separable with respect to the line process, hence the name "half quadratic." Since then, a large amount of papers were dedicated to HQ minimization and important results—including edge-preservation along with convex regularization and convergence—have been obtained. In this paper, we show that HQ minimization (multiplicative form) is equivalent to the most simple and basic method where the gradient of the cost function is linearized at each iteration step. In fact, both methods give exactly the same iterations. Furthermore, connections of HQ minimization with other methods, such as the quasi-Newton method and the generalized Weiszfeld's method, are straightforward. [ABSTRACT FROM AUTHOR]

The Fourier transform is one of the most important transformations in image processing. A major component of this influence comes from the ability to implement it efficiently on a digital computer. This paper describes a new methodology to perform a fast Fourier transform (FFT). This methodology emerges from considerations of the natural physical constraints imposed by image capture devices (camera/eye). The novel aspects of the specific FFT method described include: 1) a bit-wise reversal re-grouping operation of the conventional FFT is replaced by the use of lossless image rotation and scaling and 2) the usual arithmetic operations of complex multiplication are replaced with integer addition. The significance of the FFT presented in this paper is introduced by extending a discrete and finite image algebra, named Spiral Honeycomb image Algebra (SHIA), to a continuous version, named SHIAC. [ABSTRACT FROM AUTHOR]

Measurement of visual quality is of fundamental importance for numerous image and video processing applications, where the goal of quality assessment (QA) algorithms is to automatically assess the quality of images or videos in agreement with human quality judgments. Over the years, many researchers have taken different approaches to the problem and have contributed significant research in this area and claim to have made progress in their respective domains. It is important to evaluate the performance of these algorithms in a comparative setting and analyze the strengths and weaknesses of these methods. In this paper, we present results of an extensive subjective quality assessment study in which a total of 779 distorted images were evaluated by about two dozen human subjects. The "ground truth" image quality data obtained from about 25000 individual human quality judgments is used to evaluate the performance of several prominent full-reference image quality assessment algorithms. To the best of our knowledge, apart from video quality studies conducted by the Video Quality Experts Group, the study presented in this paper is the largest subjective image quality study in the literature in terms of number of images, distortion types, and number of human judgments per image. Moreover, we have made the data from the study freely available to the research community [1]. This would allow other researchers to easily report comparative results in the future. [ABSTRACT FROM AUTHOR]

Can images be decomposed into the sum of a geometric part and a textural part? In a theoretical breakthrough, [Y. Meyer, Oscillating Patterns in Image Processing and Nonlinear Evolution Equations. Providence, RI: American Mathematical Society, 2001] proposed variational models that force the geometric part into the space of functions with bounded variation, and the textural part into a space of oscillatory distributions. Meyer's models are simple minimization problems extending the famous total variation model. However, their numerical solution has proved challenging. It is the object of a literature rich in variants and numerical attempts. This paper starts with the linear model, which reduces to a low-pass/high-pass filter pair. A simple conversion of the linear filter pair into a nonlinear filter pair involving the total variation is introduced. This new-proposed nonlinear filter pair retains both the essential features of Meyer's models and the simplicity and rapidity of the linear model. It depends upon only one transparent parameter: the texture scale, measured in pixel mesh. Comparative experiments show a better and faster separation of cartoon from texture. One application is illustrated: edge detection. [ABSTRACT FROM AUTHOR]

Hyperspectral images exhibit strong dependencies across spatial and spectral neighbors, which have been proved to be very useful for hyperspectral image classification. State-of-the-art hyperspectral image classification algorithms use the dependencies in a heuristic way or in probabilistic frameworks but impose unreasonable assumptions on observed data. In this paper, we formulate a conditional random field (CRF) to replace such heuristics and unreasonable assumptions for the classification of hyperspectral images. Moreover, because of avoiding explicit modeling of the observed data, the proposed method can incorporate the classification of hyperspectral images with different statistics characteristics into a unified probabilistic framework. Since the usual classification task for hyperspectral images needs the proposed CRF to be trained on local samples, available global training methods cannot be directly used. Under piecewise training framework, this paper develops an efficient local method to train the CRF. It is efficiently implemented through separated trainings of simple classifiers defined by corresponding potentials. However, the independent classifier trainings may lead to over-counting problems during inference. So we further propose a strategy to combine the independently trained models to obtain final CRF model. Experiments on real-world hyperspectral data show that our algorithm is competitive with the most recent results in hyperspectral image classification. [ABSTRACT FROM AUTHOR]

Designing effective image priors is of great interest to image super-resolution (SR), which is a severely under-determined problem. An edge smoothness prior is favored since it is able to suppress the jagged edge artifact effectively. However, for soft image edges with gradual intensity transitions, it is generally difficult to obtain analytical forms for evaluating their smoothness. This paper characterizes soft edge smoothness based on a novel SoftCuts metric by generalizing the Geocuts method [1]. The proposed soft edge smoothness measure can approximate the average length of all level lines in an intensity image. Thus, the total length of all level lines can be minimized effectively by integrating this new form of prior. In addition, this paper presents a novel combination of this soft edge smoothness prior and the alpha matting technique for color image SR, by adaptively normalizing image edges according to their α-channel description. This leads to the adaptive SoftCuts algorithm, which represents a unified treatment of edges with different contrasts and scales. Experimental results are presented which demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

A novel method to reconstruct object boundaries with geodesic circular arc is proposed in this paper. Within this framework, an energy of circular arc spline is utilized to simultaneously arrange and interpolate each member in the set of sparse unorganized feature points from the desired boundaries. A general form for a family of parametric circular arc spline is firstly derived and followed by a novel method of arranging these feature points by minimizing an energy term depending on the circular arc spline configuration defined on these feature points. With regard to the fact that the energy function is usually nonconvex and nondifferentiable at its critical points, an improved scheme of particle swarm optimizer is given to find the minimum for the energy in this paper. With this improved scheme, each pair of neighboring feature points along the boundaries of the desired objects are picked out from the set of sparse unorganized feature points, and the corresponding directional chord tangent angles are computed simultaneously to finish interpolation. We show experimentally and comparatively that the proposed method can perform effectively to restrict leakage on weak boundaries and premature convergence on long concave boundaries. Besides, it has good noise robustness and can as well extract multiple and open boundaries. [ABSTRACT FROM AUTHOR]

The observation has been made by Aujol and Gilboa that the cartoon and texture components of the decomposition of an image should not be correlated, as they are generated from independent processes. They use this observation in order to choose an optimal fidelity parameter λ for the decomposition process. However, this determination can be quite inefficient since a wide range of parameters λ must be searched through before an estimated optimal parameter can be found. In the present paper, we take a different approach, in which the cartoon and texture components are explicitly decorrelated by adding a decorrelation term to the energy functional of the decomposition model of Osher, Solé, and Vese (the OSV model). Decomposition results of improved quality over those from the OSV model are obtained, as quantified by a series of new decomposition quality measures, with cartoon and texture information better separated into their respective components. A new derivation of the OSV model is developed which maintains the texture subcomponents g1 and g2 so that discrimination results similar to those from other decomposition models (e.g., from the model of Vese and Osher and Improved Edge Segregation) may be obtained. This derivation is extended to the proposed model, for which discrimination results are obtained in a substantially smaller number of iterations. [ABSTRACT FROM AUTHOR]

In this paper, effective multiresolution image representations using a combination of 2-D filter bank (FB) and directional wavelet transform (WT) are presented. The proposed methods yield simple implementation and low computation costs compared to previous 1-D and 2-D FB combinations or adaptive directional WT methods. Furthermore, they are nonredundant transforms and realize quad-tree like multiresolution representations. In applications on nonlinear approximation, image coding, and denoising, the proposed filter banks show visual quality improvements and have higher PSNR than the conventional separable WT or the contourlet. [ABSTRACT FROM AUTHOR]

This paper presents a new method for the relaxation of multiview registration error. The multiview registration problem is represented using a graph. Each node and each edge in the graph represents a 3-D data set and a pairwise registration, respectively. Assuming that all the pairwise registration processes have converged to fine results, this paper shows that the multiview registration problem can be converted into a quadratic programming problem of Lie algebra parameters. The constraints are obtained from every cycle of the graph to eliminate the accumulation errors of global registration. A linear solution is proposed to distribute the accumulation error to proper positions in the graph, as specified by the quadratic model. Since the proposed method does not involve the original 3-D data, it has low time and space complexity. Additionally, the proposed method can be embedded into a trust-region algorithm and, thus, can correctly handle the nonlinear effects of large accumulation errors, while preserving the global convergence property to the first-order critical point. Experimental results confirm both the efficiency and the accuracy of the proposed method. [ABSTRACT FROM AUTHOR]

Sparse representations of signals have drawn considerable interest in recent years. The assumption that natural signals, such as images, admit a sparse decomposition over a redundant dictionary leads to efficient algorithms for handling such sources of data. In particular, the design of well adapted dictionaries for images has been a major challenge. The K-SVD has been recently proposed for this task [1] and shown to perform very well for various grayscale image processing tasks. In this paper, we address the problem of learning dictionaries for color images and extend the K-SVD-based grayscale image denoising algorithm that appears in [21. This work puts forward ways for handling nonhomogeneous noise and missing information, paving the way to state-of-the-art results in applications such as color image denoising, demosaicing, and inpainting, as demonstrated in this paper. [ABSTRACT FROM AUTHOR]

In this paper, we put forward a novel fusion framework that mixes together label fields instead of observation data as is usually the case. Our framework takes as input two label fields: a quickly estimated and to-be-refined segmentation map and a spatial region map that exhibits the shape of the main objects of the scene. These two label fields are fused together with a global energy function that is minimized with a deterministic iterative conditional mode algorithm. As explained in the paper, the energy function may implement a pure fusion strategy or a fusion-reaction function. In the latter case, a data-related term is used to make the optimization problem well posed. We believe that the conceptual simplicity, the small number of parameters, the use of a simple and fast deterministic optimizer that admits a natural implementation on a parallel architecture are among the main advantages of our approach. Our fusion framework is adapted to various computer vision applications among which are motion segmentation, motion estimation and occlusion detection. [ABSTRACT FROM AUTHOR]

The majorize-minimize (MM) optimization technique has received considerable attention in signal and image processing applications, as well as in statistics literature. At each iteration of an MM algorithm, one constructs a tangent majorant function that majorizes the given cost function and is equal to it at the current iterate. The next iterate is obtained by minimizing this tangent majorant function, resulting in a sequence of iterates that reduces the cost function monotonically. A well-known special case of MM methods are expectation-maximization algorithms. In this paper, we expand on previous analyses of MM, due to Fessler and Hero, that allowed the tangent majorants to be constructed in iteration-dependent ways. Also, this paper overcomes an error in one of those earlier analyses. There are three main aspects in which our analysis builds upon previous work. First, our treatment relaxes many assumptions related to the structure of the cost function, feasible set, and tangent majorants. For example, the cost function can be nonconvex and the feasible set for the problem can be any convex set. Second, we propose convergence conditions, based on upper curvature bounds, that can be easier to verify than more standard continuity conditions. Furthermore, these conditions allow for considerable design freedom in the iteration-dependent behavior of the algorithm. Finally, we give an original characterization of the local region of convergence of MM algorithms based on connected (e.g., convex) tangent majorants. For such algorithms, cost function minimizers will locally attract the iterates over larger neighborhoods than typically is guaranteed with other methods. This expanded treatment widens the scope of the MM algorithm designs that can be considered for signal and image processing applications, allows us to verify the convergent behavior of previously published algorithms, and gives a fuller understanding overall of how these algorithms behave. [ABSTRACT FROM AUTHOR]

Image moments have been extensively used as feature descriptors. However, the quantization error introduced in discrete signals presents problems, especially when dealing with small-size images. This results in the fact that the invariant properties of moments are compromised. In this paper, we present a technique suitable for the calculation of moments from a continuous signal, derived by piecewise polynomial interpolation of the corresponding discrete one. The computed moments exhibit significantly increased accuracy while requiring trivial computational effort. Zernike moments are then computed using the proposed scheme and are shown to display increased stability to geometrical transformations. [ABSTRACT FROM AUTHOR]

Spatially variant apodization (SVA) is a nonlinear sidelobe reduction technique that improves sidelobe level and preserves resolution at the same time. This method implements a bidimensional finite impulse response filter with adaptive taps depending on image information. Some papers that have been previously published analyze SVA at the Nyquist rate or at higher rates focused on strip synthetic aperture radar (SAR). This paper shows that traditional SVA techniques are useless when the sensor operates with a squint angle. The reasons for this behaviour are analyzed, and a new implementation that largely improves the results is presented. The algorithm is applied to simulated SAR images in order to demonstrate the good quality achieved along with efficient computation. [ABSTRACT FROM AUTHOR]

Motion trajectories provide rich spatiotemporal information about an object's activity. This paper presents novel classification algorithms for recognizing object activity using object motion trajectory. In the proposed classification system, trajectories are segmented at points of change in curvature, and the subtrajectories are represented by their principal component analysis (PCA) coefficients. We first present a framework to robustly estimate the multivariate probability density function based on PCA coefficients of the subtrajectories using Gaussian mixture models (GMMs). We show that GMM-based modeling alone cannot capture the temporal relations and ordering between underlying entities. To address this issue, we use hidden Markov models (HMMs) with a data-driven design in terms of number of states and topology (e.g., left-right versus ergodic). Experiments using a database of over 5700 complex trajectories (obtained from UCI-KDD data archives and Columbia University Multimedia Group) subdivided into 85 different classes demonstrate the superiority of our proposed 11MM-based scheme using PCA coefficients of subtrajectories in comparison with other techniques in the literature. [ABSTRACT FROM AUTHOR]

In this paper, a Bayesian wavelet-based denoising procedure for multicomponent images is proposed. A denoising procedure is constructed that 1) fully accounts for the multicomponent image covariances, 2) makes use of Gaussian scale mixtures as prior models that approximate the marginal distributions of the wavelet coefficients well, and 3) makes use of a noise-free image as extra prior information. It is shown that such prior information is available with specific multicomponent image data of, e.g., remote sensing and biomedical imaging. Experiments are conducted in these two domains, in both simulated and real noisy conditions. [ABSTRACT FROM AUTHOR]

Lines provide important information in images, and line detection is crucial in many applications. However, most of the existing algorithms focus only on the extraction of line positions, ignoring line thickness. This paper presents a novel wide line detector using an isotropic nonlinear filter. Unlike most existing edge and line detectors which use directional derivatives, our proposed wide line detector applies a nonlinear filter to extract a line completely without any derivative. The detector is based on the isotropic responses via circular masks. A general scheme for the analysis of the robustness of the proposed wide line detector is introduced and the dynamic selection of parameters is developed. In addition, this paper investigates the relationship between the size of circular masks and the width of detected lines. A sequence of tests has been conducted on a variety of image samples and our experimental results demonstrate the feasibility and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

This paper describes new extensions to the previously published multivariate alteration detection (MAD) method for change detection in bi-temporal, multi- and hypervariate data such as remote sensing imagery. Much like boosting methods often applied in data mining work, the iteratively reweighted (IR) MAD method in a series of iterations places increasing focus on ‘difficult’ observations, here observations whose change status over time is uncertain. The MAD method is based on the established technique of canonical correlation analysis: for the multivariate data acquired at two points in time and covering the same geographical region, we calculate the canonical variates and subtract them from each other. These orthogonal differences contain maximum information on joint change in all variables (spectral bands). The change detected in this fashion is invariant to separate linear (affine) transformations in the originally measured variables at the two points in time, such as 1) changes in gain and offset in the measuring device used to acquire the data, 2) data normalization or calibration schemes that are linear (affine) in the gray values of the original variables, or 3) orthogonal or other affine transformations, such as principal component (PC) or maximum autocorrelation factor (MAF) transformations. The IR-MAD method first calculates ordinary canonical and original MAD variates. In the following iterations we apply different weights to the observations, large weights being assigned to observations that show little change, i.e., for which the sum of squared, standardized MAD variates is small, and small weights being assigned to observations for which the sum is large. Like the original MAD method, the iterative extension is invariant to linear (affine) transformations of the original variables. To stabilize solutions to the (IR-)MAD problem, some form of regularization may be needed. This is especially useful for work on hyperspectral data. This paper describes ordinary two-set canonical correlation analysis, the MAD transformation, the iterative extension, and three regularization schemes. A simple case with real Landsat Thematic Mapper (TM) data at one point in time and (partly) constructed data at the other point in time that demonstrates the superiority of the iterative scheme over the original MAD method is shown. Also, examples with SPOT High Resolution Visible data from an agricultural region in Kenya, and hyperspectral airborne HyMap data from a small rural area in southeastern Germany are given. The latter case demonstrates the need for regularization. [ABSTRACT FROM AUTHOR]

In this paper, we develop the nonsubsampled contourlet transform (NSCT) and study its applications. The construction proposed in this paper is based on a nonsubsampled pyramid structure and nonsubsampled directional filter banks. The result is a flexible multiscale, multidirection, and shift-invariant image decomposition that can be efficiently implemented via the à trous algorithm. At the core of the proposed scheme is the nonseparable two-channel nonsubsampled filter bank (NSFB). We exploit the less stringent design condition of the NSFB to design filters that lead to a NSCT with better frequency selectivity and regularity when compared to the contourlet transform. We propose a design framework based on the mapping approach, that allows for a fast implementation based on a lifting or ladder structure, and only uses one-dimensional filtering in some cases. In addition, our design ensures that the corresponding frame elements are regular, symmetric, and the frame is close to a tight one. We assess the performance of the NSCT in image denoising and enhancement applications. In both applications the NSCT compares favorably to other existing methods in the literature. [ABSTRACT FROM AUTHOR]

Transform-coded images exhibit distortions that fall outside of the assumptions of traditional denoising techniques. In this paper, we use tools from robust signal processing to construct linear, worst-case estimators for the denoising of transform compressed images. We show that while standard denoising is fundamentally determined by statistical models for images alone, the distortions induced by transform coding are heavily dependent on the structure of the transform used. Our method, thus, uses simple models for the image and for the quantization error, with the latter capturing the transform dependency. Based on these models, we derive optimal, linear estimators of the original image that are optimal in the mean-squared error sense for the worst-case cross correlation between the original and the quantization error. Our construction is transform agnostic and is applicable to transforms from block discrete cosine transforms to wavelets. Furthermore, our approach is applicable to different types of image statistics and can also serve as an optimization tool for the design of transforms/quantizers. Through the interaction of the source and quantizer models, our work provides useful insights and is instrumental in identifying and removing quantization artifacts from general signals coded with general transforms. As we decouple the modeling and processing steps, we allow for the construction of many different types of estimators depending on the desired sophistication and available computational complexity. In the low end of this spectrum, our lookup table based estimator, which can be deployed in low complexity environments, provides competitive PSNR values with some of the best results in the literature. [ABSTRACT FROM AUTHOR]

Contrast enhancement has an important role in image processing applications. Conventional contrast enhancement techniques either often fail to produce satisfactory results for a broad variety of low-contrast images, or cannot be automatically applied to different images, because their parameters must be specified manually to produce a satisfactory result for a given image. This paper describes a new automatic method for contrast enhancement. The basic procedure is to first group the histogram components of a low-contrast image into a proper number of bins according to a selected criterion, then redistribute these bins uniformly over the grayscale, and finally ungroup the previously grouped gray-levels. Accordingly, this new technique is named gray-level grouping (GLG). GLG not only produces results superior to conventional contrast enhancement techniques, but is also fully automatic in most circumstances, and is applicable to a broad variety of images. An extension of GLG, selective GLG (SGLG), and its variations will be discussed in Part II of this paper. SGLG selectively groups and ungroups histogram components to achieve specific application purposes, such as eliminating background noise, enhancing a specific segment of the histogram, and so on. The extension of GLG to color images will also be discussed in Part II. [ABSTRACT FROM AUTHOR]

This is Part II of the paper, "Gray-Level Grouping (GLG): an Automatic Method for Optimized Image Contrast Enhancement". Part I of this paper introduced a new automatic contrast enhancement technique: gray-level grouping (GLG). GLG is a general and powerful technique, which can be conveniently applied to a broad variety of low-contrast images and outperforms conventional contrast enhancement techniques. However, the basic GLG method still has limitations and cannot enhance certain classes of low-contrast images well, e.g., images with a noisy background. The basic GLG also cannot fulfill certain special application purposes, e.g., enhancing only part of an image which corresponds to a certain segment of the image histogram. In order to break through these limitations, this paper introduces an extension of the basic GLG algorithm, selective gray-level grouping (SGLG), which groups the histogram components in different segments of the grayscale using different criteria and, hence, is able to enhance different parts of the histogram to various extents. This paper also introduces two new preprocessing methods to eliminate background noise in noisy low-contrast images so that such images can be properly enhanced by the (S)GLG technique. The extension of (S)GLG to color images is also discussed in this paper. SGLG and its variations extend the capability of the basic GLG to a larger variety of low-contrast images, and can fulfill special application requirements. SGLG and its variations not only produce results superior to conventional contrast enhancement techniques, but are also fully automatic under most circumstances, and are applicable to a broad variety of images. [ABSTRACT FROM AUTHOR]

The separation of image content into semantic parts plays a vital role in applications such as compression, enhancement, restoration, and more. In recent years, several pioneering works suggested such a separation be based on variational fore mutation and others using independent component analysis and sparsity. This paper presents a novel method for separating images into texture and piecewise smooth (cartoon) parts, exploiting both the variational and the sparsity mechanisms. The method combines the basis pursuit denoising (BPDN) algorithm and the total-variation (TV) regularization scheme. The basic idea presented in this paper is the use of two appropriate dictionaries, one for the representation of textures and the other for the natural scene parts assumed to be piecewise smooth. Both dictionaries are chosen such that they lead to sparse representations over one type of image-content (either texture or piecewise smooth). The use of the BPDN with the two amalgamed dictionaries leads to the desired separation, along with noise removal as a by-product. As the need to choose proper dictionaries is generally hard, a TV regularization is employed to better direct the separation process and reduce ringing artifacts. We present a highly efficient numerical scheme to solve the combined optimization problem posed by our model and to show several experimental results that validate the algorithm's performance. [ABSTRACT FROM AUTHOR]

Color device calibration is traditionally performed using one-dimensional (1-D) per-channel tone-response corrections (TRCs). While 1-D TRCs are attractive in view of their low implementation complexity and efficient real-time processing of color images, their use severely restricts the degree of control that can be exercised along various device axes. A typical example is that per separation (or per-channel), TRCs in a printer can be used to either ensure gray balance along the C = M = Y axis or to provide a linear response in delta-E units along each of the individual (C, M, and Y) axis, but not both. This paper proposes a novel two-dimensional color correction architecture that enables much greater control over the device color gamut with a modest increase in implementation cost. Results show significant improvement in calibration accuracy and stability when compared to traditional 1-D calibration. Superior cost quality tradeoffs (over 1-D methods) are also achieved for emulation of one color device on another. [ABSTRACT FROM AUTHOR]

This paper presents a new approach to represent an image by "verge points," which are defined as high-curvature points on the image surface. This representation offers a compact and reversible way to preserve the essence of the original image. Various applications, such as compression, edge detection, image enhancement, and image editing, can be achieved based on this representation. In this paper, the whole procedure for verge point representation is presented. Based on these verge points, image reconstruction can be easily achieved via iterative linear interpolation. These extracted verge points with compatible properties are further linked into verge curves to offer more compact representation. Progressive representation is also developed based on a multiscale extraction scheme. Some potential applications are then presented to demonstrate the versatility of this representation. [ABSTRACT FROM AUTHOR]

This paper advances a new framework for chromatic filtering of color images. The chromatic content of a color image is encoded in the CIE u' v' chromaticity coordinates whereas the achromatic content is encoded as CIE Y tristimulus value. Within the u' v' chromaticity diagram, colors are added according to the well-known center of gravity law of additive color mixtures, which is generalized here into a nonlinear filtering scheme for processing the two chromatic signals u' and v'. The achromatic channel Y can be processed with traditional filtering schemes, either linear or nonlinear, depending on the specific task at hand. The most interesting characteristics of the new filtering scheme are: 1) the elimination of color smearing effects along edges between bright and dark areas; 2) the possibility of processing chromatic components in a noniterative fashion through linear convolution operations; and 3) the consequent amenability to computationailly efficient implementations with fast Fourier transform. The paper includes several examples with both synthetic and real images where the performance of the new filtering method is compared with that of other color image processing algorithms. [ABSTRACT FROM AUTHOR]

This paper investigates the use of a qualitative inverse scattering method known as the linear sampling method (LSM) for imaging underwater scenes using limited aperture receiver configurations. The LSM is based on solving a set of unstable integral equations known as the far-field equations and whose stability breaks down even further for under-sampled observation aperture data. Based on the results of a recent study concerning multi-frequency LSM imaging, we propose an iterative inversion method that is founded upon a compressive sensing framework. In particular, we leverage multi-frequency diversity in the data by imposing a partial frequency variation prior on the solution which we show is justified when the frequency bandwidth is sampled finely enough. We formulate an alternating direction method of multiplier approach to minimize the proposed cost function. Proof of concept is established through numerically generated data as well as experimental acoustic measurements taken in a shallow pool facility at the U.S Naval Research Laboratory. [ABSTRACT FROM AUTHOR]

In this paper, we propose an extremely efficient algorithm for visual re-ranking. By considering the original pairwise distance in the contextual space, we develop a feature vector called sparse contextual activation (SCA) that encodes the local distribution of an image. Hence, re-ranking task can be simply accomplished by vector comparison under the generalized Jaccard metric, which has its theoretical meaning in the fuzzy set theory. In order to improve the time efficiency of re-ranking procedure, inverted index is successfully introduced to speed up the computation of generalized Jaccard metric. As a result, the average time cost of re-ranking for a certain query can be controlled within 1 ms. Furthermore, inspired by query expansion, we also develop an additional method called local consistency enhancement on the proposed SCA to improve the retrieval performance in an unsupervised manner. On the other hand, the retrieval performance using a single feature may not be satisfactory enough, which inspires us to fuse multiple complementary features for accurate retrieval. Based on SCA, a robust feature fusion algorithm is exploited that also preserves the characteristic of high time efficiency. We assess our proposed method in various visual re-ranking tasks. Experimental results on Princeton shape benchmark (3D object), WM-SRHEC07 (3D competition), YAEL data set B (face), MPEG-7 data set (shape), and Ukbench data set (image) manifest the effectiveness and efficiency of SCA. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel algorithm to simultaneously accomplish color quantization and dithering of images. This is achieved by minimizing a perception-based cost function, which considers pixel-wise differences between filtered versions of the quantized image and the input image. We use edge aware filters in defining the cost function to avoid mixing colors on the opposite sides of an edge. The importance of each pixel is weighted according to its saliency. To rapidly minimize the cost function, we use a modified multi-scale iterative conditional mode (ICM) algorithm, which updates one pixel a time while keeping other pixels unchanged. As ICM is a local method, careful initialization is required to prevent termination at a local minimum far from the global one. To address this problem, we initialize ICM with a palette generated by a modified median-cut method. Compared with previous approaches, our method can produce high-quality results with a fewer visual artifacts but also requires significantly less computational effort. [ABSTRACT FROM AUTHOR]

There exist a significant number of benchmarks for evaluating the performance of boundary detection algorithms, most of them relying on some sort of comparison of the automatically-generated boundaries with human-labeled ones. Such benchmarks are composed of a representative image data set, as well as a comparison measure on the universe of boundary images. Despite many such data sets and measures have been proposed, there is no clear way of knowing which combinations of them are the most suitable for the task. In this paper, we introduce four criteria that allow for a sensible evaluation of the performance of a comparison measure on a given data set. The criteria mimic the way in which humans understand boundary images, as well as their ability to recognize the underlying scenes. These criteria can, as a final goal, quantify the ability of the boundary detection benchmarks to evaluate the performance of boundary detection methods, either edge-based or segmentation-based. [ABSTRACT FROM AUTHOR]

In this paper, we address the problem of imaging membrane proteins for single-particle cryo-electron microscopy reconstruction of the isolated protein structure. More precisely, we propose a method for learning and removing the interfering vesicle signals from the micrograph, prior to reconstruction. In our approach, we estimate the subspace of the vesicle structures and project the micrographs onto the orthogonal complement of this subspace. We construct a 2D statistical model of the vesicle structure, based on higher order singular value decomposition (HOSVD), by considering the structural symmetries of the vesicles in the polar coordinate plane. We then propose to lift the HOSVD model to a novel hierarchical model by summarizing the multidimensional HOSVD coefficients by their principal components. Along with the model, a solid vesicle normalization scheme and model selection criterion are proposed to make a compact and general model. The results show that the vesicle structures are accurately separated from the background by the HOSVD model that is also able to adapt to the asymmetries of the vesicles. This is a promising result and suggests even wider applicability of the proposed approach in learning and removal of statistical structures. [ABSTRACT FROM PUBLISHER]

Dilated convolutions support expanding receptive field without parameter exploration or resolution loss, which turn out to be suitable for pixel-level prediction problems. In this paper, we propose multiscale single image super-resolution (SR) based on dilated convolutions. We adopt dilated convolutions to expand the receptive field size without incurring additional computational complexity. We mix standard convolutions and dilated convolutions in each layer, called mixed convolutions, i.e., in the mixed convolutional layer, and the feature extracted by dilated convolutions and standard convolutions are concatenated. We theoretically analyze the receptive field and intensity of mixed convolutions to discover their role in SR. Mixed convolutions remove blind spots and capture the correlation between low-resolution (LR) and high-resolution (HR) image pairs successfully, thus achieving good generalization ability. We verify those properties of mixed convolutions by training 5-layer and 10-layer networks. We also train a 20-layer deep network to compare the performance of the proposed method with those of the state-of-the-art ones. Moreover, we jointly learn maps with different scales from a LR image to its HR one in a single network. Experimental results demonstrate that the proposed method outperforms the state-of-the-art ones in terms of PSNR and SSIM, especially for a large-scale factor. [ABSTRACT FROM AUTHOR]

This paper proposes a novel scheme of scalable coding for encrypted images. In the encryption phase, the original pixel values are masked by a modulo-256 addition with pseudorandom numbers that are derived from a secret key. After decomposing the encrypted data into a downsampled subimage and several data sets with a multiple-resolution construction, an encoder quantizes the subimage and the Hadamard coefficients of each data set to reduce the data amount. Then, the data of quantized subimage and coefficients are regarded as a set of bitstreams. At the receiver side, while a subimage is decrypted to provide the rough information of the original content, the quantized coefficients can be used to reconstruct the detailed content with an iteratively updating procedure. Because of the hierarchical coding mechanism, the principal original content with higher resolution can be reconstructed when more bitstreams are received. [ABSTRACT FROM AUTHOR]

A computational camera uses a combination of optics and processing to produce images that cannot be captured with traditional cameras. In the last decade, computational imaging has emerged as a vibrant field of research. A wide variety of computational cameras has been demonstrated to encode more useful visual information in the captured images, as compared with conventional cameras. In this paper, we survey computational cameras from two perspectives. First, we present a taxonomy of computational camera designs according to the coding approaches, including object side coding, pupil plane coding, sensor side coding, illumination coding, camera arrays and clusters, and unconventional imaging systems. Second, we use the abstract notion of light field representation as a general tool to describe computational camera designs, where each camera can be formulated as a projection of a high-dimensional light field to a 2-D image sensor. We show how individual optical devices transform light fields and use these transforms to illustrate how different computational camera designs (collections of optical devices) capture and encode useful visual information. [ABSTRACT FROM AUTHOR]

Image quality assessment (IQA) aims to use computational models to measure the image quality consistently with subjective evaluations. The well-known structural similarity index brings IQA from pixel- to structure-based stage. In this paper, a novel feature similarity (FSIM) index for full reference IQA is proposed based on the fact that human visual system (HVS) understands an image mainly according to its low-level features. Specifically, the phase congruency (PC), which is a dimensionless measure of the significance of a local structure, is used as the primary feature in FSIM. Considering that PC is contrast invariant while the contrast information does affect HVS' perception of image quality, the image gradient magnitude (GM) is employed as the secondary feature in FSIM. PC and GM play complementary roles in characterizing the image local quality. After obtaining the local quality map, we use PC again as a weighting function to derive a single quality score. Extensive experiments performed on six benchmark IQA databases demonstrate that FSIM can achieve much higher consistency with the subjective evaluations than state-of-the-art IQA metrics. [ABSTRACT FROM PUBLISHER]

Software realizations of computationally-demanding image processing tasks (e.g., image transforms and convolution) do not currently provide graceful degradation when their clock-cycles budgets are reduced, e.g., when delay deadlines are imposed in a multitasking environment to meet throughput requirements. This is an important obstacle in the quest for full utilization of modern programmable platforms' capabilities since worst-case considerations must be in place for reasonable quality of results. In this paper, we propose (and make available online) platform-independent software designs performing bitplane-based computation combined with an incremental packing framework in order to realize block transforms, 2-D convolution and frame-by-frame block matching. The proposed framework realizes incremental computation: progressive processing of input-source increments improves the output quality monotonically. Comparisons with the equivalent nonincremental software realization of each algorithm reveal that, for the same precision of the result, the proposed approach can lead to comparable or faster execution, while it can be arbitrarily terminated and provide the result up to the computed precision. Application examples with region-of-interest based incremental computation, task scheduling per frame, and energy-distortion scalability verify that our proposal provides significant performance scalability with graceful degradation. [ABSTRACT FROM AUTHOR]

This paper proposes a robust single-image super-resolution method for enlarging low quality web image/video degraded by downsampling and compression. To simultaneously improve the resolution and perceptual quality of such web image/video, we bring forward a practical solution which combines adaptive regularization and learning-based super-resolution. The contribution of this work is twofold. First, we propose to analyze the image energy change characteristics during the iterative regularization process, i.e., the energy change ratio between primitive (e.g., edges, ridges and corners) and nonprimitive fields. Based on the revealed convergence property of the energy change ratio, appropriate regularization strength can then be determined to well balance compression artifacts removal and primitive components preservation. Second, we verify that this adaptive regularization can steadily and greatly improve the pair matching accuracy in learning-based super-resolution. Consequently, their combination effectively eliminates the quantization noise and meanwhile faithfully compensates the missing high-frequency details, yielding robust super-resolution performance in the compression scenario. Experimental results demonstrate that our solution produces visually pleasing enlargements for various web images/videos. [ABSTRACT FROM AUTHOR]

The Bag-of-Words (BoW) model is a promising image representation technique for image categorization and annotation tasks. One critical limitation of existing BoW models is that much semantic information is lost during the codebook generation process, an important step of BoW. This is because the codebook generated by BoW is often obtained via building the codebook simply by clustering visual features in Euclidian space. However, visual features related to the same semantics may not distribute in clusters in the Euclidian space, which is primarily due to the semantic gap between low-level features and high-level semantics. In this paper, we propose a novel scheme to learn optimized BoW models, which aims to map semantically related features to the same visual words. In particular, we consider the distance between semantically identical features as a measurement of the semantic gap, and attempt to learn an optimized codebook by minimizing this gap, aiming to achieve the minimal loss of the semantics. We refer to such kind of novel codebook as semantics-preserving codebook (SPC) and the corresponding model as the Semantics-Preserving Bag-of-Words (SPBoW) model. Extensive experiments on image annotation and object detection tasks with public testbeds from MIT's Labelme and PASCAL VOC challenge databases show that the proposed SPC learning scheme is effective for optimizing the codebook generation process, and the SPBoW model is able to greatly enhance the performance of the existing BoW model. [ABSTRACT FROM AUTHOR]

The distance between a straight line and a straight line segment in the image space is proposed in this paper. Based on this distance, the neighborhood of a straight line segment is defined and mapped into the parameter space to obtain the parameter space neighborhood of the straight line segment. The neighborhood mapping between the image space and parameter space is a one to one reversible map. The mapped region in the parameter space is analytically derived and it is proved that it can be effi- ciently approximated by a quadrangle. The proposed straight line segment neighborhood technique for the HT outperforms conventional straight line neighborhood methods currently used with existing HT variations. In contrast to the straight line neighborhoods used in existing HT variations, the proposed straight line segment neighborhood has several advantages including: 1) the detection error of the proposed neighborhood is not affected by the length of the straight line segments; 2) a precision requirement in the image space described using the proposed distance can be explicitly resolved using the proposed formulation; 3) the proposed neighborhood has the ability to distinguish between segments belonging to the same straight line. A variety of experiments are executed to demonstrate that the proposed neighborhood has a variety of interesting properties of high practical value. [ABSTRACT FROM AUTHOR]

This paper proposes a selective data pruning-based compression scheme to improve the rate-distortion relation of compressed images and video sequences. The original frames are pruned to a smaller size before compression. After decoding, they are interpolated back to their original size by an edge-directed interpolation method. The data pruning phase is optimized to obtain the minimal distortion in the interpolation phase. Furthermore, a novel high-order interpolation is proposed to adapt the interpolation to several edge directions in the current frame. This high-order filtering uses more surrounding pixels in the frame than the fourth-order edge-directed method and it is more robust. The algorithm is also considered for multiframe-based interpolation by using spatio-temporally surrounding pixels coming from the previous frame. Simulation results are shown for both image interpolation and coding applications to validate the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

In the literature of psychophysics and neurophysiology, many studies have shown that both global and local features are crucial for face representation and recognition. This paper proposes a novel face recognition method which exploits both global and local discriminative features. In this method, global features are extracted from the whole face images by keeping the low-frequency coefficients of Fourier transform, which we believe encodes the holistic facial information, such as facial contour. For local feature extraction, Gabor wavelets are exploited considering their biological relevance. After that, Fisher's linear discriminant (FLD) is separately applied to the global Fourier features and each local patch of Gabor features. Thus, multiple FLD classifiers are obtained, each embodying different facial evidences for face recognition. Finally, all these classifiers are combined to form hierarchical ensemble classifier. We evaluate the proposed method using two large-scale face databases: FERET and FRGC version 2.0. Experiments show that the results of our method are impressively better than the best known results with the same evaluation protocol. [ABSTRACT FROM AUTHOR]

Three-dimensional reconstruction of vessels from digital X-ray angiographic images is a powerful technique that compensates for limitations in angiography. It can provide physicians with the ability to accurately inspect the complex arterial network and to quantitatively assess disease induced vascular alterations in three dimensions. In this paper, both the projection principle of single view angiography and mathematical modeling of two view angiographies are studied in detail. The movement of the table, which commonly occurs during clinical practice, complicates the reconstruction process. On the basis of the pinhole camera model] and existing optimization methods, an algorithm is developed for 3-D reconstruction of coronary arteries from two uncalibrated monoplane angiographic images. A simple and effective perspective projection model is proposed for the 3-D reconstruction of coronary arteries. A nonlinear optimization method is employed for refinement of the 3-D structure of the vessel skeletons, which takes the influence of table movement into consideration. An accurate model is suggested for the calculation of contour points of the vascular surface, which fully utilizes the information in the two projections. In our experiments with phantom and patient angiograms, the vessel centerlines are reconstructed in 3-D space with a mean positional accuracy of 0.665 mm and with a mean back projection error of 0.259 mm. This shows that the algorithm put forward in this paper is very effective and robust. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel solution to an arbitrary noncausal, multidimensional hidden Markov model (HMM) for image and video classification. First, we show that the noncausal model can be solved by splitting it into multiple causal HMMs and simultaneously solving each causal HMM using a fully synchronous distributed computing framework, therefore referred to as distributed HMMs. Next we present an approximate solution to the multiple causal HMMs that is based on an alternating updating scheme and assumes a realistic sequential computing framework. The parameters of the distributed causal HMMs are estimated by extending the classical 1-D training and classification algorithms to multiple dimensions. The proposed extension to arbitrary causal, multidimensional HMMs allows state transitions that are dependent on all causal neighbors. We, thus, extend three fundamental algorithms to multidimensional causal systems, i.e., 1) expectation-maximization (EM), 2) general forward-backward (GFB), and 3) Viterbi algorithms. In the simulations, we choose to limit ourselves to a noncausal 2-D model whose noncausality is along a single dimension, in order to significantly reduce the computational complexity. Simulation results demonstrate the superior performance, higher accuracy rate, and applicability of the proposed noncausal HMM framework to image and video classification. [ABSTRACT FROM AUTHOR]

The logarithmic image processing (LIP) model is a mathematical theory that provides new operations for image processing. The contrast definition has been shown to be consistent with some important physical laws and characteristics of human visual system. In this paper, we establish an information-theoretic interpretation of the contrast definition. We show that it can be expressed as a combination of the relative entropy and Shannon's information content. Based on this new interpretation, we propose an adaptive algorithm for enhancing the contrast and sharpness of noisy images. [ABSTRACT FROM AUTHOR]

This paper proposes a novel approach to extract image features for texture classification. The proposed features are robust to image rotation, less sensitive to histogram equalization and noise. It comprises of two sets of features: dominant local binary patterns (DLBP) in a texture image and the supplementary features extracted by using the circularly symmetric Gabor filter responses. The dominant local binary pattern method makes use of the most frequently occurred patterns to capture descriptive textural information, while the Gabor-based features aim at supplying additional global textural information to the DLBP features. Through experiments, the proposed approach has been intensively evaluated by applying a large number of classification tests to histogram-equalized, randomly rotated and noise corrupted images in Outex, Brodatz, Meastex, and CUReT texture image databases. Our method has also been compared with six published texture features in the experiments. It is experimentally demonstrated that the proposed method achieves the highest classification accuracy in various texture databases and image conditions. [ABSTRACT FROM AUTHOR]

In this paper, we aim to handle the intraclass variation resulting from the geometric transformation and the illumination change for more robust texture classification. To this end, we propose a novel feature descriptor called Radon representation-based feature descriptor (RRFD). RRFD converts the original pixel represented images into Radon-pixel images by using the Radon transform. The new Radon-pixel representation is more informative in geometry and has a much lower dimension. Subsequently, RRFD efficiently achieves affine invariance by projecting an image (or an image patch) from the space of Radon-pixel pairs onto an invariant feature space by using a ratiogram, i.e., the histogram of ratios between the areas of triangle pairs. The illumination invariance is also achieved by defining an illumination invariant distance metric on the invariant feature space. Comparing to the existing Radon transform-based texture features, which only achieve rotation and/or scaling invariance, RRFD achieves affine invariance. The experimental results on CUReT show that RRFD is a powerful feature descriptor that is suitable for texture classification. [ABSTRACT FROM AUTHOR]