Search

Driven by recent vision and graphics applications such as image segmentation and object recognition, computing pixel-accurate saliency values to uniformly highlight foreground objects becomes increasingly important. In this paper, we propose a unified framework called pixelwise image saliency aggregating (PISA) various bottom-up cues and priors. It generates spatially coherent yet detail-preserving, pixel-accurate, and fine-grained saliency, and overcomes the limitations of previous methods, which use homogeneous superpixel based and color only treatment. PISA aggregates multiple saliency cues in a global context, such as complementary color and structure contrast measures, with their spatial priors in the image domain. The saliency confidence is further jointly modeled with a neighborhood consistence constraint into an energy minimization formulation, in which each pixel will be evaluated with multiple hypothetical saliency levels. Instead of using global discrete optimization methods, we employ the cost-volume filtering technique to solve our formulation, assigning the saliency levels smoothly while preserving the edge-aware structure details. In addition, a faster version of PISA is developed using a gradient-driven image subsampling strategy to greatly improve the runtime efficiency while keeping comparable detection accuracy. Extensive experiments on a number of public data sets suggest that PISA convincingly outperforms other state-of-the-art approaches. In addition, with this work, we also create a new data set containing 800 commodity images for evaluating saliency detection. [ABSTRACT FROM PUBLISHER]

Objects that do not lie at the focal distance of a digital camera generate defocused regions in the captured image. This paper presents a new edge-based method for spatially varying defocus blur estimation using a single image based on reblurred gradient magnitudes. The proposed approach initially computes a scale-consistent edge map of the input image and selects a local reblurring scale aiming to cope with noise, edge mis-localization, and interfering edges. An initial blur estimate is computed at the detected scale-consistent edge points and a novel connected edge filter is proposed to smooth the sparse blur map based on pixel connectivity within detected edge contours. Finally, a fast guided filter is used to propagate the sparse blur map through the whole image. Experimental results show that the proposed approach presents a very good compromise between estimation error and running time when compared with the state-of-the-art methods. We also explore our blur estimation method in the context of image deblurring, and show that metrics typically used to evaluate blur estimation may not correlate as expected with the visual quality of the deblurred image. [ABSTRACT FROM PUBLISHER]

Texture filtering seeks to smooth out textured details in order to present structures prominently. To filter out multi-scale textured details while preserving structures, some methods propose to adjust the size of the filtering windows by handling the pixels near structures with small windows and the other pixels with large windows. Unfortunately, their adjustment measures are not very effective in treating complex situations. They may handle the pixels inside small structures with large windows, which overly smooths them, or they may be incapable of smoothing out large-scale textures. With regard to this, in this paper, we present a novel method that adjusts the window size for a pixel by checking similar pixels in its neighborhood. In general, pixels nearer structures have fewer similar pixels in their neighborhoods than the pixels that are farther away from structures. Thus, our adjustment can adaptively adjust window sizes by the distances from pixels to structures, which improves the potential to treat complex situations. The experimental results show that we can well filter out very large-scale textures while preserving small structures with high quality in a manner superior to the state-of-the-art methods. In addition, our method is very simple and efficient, and can be used easily in applications. [ABSTRACT FROM AUTHOR]

In this paper, we are concerned with image downsampling using subpixel techniques to achieve superior sharpness for small liquid crystal displays (LCDs). Such a problem exists when a high-resolution image or video is to be displayed on low-resolution display terminals. Limited by the low-resolution display, we have to shrink the image. Signal-processing theory tells us that optimal decimation requires low-pass filtering with a suitable cutoff frequency, followed by downsampling. In doing so, we need to remove many useful image details causing blurring. Subpixel-based downsampling, taking advantage of the fact that each pixel on a color LCD is actually composed of individual red, green, and blue subpixel stripes, can provide apparent higher resolution. In this paper, we use frequency-domain analysis to explain what happens in subpixel-based downsampling and why it is possible to achieve a higher apparent resolution. According to our frequency-domain analysis and observation, the cutoff frequency of the low-pass filter for subpixel-based decimation can be effectively extended beyond the Nyquist frequency using a novel antialiasing filter. Applying the proposed filters to two existing subpixel downsampling schemes called direct subpixel-based downsampling (DSD) and diagonal DSD (DDSD), we obtain two improved schemes, i.e., DSD based on frequency-domain analysis (DSD-FA) and DDSD based on frequency-domain analysis (DDSD-FA). Experimental results verify that the proposed DSD-FA and DDSD-FA can provide superior results, compared with existing subpixel or pixel-based downsampling methods. [ABSTRACT FROM AUTHOR]

Defocus blur detection is an important and challenging task in computer vision and digital imaging fields. Previous work on defocus blur detection has put a lot of effort into designing local sharpness metric maps. This paper presents a simple yet effective method to automatically obtain the local metric map for defocus blur detection, which based on the feature learning of multiple convolutional neural networks (ConvNets). The ConvNets automatically learn the most locally relevant features at the super-pixel level of the image in a supervised manner. By extracting convolution kernels from the trained neural network structures and processing it with principal component analysis, we can automatically obtain the local sharpness metric by reshaping the principal component vector. Meanwhile, an effective iterative updating mechanism is proposed to refine the defocus blur detection result from coarse to fine by exploiting the intrinsic peculiarity of the hyperbolic tangent function. The experimental results demonstrate that our proposed method consistently performed better than the previous state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Image processing in hexagonal lattice has many advantages rather than square lattice. Researchers have addressed benefits of hexagonal structure in applications such as binarization, rotation, scaling and edge detection. Approximately all existing hardwares for capturing and displaying images are based on square lattice. Therefore, the best way for using advantages of hexagonal lattice is to find a proper software approach to convert square pixels to hexagonal ones. This paper presents a hexagonal platform based on interpolation which addresses three existing hexagonal challenges including imperfect hexagonal shape, inaccurate intensity level of hexagonal pixels and lower resolution in hexagonal space. The proposed interpolation is computed by overlaps between square and hexagonal pixels. Overlap types are formulated mathematically in 8 separate cases. Each overlap case is detected automatically and used to compute final gray-level intensity of hexagonal pixels. It is mathematically and experimentally shown that the proposed method satisfies necessary conditions for square-to-hexagonal conversion. The proposed scheme is evaluated on synthetic and real images with 10 different levels of noise in interpolation and edge detection applications. In synthetic images, the proposed method achieves the best figure of merit (FOM) 99.92% and 98.67% in high and low SNRs 100 and 20, respectively. Also, the proposed method outperforms existing state of the art hexagonal lattices with interclass correlation coefficient (ICC) 84.18% and mean rating 7.7 (out of 9) in real images. [ABSTRACT FROM AUTHOR]

Defocus blur detection (DBD), which has been widely applied to various fields, aims to detect the out-of-focus or in-focus pixels from a single image. Despite the fact that the deep learning based methods applied to DBD have outperformed the hand-crafted feature based methods, the performance cannot still meet our requirement. In this paper, a novel network is established for DBD. Unlike existing methods which only learn the projection from the in-focus part to the ground-truth, both in-focus and out-of-focus pixels, which are completely and symmetrically complementary, are taken into account. Specifically, two symmetric branches are designed to jointly estimate the probability of focus and defocus pixels, respectively. Due to their complementary constraint, each layer in a branch is affected by an attention obtained from another branch, effectively learning the detailed information which may be ignored in one branch. The feature maps from these two branches are then passed through a unique fusion block to simultaneously get the two-channel output measured by a complementary loss. Additionally, instead of estimating only one binary map from a specific layer, each layer is encouraged to estimate the ground truth to guide the binary map estimation in its linked shallower layer followed by a top-to-bottom combination strategy, gradually exploiting the global and local information. Experimental results on released datasets demonstrate that our proposed method remarkably outperforms state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Helping robots understand indoor scenes has considerable value in computer vision. However, due to the diversity of indoor scenes, understanding them remains a big challenge. There are many spatial right-angles in indoor scenes. These spatial right-angles are projected into diverse 2D projections. These projections can be considered a composition of a pair of lines (line-pairs). Given the vanishing points (VPs), line segments can be assigned to 1 of 3 main orthogonal directions. The line-pairs (intersection of 2 lines), such that each of them converges to a different VP, are likely to be the projection of a spatial right-angle onto the image plane. These projections may enable us to estimate their original orientation and position in 3D scenes. In this paper, we presented a method to efficiently understand indoor scenes from a single image, without training or any knowledge of the camera’s internal calibration. Through geometric inference of line-pairs, it is possible to find these spatial right-angle projections. Then, these projections can be assigned to different clusters, and the line that lies in the neighbor-cluster helps us estimate the layout of the indoor scene. The proposed approach required no prior training. We compared the room layout estimated by our algorithm against the room box ground truth, measuring the percentage of pixels that were correctly classified. These experiments showed that our method estimated not only room layout, but also details of the indoor scene. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel L0 -regularized optimization framework for image downscaling. The optimization is driven by two L0 -regularized priors. The first prior, gradient-ratio prior, is based on the observation that the number of edges in the downscaled image is approximately inverse square proportional to the downscaling factor. By introducing L0 norm sparsity to the gradient ratio, the downscaled image is able to preserve the most salient edges as well as the visual perception of the original image. The second prior, downsampling prior, is to constrain the downsampling matrix so that pixels of the downscaled image are estimated according to those optimal neighboring pixels. Extensive experiments on the Urban100 and BSDS500 data sets show that the proposed algorithm achieves superior performance over the state-of-the-arts, in terms of both quality and robustness. [ABSTRACT FROM PUBLISHER]

A large number of color filter arrays (CFAs), periodic or aperiodic, have been proposed. To reconstruct images from all different CFAs and compare their imaging quality, a universal demosaicking method is needed. This paper proposes a new universal demosaicking method based on inter-pixel chrominance capture and optimal demosaicking transformation. It skips the commonly used step to estimate the luminance component at each pixel, and thus, avoids the associated estimation error. Instead, we directly use the acquired CFA color intensity at each pixel as an input component. Two independent chrominance components are estimated at each pixel based on the inter-pixel chrominance in the window, which is captured with the difference of CFA color values between the pixel of interest and its neighbors. Two mechanisms are employed for the accurate estimation: distance-related and edge-sensing weighting to reflect the confidence levels of the inter-pixel chrominance components, and pseudoinverse-based estimation from the components in a window. Then from the acquired CFA color component and two estimated chrominance components, the three primary colors are reconstructed by a linear color transform, which is optimized for the least transform error. Our experiments show that the proposed method is much better than other published universal demosaicking methods. [ABSTRACT FROM PUBLISHER]

Object proposal is essential for current state-of-the-art object detection pipelines. However, the existing proposal methods generally fail in producing results with satisfying localization accuracy. The case is even worse for small objects, which, however, are quite common in practice. In this paper, we propose a novel scale-aware pixelwise object proposal network (SPOP-net) to tackle the challenges. The SPOP-net can generate proposals with high recall rate and average best overlap, even for small objects. In particular, in order to improve the localization accuracy, a fully convolutional network is employed which predicts locations of object proposals for each pixel. The produced ensemble of pixelwise object proposals enhances the chance of hitting the object significantly without incurring heavy extra computational cost. To solve the challenge of localizing objects at small scale, two localization networks, which are specialized for localizing objects with different scales are introduced, following the divide-and-conquer philosophy. Location outputs of these two networks are then adaptively combined to generate the final proposals by a large-/small-size weighting network. Extensive evaluations on PASCAL VOC 2007 and COCO 2014 show the SPOP network is superior over the state-of-the-art models. The high-quality proposals from SPOP-net also significantly improve the mean average precision of object detection with Fast-Regions with CNN features framework. Finally, the SPOP-net (trained on PASCAL VOC) shows great generalization performance when testing it on ILSVRC 2013 validation set. [ABSTRACT FROM PUBLISHER]

In this paper, we present a novel algorithm to simultaneously accomplish color quantization and dithering of images. This is achieved by minimizing a perception-based cost function, which considers pixel-wise differences between filtered versions of the quantized image and the input image. We use edge aware filters in defining the cost function to avoid mixing colors on the opposite sides of an edge. The importance of each pixel is weighted according to its saliency. To rapidly minimize the cost function, we use a modified multi-scale iterative conditional mode (ICM) algorithm, which updates one pixel a time while keeping other pixels unchanged. As ICM is a local method, careful initialization is required to prevent termination at a local minimum far from the global one. To address this problem, we initialize ICM with a palette generated by a modified median-cut method. Compared with previous approaches, our method can produce high-quality results with a fewer visual artifacts but also requires significantly less computational effort. [ABSTRACT FROM AUTHOR]

Image denoising is a fundamental operation in image processing, and its applications range from the direct (photographic enhancement) to the technical (as a subproblem in image reconstruction algorithms). In many applications, the number of pixels has continued to grow, while the serial execution speed of computational hardware has begun to stall. New image processing algorithms must exploit the power offered by massively parallel architectures like graphics processing units (GPUs). This paper describes a family of image denoising algorithms well-suited to the GPU. The algorithms iteratively perform a set of independent, parallel 1D pixel-update subproblems. To match GPU memory limitations, they perform these pixel updates in-place and only store the noisy data, denoised image, and problem parameters. The algorithms can handle a wide range of edge-preserving roughness penalties, including differentiable convex penalties and anisotropic total variation. Both algorithms use the majorize–minimize framework to solve the 1D pixel update subproblem. Results from a large 2D image denoising problem and a 3D medical imaging denoising problem demonstrate that the proposed algorithms converge rapidly in terms of both iteration and run-time. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a novel image interpolation algorithm via graph-based Bayesian label propagation. The basic idea is to first create a graph with known and unknown pixels as vertices and with edge weights encoding the similarity between vertices, then the problem of interpolation converts to how to effectively propagate the label information from known points to unknown ones. This process can be posed as a Bayesian inference, in which we try to combine the principles of local adaptation and global consistency to obtain accurate and robust estimation. Specially, our algorithm first constructs a set of local interpolation models, which predict the intensity labels of all image samples, and a loss term will be minimized to keep the predicted labels of the available low-resolution (LR) samples sufficiently close to the original ones. Then, all of the losses evaluated in local neighborhoods are accumulated together to measure the global consistency on all samples. Moreover, a graph-Laplacian-based manifold regularization term is incorporated to penalize the global smoothness of intensity labels, such smoothing can alleviate the insufficient training of the local models and make them more robust. Finally, we construct a unified objective function to combine together the global loss of the locally linear regression, square error of prediction bias on the available LR samples, and the manifold regularization term. It can be solved with a closed-form solution as a convex optimization problem. Experimental results demonstrate that the proposed method achieves competitive performance with the state-of-the-art image interpolation algorithms. [ABSTRACT FROM PUBLISHER]

This paper proposes the use of an adaptive locally oriented coordinate frame when calculating an optical flow field. The coordinate frame is aligned with the least curvature direction in a local window about each pixel. This has advantages to both fitting the flow field to the image data and in imposing smoothness constraints between neighboring pixels. In terms of fitting, robustness is obtained to a wider variety of image motions due to the extra invariance provided by the coordinate frame. Smoothness constraints are naturally propagated along image boundaries which often correspond to motion boundaries. In addition, moving objects can be efficiently segmented in the least curvature direction. We show experimentally the benefits of the method and demonstrate robustness to fast rotational motion, such as what often occurs in human motion. [ABSTRACT FROM AUTHOR]

We present an automatic vehicle detection system for aerial surveillance in this paper. In this system, we escape from the stereotype and existing frameworks of vehicle detection in aerial surveillance, which are either region based or sliding window based. We design a pixelwise classification method for vehicle detection. The novelty lies in the fact that, in spite of performing pixelwise classification, relations among neighboring pixels in a region are preserved in the feature extraction process. We consider features including vehicle colors and local features. For vehicle color extraction, we utilize a color transform to separate vehicle colors and nonvehicle colors effectively. For edge detection, we apply moment preserving to adjust the thresholds of the Canny edge detector automatically, which increases the adaptability and the accuracy for detection in various aerial images. Afterward, a dynamic Bayesian network (DBN) is constructed for the classification purpose. We convert regional local features into quantitative observations that can be referenced when applying pixelwise classification via DBN. Experiments were conducted on a wide variety of aerial videos. The results demonstrate flexibility and good generalization abilities of the proposed method on a challenging data set with aerial surveillance images taken at different heights and under different camera angles. [ABSTRACT FROM AUTHOR]

Wavelets with composite dilations provide a general framework for the construction of waveforms defined not only at various scales and locations, as traditional wavelets, but also at various orientations and with different scaling factors in each coordinate. As a result, they are useful to analyze the geometric information that often dominate multidimensional data much more efficiently than traditional wavelets. The shearlet system, for example, is a particular well-known realization of this framework, which provides optimally sparse representations of images with edges. In this paper, we further investigate the constructions derived from this approach to develop critically sampled wavelets with composite dilations for the purpose of image coding. Not only do we show that many nonredundant directional constructions recently introduced in the literature can be derived within this setting, but we also introduce new critically sampled discrete transforms that achieve much better nonlinear approximation rates than traditional discrete wavelet transforms and outperform the other critically sampled multiscale transforms recently proposed. [ABSTRACT FROM AUTHOR]

Vision-based road detection in unstructured environments is a challenging problem as there are hardly any discernible and invariant features that can characterize the road or its boundaries in such environments. However, a salient and consistent feature of most roads or tracks regardless of type of the environments is that their edges, boundaries, and even ruts and tire tracks left by previous vehicles on the path appear to converge into a single point known as the vanishing point. Hence, estimating this vanishing point plays a pivotal role in the determination of the direction of the road. In this paper, we propose a novel methodology based on image texture analysis for the fast estimation of the vanishing point in challenging and unstructured roads. The key attributes of the methodology consist of the optimal local dominant orientation method that uses joint activities of only four Gabor filters to precisely estimate the local dominant orientation at each pixel location in the image plane, the weighting of each pixel based on its dominant orientation, and an adaptive distance-based voting scheme for the estimation of the vanishing point. A series of quantitative and qualitative analyses are presented using natural data sets from the Defense Advanced Research Projects Agency Grand Challenge projects to demonstrate the effectiveness and the accuracy of the proposed methodology. [ABSTRACT FROM PUBLISHER]

This paper proposes a low-distortion transform for prediction-error expansion reversible watermarking. The transform is derived by taking a simple linear predictor and by embedding the expanded prediction error not only into the current pixel but also into its prediction context. The embedding ensures the minimization of the square error introduced by the watermarking. The proposed transform introduces less distortion than the classical prediction-error expansion for complex predictors such as the median edge detector or the gradient-adjusted predictor. Reversible watermarking algorithms based on the proposed transform are analyzed. Experimental results are provided. [ABSTRACT FROM PUBLISHER]

It is well known that spatial averaging can be realized (in space or frequency domain) using algorithms whose complexity does not scale with the size or shape of the filter. These fast algorithms are generally referred to as constant-time or O(1) algorithms in the image-processing literature. Along with the spatial filter, the edge-preserving bilateral filter involves an additional range kernel. This is used to restrict the averaging to those neighborhood pixels whose intensity are similar or close to that of the pixel of interest. The range kernel operates by acting on the pixel intensities. This makes the averaging process nonlinear and computationally intensive, particularly when the spatial filter is large. In this paper, we show how the O(1) averaging algorithms can be leveraged for realizing the bilateral filter in constant time, by using trigonometric range kernels. This is done by generalizing the idea presented by Porikli, i.e., using polynomial kernels. The class of trigonometric kernels turns out to be sufficiently rich, allowing for the approximation of the standard Gaussian bilateral filter. The attractive feature of our approach is that, for a fixed number of terms, the quality of approximation achieved using trigonometric kernels is much superior to that obtained by Porikli using polynomials. [ABSTRACT FROM AUTHOR]

In this paper, a simple and effective image-magnification algorithm based on intervals is proposed. A low-resolution image is magnified to form a high-resolution image using a block-expanding method. Our proposed method associates each pixel with an interval obtained by a weighted aggregation of the pixels in its neighborhood. From the interval and with a linear K\alpha operator, we obtain the magnified image. Experimental results show that our algorithm provides a magnified image with better quality (peak signal-to-noise ratio) than several existing methods. [ABSTRACT FROM PUBLISHER]

Traditional methods for image downsampling commit to remove the aliasing artifacts. However, the influences on the quality of the image interpolated from the downsampled one are usually neglected. To tackle this problem, in this paper, we propose an interpolation-dependent image downsampling (IDID), where interpolation is hinged to downsampling. Given an interpolation method, the goal of IDID is to obtain a downsampled image that minimizes the sum of square errors between the input image and the one interpolated from the corresponding downsampled image. Utilizing a least squares algorithm, the solution of IDID is derived as the inverse operator of upsampling. We also devise a content-dependent IDID for the interpolation methods with varying interpolation coefficients. Numerous experimental results demonstrate the viability and efficiency of the proposed IDID. [ABSTRACT FROM PUBLISHER]

This paper presents, for the first time, a unified blind method for multi-image super-resolution (MISR or SR), single-image blur deconvolution (SIBD), and multi-image blur deconvolution (MIBD) of low-resolution (LR) images degraded by linear space-invariant (LSI) blur, aliasing, and additive white Gaussian noise (AWGN). The proposed approach is based on alternating minimization (AM) of a new cost function with respect to the unknown high-resolution (HR) image and blurs. The regularization term for the HR image is based upon the Huber-Markov random field (HMRF) model, which is a type of variational integral that exploits the piecewise smooth nature of the HR image. The blur estimation process is supported by an edge-emphasizing smoothing operation, which improves the quality of blur estimates by enhancing strong soft edges toward step edges, while filtering out weak structures. The parameters are updated gradually so that the number of salient edges used for blur estimation increases at each iteration. For better performance, the blur estimation is done in the filter domain rather than the pixel domain, i.e., using the gradients of the LR and HR images. The regularization term for the blur is Gaussian (L2 norm), which allows for fast noniterative optimization in the frequency domain. We accelerate the processing time of SR reconstruction by separating the upsampling and registration processes from the optimization procedure. Simulation results on both synthetic and real-life images (from a novel computational imager) confirm the robustness and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, a new method for supervised hyperspectral data classification is proposed. In particular, the notion of stochastic minimum spanning forest (MSF) is introduced. For a given hyperspectral image, a pixelwise classification is first performed. From this classification map, M marker maps are generated by randomly selecting pixels and labeling them as markers for the construction of MSFs. The next step consists in building an MSF from each of the M marker maps. Finally, all the M realizations are aggregated with a maximum vote decision rule in order to build the final classification map. The proposed method is tested on three different data sets of hyperspectral airborne images with different resolutions and contexts. The influences of the number of markers and of the number of realizations M on the results are investigated in experiments. The performance of the proposed method is compared to several classification techniques (both pixelwise and spectral–spatial) using standard quantitative criteria and visual qualitative evaluation. [ABSTRACT FROM AUTHOR]

It is difficult to directly apply existing binarization approaches to the barcode images captured by mobile device due to their low quality. This paper proposes a novel scheme for the binarization of such images. The barcode and background regions are differentiated by the number of edge pixels in a search window. Unlike existing approaches that center the pixel to be binarized with a window of fixed size, we propose to shift the window center to the nearest edge pixel so that the balance of the number of object and background pixels can be achieved. The window size is adaptive either to the minimum distance to edges or minimum element width in the barcode. The threshold is calculated using the statistics in the window. Our proposed method has demonstrated its capability in handling the nonuniform illumination problem and the size variation of objects. Experimental results conducted on 350 images captured by five mobile phones achieve about 100% of recognition rate in good lighting conditions, and about 95% and 83% in bad lighting conditions. Comparisons made with nine existing binarization methods demonstrate the advancement of our proposed scheme. [ABSTRACT FROM PUBLISHER]

In this paper, we introduce a new demosaicing algorithm that can be used for various sensor images captured by digital cameras equipped with various red–green–blue color filter arrays. Our algorithm enhances the universal demosaicing algorithm of Lukac et al by defining a new spectral interpolation model that exploits not only the information on the color of pixels but also the relative distance between neighboring pixels within an image. Moreover, we include an edge-detection model that makes our algorithm adaptive and reduces the presence of color shifts and artifacts. A series of tests has been made on images of the Kodak database, and our algorithm performs better than the universal demosaicing algorithm with regard to both subjective and objective evaluation. The versatility of our demosaicing algorithm is also highlighted through an application to the issue of color image resampling, and we obtain conclusive experimental results. [ABSTRACT FROM PUBLISHER]

The classic centroidal Voronoi tessellation (CVT) model and its generalizations work quite well at extracting uniformly colored objects, but often fail to handle images with distinct color distribution or strong inhomogeneous intensity. To resolve this problem within the CVT methodology, in this paper we incorporate the information of local variation of colors/intensities and the length of boundaries into the energy functional and develop a new model called the Local Variation and Edge-Weighted Centroidal Voronoi Tessellation (LVEWCVT) for image segmentation. Its mathematical formulation and practical implementations are also discussed and given. We test the LVEWCVT method on various type of segments and also compare it with several state-of-art algorithms using extensive segmentation examples, the results demonstrate excellent performance and competence of the proposed method. [ABSTRACT FROM PUBLISHER]

The problem of creating artifact-free upscaled images appearing sharp and natural to the human observer is probably more interesting and less trivial than it may appear. The solution to the problem, often referred to also as “single-image super-resolution,” is related both to the statistical relationship between low-resolution and high-resolution image sampling and to the human perception of image quality. In many practical applications, simple linear or cubic interpolation algorithms are applied for this task, but the results obtained are not really satisfactory, being affected by relevant artifacts like blurring and jaggies. Several methods have been proposed to obtain better results, involving simple heuristics, edge modeling, or statistical learning. The most powerful ones, however, present a high computational complexity and are not suitable for real-time applications, while fast methods, even if edge adaptive, are not able to provide artifacts-free images. In this paper, we describe a new upscaling method (iterative curvature-based interpolation) based on a two-step grid filling and an iterative correction of the interpolated pixels obtained by minimizing an objective function depending on the second-order directional derivatives of the image intensity. We show that the constraints used to derive the function are related with those applied in another well-known interpolation method, providing good results but computationally heavy (i.e., new edge-directed interpolation (NEDI) refid="ref11"/. The high quality of the images enlarged with the new method is demonstrated with objective and subjective tests, while the computation time is reduced of one to two orders of magnitude with respect to NEDI so that we were able, using a graphics processing unit implementation based on the nVidia Compute Unified Device Architecture technology, to obtain real-time performances. [ABSTRACT FROM AUTHOR]

This paper presents a no-reference image blur metric that is based on the study of human blur perception for varying contrast values. The metric utilizes a probabilistic model to estimate the probability of detecting blur at each edge in the image, and then the information is pooled by computing the cumulative probability of blur detection (CPBD). The performance of the metric is demonstated by comparing it with existing no-reference sharpness/blurriness metrics for various publicly available image databases. [ABSTRACT FROM PUBLISHER]

This paper is concerned with partial differential equation (PDE)-based image denoising for random-valued impulse noise. We introduce the notion of ENI (the abbreviation for “edge pixels, noisy pixels, and interior pixels”) that denotes the number of homogeneous pixels in a local neighborhood and is significantly different for edge pixels, noisy pixels, and interior pixels. We redefine the controlling speed function and the controlling fidelity function to depend on ENI. According to our two controlling functions, the diffusion and fidelity process at edge pixels, noisy pixels, and interior pixels can be selectively carried out. Furthermore, a class of second-order improved and edge-preserving PDE denoising models is proposed based on the two new controlling functions in order to deal with random-valued impulse noise reliably. We demonstrate the performance of the proposed PDEs via application to five standard test images, corrupted by random-valued impulse noise with various noise levels and comparison with the related second-order PDE models and the other special filtering methods for random-valued impulse noise. Our two controlling functions are extended to automatically other PDE models. [ABSTRACT FROM PUBLISHER]

Anatomical structure tracing on cephalograms is a significant way to obtain cephalometric analysis. Cephalometric analysis is divided in two categories, manual and automatic approaches. The manual approach is limited in accuracy and repeatability due to differences in inter- and intra-personal marking. In this paper, we have attempted to develop and test a novel method for automatic localization of craniofacial structures based on the detected edges in the region of interest. Before edge detection of the particular region, the region was filtered by adaptive non local filter for noise removal by keeping the edge information undisturbed. According to the gray-scale feature at the different regions of the cephalograms, modified Canny edge detection algorithm for obtaining tissue contour was proposed. With the application of morphological opening and edge linking approaches, an improved bidirectional contour tracing methodology was proposed by an interactive selection of the starting edge pixels, the tracking process searches repetitively for an edge pixel at the neighborhood of previously searched edge pixel to segment images, and then craniofacial structures are obtained. The effectiveness of the algorithm is demonstrated by the preliminary experimental results obtained with the proposed method. [ABSTRACT FROM PUBLISHER]

Whenever one wants to distinguish, recognize, and/or measure objects (connected components) in binary images, labeling is required. This paper presents two efficient label-equivalence-based connected-component labeling algorithms for 3-D binary images. One is voxel based and the other is run based. For the voxel-based one, we present an efficient method of deciding the order for checking voxels in the mask. For the run-based one, instead of assigning each foreground voxel, we assign each run a provisional label. Moreover, we use run data to label foreground voxels without scanning any background voxel in the second scan. Experimental results have demonstrated that our voxel-based algorithm is efficient for 3-D binary images with complicated connected components, that our run-based one is efficient for those with simple connected components, and that both are much more efficient than conventional 3-D labeling algorithms. [ABSTRACT FROM PUBLISHER]

A novel ellipse detector based upon edge following is proposed in this paper. The detector models edge connectivity by line segments and exploits these line segments to construct a set of elliptical-arcs. Disconnected elliptical-arcs which describe the same ellipse are identified and grouped together by incrementally finding optimal pairings of elliptical-arcs. We extract hypothetical ellipses of an image by fitting an ellipse to the elliptical-arcs of each group. Finally, a feedback loop is developed to sieve out low confidence hypothetical ellipses and to regenerate a better set of hypothetical ellipses. In this aspect, the proposed algorithm performs self-correction and homes in on “difficult” ellipses. Detailed evaluation on synthetic images shows that the algorithm outperforms existing methods substantially in terms of recall and precision scores under the scenarios of image cluttering, salt-and-pepper noise and partial occlusion. Additionally, we apply the detector on a set of challenging real-world images. Successful detection of ellipses present in these images is demonstrated. We are not aware of any other work that can detect ellipses from such difficult images. Therefore, this work presents a significant contribution towards ellipse detection. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel generic image prior—gradient profile prior, which implies the prior knowledge of natural image gradients. In this prior, the image gradients are represented by gradient profiles, which are 1-D profiles of gradient magnitudes perpendicular to image structures. We model the gradient profiles by a parametric gradient profile model. Using this model, the prior knowledge of the gradient profiles are learned from a large collection of natural images, which are called gradient profile prior. Based on this prior, we propose a gradient field transformation to constrain the gradient fields of the high resolution image and the enhanced image when performing single image super-resolution and sharpness enhancement. With this simple but very effective approach, we are able to produce state-of-the-art results. The reconstructed high resolution images or the enhanced images are sharp while have rare ringing or jaggy artifacts. [ABSTRACT FROM AUTHOR]

This paper presents a multiphase fuzzy region competition model that takes into account spatial and frequency information for image segmentation. In the proposed energy functional, each region is represented by a fuzzy membership function and a data fidelity term that measures the conformity of spatial and frequency data within each region to (generalized) Gaussian densities whose parameters are determined jointly with the segmentation process. Compared with the classical region competition model, our approach gives soft segmentation results via the fuzzy membership functions, and moreover, the use of frequency data provides additional region information that can improve the overall segmentation result. To efficiently solve the minimization of the energy functional, we adopt an alternate minimization procedure and make use of Chambolle's fast duality projection algorithm. We apply the proposed method to synthetic and natural textures as well as real-world natural images. Experimental results show that our proposed method has very promising segmentation performance compared with the current state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

This paper deals with the theory and applications of spatially-variant discrete mathematical morphology. We review and formalize the definition of spatially variant dilation/erosion and opening/closing for binary and gray-level images using exclusively the structuring function, without resorting to complement. This theoretical framework allows to build morphological operators whose structuring elements can locally adapt their shape and orientation across the dominant direction of the structures in the image. The shape and orientation of the structuring element at each pixel are extracted from the image under study: the orientation is given by means of a diffusion process of the average square gradient field, which regularizes and extends the orientation information from the edges of the objects to the homogeneous areas of the image; and the shape of the orientated structuring elements can be linear or it can be given by the distance to relevant edges of the objects. The proposed filters are used on binary and gray-level images for enhancement of anisotropic features such as coherent, flow-like structures. Results of spatially-variant erosions/dilations and openings/closings-based filters prove the validity of this theoretical sound and novel approach. [ABSTRACT FROM AUTHOR]

Aliasing is a common artifact in low-resolution (LR) images generated by a downsampling process. Recovering the original high-resolution image from its LR counterpart while at the same time removing the aliasing artifacts is a challenging image interpolation problem. Since a natural image normally contains redundant similar patches, the values of missing pixels can be available at texture-relevant LR pixels. Based on this, we propose an iterative multiscale semilocal interpolation method that can effectively address the aliasing problem. The proposed method estimates each missing pixel from a set of texture-relevant semilocal LR pixels with the texture similarity iteratively measured from a sequence of patches of varying sizes. Specifically, in each iteration, top texture-relevant LR pixels are used to construct a data fidelity term in a maximum a posteriori estimation, and a bilateral total variation is used as the regularization term. Experimental results compared with existing interpolation methods demonstrate that our method can not only substantially alleviate the aliasing problem but also produce better results across a wide range of scenes both in terms of quantitative evaluation and subjective visual quality. [ABSTRACT FROM PUBLISHER]

For economic reasons, most digital cameras use color filter arrays instead of beam splitters to capture image data. As a result of this, only one of the required three color samples becomes available at each pixel location and the other two need to be interpolated. This process is called Color Filter Array (CFA) interpolation or demosaicing. Many demosaicing algorithms have been introduced over the years to improve subjective and objective interpolation quality. We propose an orientation-free edge strength filter and apply it to the demosaicing problem. Edge strength filter output is utilized both to improve the initial green channel interpolation and to apply the constant color difference rule adaptively. This simple edge directed method yields visually pleasing results with high CPSNR. [ABSTRACT FROM PUBLISHER]

In computer vision, there are many applications, where it is advantageous to process an image in the gradient domain and then reintegrate the gradient field: important examples include shadow removal, lightness calculation, and data fusion. A serious problem with this approach is that the reconstruction step often introduces artefacts—commonly, smoothed and smeared edges—to the recovered image. This is a result of the inherent ill-posedness of reintegrating a nonintegrable field. Artefacts can be diminished but not removed, by using complex to highly complex reintegration techniques. Here, we present a remarkably simple (and on the face of it naive) algorithm for reconstructing gradient fields. Suppose we start with a multichannel original, and from it derive a (possibly one of many) 1-D gradient field; for many applications, the derived gradient field will be nonintegrable. Here, we propose a lookup-table-based map relating the multichannel original to a reconstructed scalar output image, whose gradient best matches the target gradient field. The idea, at base, is that if we learn how to map the gradients of the multichannel original onto the desired output gradient, and then using the lookup table (LUT) constraint, we effectively derive the mapping from the multichannel input to the desired, reintegrated, image output. While this map could take a variety of forms, here we derive the best map from the multichannel gradient as a (nonlinear) function of the input to each of the target scalar gradients. In this framework, reconstruction is a simple equation-solving exercise of low dimensionality. One obvious application of our method is to the image-fusion problem, e.g., the problem of converting a color or higher-D image into grayscale. We will show, through extensive experiments and complementary theoretical arguments, that our straightforward method preserves the target contrast as well as do complex previous reintegration methods, but without artefacts, and with a substantially cheaper computational cost. Finally, we demonstrate the generality of the method by applying it to gradient field reconstruction in an additional area, the shading recovery problem. [ABSTRACT FROM AUTHOR]

We present the first ratio-based image segmentation method that allows imposing curvature regularity of the region boundary. Our approach is a generalization of the ratio framework pioneered by Jermyn and Ishikawa so as to allow penalty functions that take into account the local curvature of the curve. The key idea is to cast the segmentation problem as one of finding cyclic paths of minimal ratio in a graph where each graph node represents a line segment. Among ratios whose discrete counterparts can be globally minimized with our approach, we focus in particular on the elastic ratio $ \displaystyle\int_0^\cal L(C)\nabla I(C(s))\cdot(C^\prime(s))^\perp\, ds\over \displaystyle \nu\, \cal L(C)+\int_0^\cal L(C)\vert \kappa_C(s)\vert ^q\,ds $that depends, given an image I, on the oriented boundary C of the segmented region candidate. Minimizing this ratio amounts to finding a curve, neither small nor too curvy, through which the brightness flux is maximal. We prove the existence of minimizers for this criterion among continuous curves with mild regularity assumptions. We also prove that the discrete minimizers provided by our graph-based algorithm converge, as the resolution increases, to continuous minimizers. In contrast to most existing segmentation methods with computable and meaningful, i.e., nondegenerate, global optima, the proposed approach is fully unsupervised in the sense that it does not require any kind of user input such as seed nodes. Numerical experiments demonstrate that curvature regularity allows substantial improvement of the quality of segmentations. Furthermore, our results allow drawing conclusions about global optima of a parameterization-independent version of the snakes functional: the proposed algorithm allows determining parameter values where the functional has a meaningful solution and simultaneously provides the corresponding global solution. [ABSTRACT FROM PUBLISHER]

We introduce a novel image zooming algorithm, called the curvature interpolation method (CIM), which is partial-differential-equation (PDE)-based and easy to implement. In order to minimize artifacts arising in image interpolation such as image blur and the checkerboard effect, the CIM first evaluates the curvature of the low-resolution image. After interpolating the curvature to the high-resolution image domain, the CIM constructs the high-resolution image by solving a linearized curvature equation, incorporating the interpolated curvature as an explicit driving force. It has been numerically verified that the new zooming method can produce clear images of sharp edges which are already denoised and superior to those obtained from linear methods and PDE-based methods of no curvature information. Various results are given to prove effectiveness and reliability of the new method. [ABSTRACT FROM AUTHOR]

The authors present a computationally efficient technique for maximum a posteriori (MAP) estimation of images in the presence of both blur and noise. The image is divided into statistically independent regions. Each region is modelled with a WSS Gaussian prior. Classical Wiener filter theory is used to generate a set of convex sets in the solution space, with the solution to the MAP estimation problem lying at the intersection of these sets. The proposed algorithm uses an underlying segmentation of the image, and a means of determining the segmentation and refining it are described. The algorithm is suitable for a range of image restoration problems, as it provides a computationally efficient means to deal with the shortcomings of Wiener filtering without sacrificing the computational simplicity of the filtering approach. The algorithm is also of interest from a theoretical viewpoint as it provides a continuum of solutions between Wiener filtering and Inverse filtering depending upon the segmentation used. We do not attempt to show here that the proposed method is the best general approach to the image reconstruction problem. However, related work referenced herein shows excellent performance in the specific problem of demosaicing. [ABSTRACT FROM PUBLISHER]

Displays and content have various resolutions and aspect ratios, requiring an image downscaler to adaptively reduce the image resolution. However, research on downscaling has garnered less attention than upscaling, including super-resolution. In practical display systems, simple interpolation, such as a bicubic filter that cannot preserve image details well, is still widely used for image downscaling rather than frame optimization-based or learning-based methods because of following reasons: frame optimization-based methods can effectively preserve image details after downscaling but are difficult to implement due to hardware costs. Learning-based methods have not been developed because defining a target downscaled image for training is difficult and training all downscaling factors is impossible. We propose a novel kernel-learning-based image downscaler to improve detail-preservation quality while supporting arbitrary downscaling factors using simple linear mapping. For this, a method to produce the ideal target downscaling result considering aliasing artifacts and detail preservation after downscaling is proposed. Then, we propose a training technique using the positional relationship between input and output pixels and a hierarchical region analysis to reproduce target images through simple kernel-based linear mapping. Lastly, a kernel-sharing technique is proposed to generate downscaling results for downscaling factors using a minimum number of trained kernels. In the simulation results, the proposed method demonstrated excellent edge preservation by improving the recall, precision, and F1 score, measuring the edge consistency between input and downscaled images, by up to 0.141, 0.079, 0.053, respectively, compared to benchmark methods. In a paired-comparison-based user study, the proposed method obtained the highest preference among benchmark methods using simple operations. [ABSTRACT FROM AUTHOR]

We introduce a new edge-directed interpolator based on locally defined, straight line approximations of image isophotes. Spatial derivatives of image intensity are used to describe the principal behavior of pixel-intersecting isophotes in terms of their slopes. The slopes are determined by inverting a tridiagonal matrix and are forced to vary linearly from pixel-to-pixel within segments. Image resizing is performed by interpolating along the approximated isophotes. The proposed method can accommodate arbitrary scaling factors, provides state-of-the-art results in terms of PSNR as well as other quantitative visual quality metrics, and has the advantage of reduced computational complexity that is directly proportional to the number of pixels. [ABSTRACT FROM AUTHOR]

We propose a simple preconditioning method for accelerating the solution of edge-preserving image super-resolution (SR) problems in which a linear shift-invariant point spread function is employed. Our technique involves reordering the high-resolution (HR) pixels in a similar manner to what is done in preconditioning methods for quadratic SR formulations. However, due to the edge preserving requirements, the Hessian matrix of the cost function varies during the minimization process. We develop an efficient update scheme for the preconditioner in order to cope with this situation. Unlike some other acceleration strategies that round the displacement values between the low-resolution (LR) images on the HR grid, the proposed method does not sacrifice the optimality of the observation model. In addition, we describe a technique for preconditioning SR problems involving rational magnification factors. The use of such factors is motivated in part by the fact that, under certain circumstances, optimal SR zooms are nonintegers. We show that, by reordering the pixels of the LR images, the structure of the problem to solve is modified in such a way that preconditioners based on circulant operators can be used. [ABSTRACT FROM PUBLISHER]

We propose a new Bayesian network (BN) model for both automatic and interactive image segmentation. A multilayer BN is constructed from an oversegmentation to model the statistical dependencies among superpixel regions, edge segments, vertices, and their measurements. The BN also incorporates various local constraints to further restrain the relationships among these image entities. Given the BN model and various image measurements, belief propagation is performed to update the probability of each node. Image segmentation is generated by the most probable explanation inference of the true states of both region and edge nodes from the updated BN. Besides the automatic image segmentation, the proposed model can also be used for interactive image segmentation. While existing interactive segmentation (IS) approaches often passively depend on the user to provide exact intervention, we propose a new active input selection approach to provide suggestions for the user's intervention. Such intervention can be conveniently incorporated into the BN model to perform actively IS. We evaluate the proposed model on both the Weizmann dataset and VOC2006 cow images. The results demonstrate that the BN model can be used for automatic segmentation, and more importantly, for actively IS. The experiments also show that the IS with active input selection can improve both the overall segmentation accuracy and efficiency over the IS with passive intervention. [ABSTRACT FROM PUBLISHER]

We propose a general purpose image segmentation framework, which involves feature extraction and classification in feature space, followed by flooding and merging in spatial domain. Region growing is based on the computed local measurements and distances from the distribution of features describing the different classes. Using the properties of the label dependent distances spatial coherence is ensured, since the image features are described globally. The distribution of the features for the different classes are obtained by block-wise unsupervised clustering based on the construction of the minimum spanning tree of the blocks' grid using the Mallows distance and the equipartition of the resulting tree. The final clustering is obtained by using the k-centroids algorithm. With high probability and under topological constraints, connected components of the maximum likelihood classification map are used to compute a map of initially labelled pixels. An efficient flooding algorithm is introduced, namely, Priority Multi-Class Flooding Algorithm (PMCFA), that assign pixels to labels using Bayesian dissimilarity criteria. A new region merging method, which incorporates boundary information, is introduced for obtaining the final segmentation map. Therefore, the merging stage is based on region features and edge localization. Segmentation results on the Berkeley benchmark data set demonstrate the effectiveness of the proposed methods. [ABSTRACT FROM PUBLISHER]