Search

Heterogeneous transfer learning has recently gained much attention as a new machine learning paradigm in which the knowledge can be transferred from source domains to target domains in different feature spaces. Existing works usually assume that source domains can provide accurate and useful knowledge to be transferred to target domains for learning. In practice, there may be noise appearing in given source (text) and target (image) domains data, and thus, the performance of transfer learning can be seriously degraded. In this paper, we propose a robust and non-negative collective matrix factorization model to handle noise in text-to-image transfer learning, and make a reliable bridge to transfer accurate and useful knowledge from the text domain to the image domain. The proposed matrix factorization model can be solved by an efficient iterative method, and the convergence of the iterative method can be shown. Extensive experiments on real data sets suggest that the proposed model is able to effectively perform transfer learning in noisy text and image domains, and it is superior to the popular existing methods for text-to-image transfer learning. [ABSTRACT FROM AUTHOR]

Presents information on a study which developed an adaptive genetic algorithm to find the ridges in paper fingerprints. Formulation of the ridge estimation problem; Problem solution using the proposed adaptive genetic algorithm; Results and conclusions.

A quick-response code (QR code) is a two-dimensional code akin to a barcode that encodes a message of limited length. In this paper, we present a variant of QR code, a two-layer QR code. Its two-layer structure can display two alternative messages when scanned from two different directions. We propose a method to generate such two-layer QR codes encoding two given messages in a few seconds. We also demonstrate the robustness of our method on both synthetic and fabricated examples. All source code will be made publicly available (https://github.com/yuantailing/two-layer-qrcode). [ABSTRACT FROM AUTHOR]

Curvilinear structure restoration in image processing procedures is a difficult task, which can be compounded when these structures are thin, i.e., when their smallest dimension is close to the resolution of the sensor. Many recent restoration methods involve considering a local gradient-based regularization term as prior, assuming gradient sparsity. An isotropic gradient operator is typically not suitable for thin curvilinear structures, since gradients are not sparse for these. In this paper, we propose a mixed gradient operator that combines a standard gradient in the isotropic image regions, and a directional gradient in the regions where specific orientations are likely. In particular, such information can be provided by curvilinear structure detectors (e.g., RORPO or Frangi filters). Our proposed mixed gradient operator, that can be viewed as a companion tool of such detectors, is proposed in a discrete framework and its formulation/computation holds in any dimension; in other words, it is valid in $\mathbb Z^{n}$ , ${n}\geq 1$. We show how this mixed gradient can be used to construct image priors that take edge orientation, as well as intensity, into account, and then involved in various image processing tasks while preserving curvilinear structures. The experiments carried out on 2D, 3D, real, and synthetic images illustrate the relevance of the proposed gradient, and its use in variational frameworks for both denoising and segmentation tasks. [ABSTRACT FROM AUTHOR]

Hypergraph learning has been widely exploited in various image processing applications, due to its advantages in modeling the high-order information. Its efficacy highly depends on building an informative hypergraph structure to accurately and robustly formulate the underlying data correlation. However, the existing hypergraph learning methods are sensitive to non-Gaussian noise, which hurts the corresponding performance. In this paper, we present a noise-resistant hypergraph learning model, which provides superior robustness against various non-Gaussian noises. In particular, our model adopts low-rank representation to construct a hypergraph, which captures the globally linear data structure as well as preserving the grouping effect of highly correlated data. We further introduce a correntropy-induced local metric to measure the reconstruction errors, which is particularly robust to non-Gaussian noises. Finally, the Frobenious-norm-based regularization is proposed to combine with the low-rank regularizer, which enables our model to regularize the singular values of the coefficient matrix. By such, the non-zero coefficients are selected to generate a hyperedge set as well as the hyperedge weights. We have evaluated the proposed hypergraph model in the tasks of image clustering and semi-supervised image classification. Quantitatively, our scheme significantly enhances the performance of the state-of-the-art hypergraph models on several benchmark data sets. [ABSTRACT FROM AUTHOR]

Most variational formulations for structure-texture image decomposition force the structure images to have small norm in some functional spaces and to share a common notion of edges, i.e., large-gradients or large-intensity differences. However, such a definition makes it difficult to distinguish structure edges from oscillations that have fine spatial scale but high contrast. In this paper, we introduce a new model by learning deep variational priors for structure images without explicit training data. An alternating direction method of a multiplier algorithm and its modular structure are adopted to plug deep variational priors into an iterative smoothing process. The central observations are that convolution neural networks (CNNs) can replace the total variation prior, and are indeed powerful to capture the natures of structure and texture. We show that our learned priors using CNNs successfully differentiate high-amplitude details from structure edges, and avoid halo artifacts. Different from previous data-driven smoothing schemes, our formulation provides another degree of freedom to produce continuous smoothing effects. Experimental results demonstrate the effectiveness of our approach on various computational photography and image processing applications, including texture removal, detail manipulation, HDR tone-mapping, and non-photorealistic abstraction. [ABSTRACT FROM AUTHOR]

In daily photography, it is common to capture images in the reflection of an unwanted scene. This circumstance arises frequently when imaging through a semi-reflecting material such as glass. The unwanted reflection will affect the visibility of the background image and introduce ambiguity that perturbs the subsequent analysis on the image. It is a very challenging task to remove the reflection of an image since the problem is severely ill-posed. In this paper, we propose a novel algorithm to solve the reflection removal problem based on light field (LF) imaging. For the proposed algorithm, we first show that the strong gradient points of an LF epipolar plane image (EPI) are preserved after adding to the EPI of another LF image. We can then make use of these strong gradient points to give a rough estimation of the background and reflection. Rather than assuming that the background and reflection have absolutely different disparity ranges, we propose a sandwich layer model to allow them to have common disparities, which is more realistic in practical situations. Then, the background image is refined by recovering the components in the shared disparity range using an iterative enhancement process. Our experimental results show that the proposed algorithm achieves superior performance over traditional approaches both qualitatively and quantitatively. These results verify the robustness of the proposed algorithm when working with images captured from real-life scenes. [ABSTRACT FROM AUTHOR]

Random walk is a popular and efficient algorithm for image segmentation, especially for extracting regions of interest (ROIs). One difficulty with the random walk algorithm is the requirement for solving a huge sparse linear system when applied to large images. Another limitation is its sensitivity to seeds distribution, i.e., the segmentation result depends on the number of seeds as well as their placement, which puts a burden on users. In this paper, we first propose a continuous random walk model with explicit coherence regularization (CRWCR) for the extracted ROI, which helps to reduce the seeds sensitivity, so as to reduce the user interactions. Then, a very efficient algorithm to solve the CRWCR model will be developed, which helps to remove the difficulty of solving huge linear systems. Our algorithm consists of two stages: initialization by performing one-dimensional random walk sweeping based on user-provided seeds, followed by the alternating direction scheme, i.e., Peaceman–Rachford scheme for further correction. The first stage aims to provide a good initial guess for the ROI, and it is very fast since we just solve a limited number of one-dimensional random walk problems. Then, this initial guess is evolved to the ideal solution by applying the second stage, which should also be very efficient since it fits well for GPU computing, and 10 iterations are usually sufficient for convergence. Numerical experiments are provided to validate the proposed model as well as the efficiency of the two-stage algorithm. [ABSTRACT FROM AUTHOR]

Nonlocal texture similarity and local intensity smoothness are both essential for solving most image inpainting problems. In this paper, we propose a novel image inpainting algorithm that is capable of reproducing the underlying textural details using a nonlocal texture measure and also smoothing pixel intensity seamlessly in order to achieve natural-looking inpainted images. For matching texture, we propose a Gaussian-weighted nonlocal texture similarity measure to obtain multiple candidate patches for each target patch. To compute the pixel intensity, we apply the $\alpha $ -trimmed mean filter to the candidate patches to inpaint the target patch pixel-by-pixel. The proposed algorithm is compared with four current image inpainting algorithms under different scenarios, including object removal, texture synthesis, and error concealment. Experimental results show that the proposed algorithm outperforms the existing algorithms when inpainting large missing regions in images with texture and geometric structures. [ABSTRACT FROM AUTHOR]

Recognizing objects from simultaneously sensed photometric (RGB) and depth channels is a fundamental yet practical problem in many machine vision applications, such as robot grasping and autonomous driving. In this paper, we address this problem by developing a cross-modal attentional context (CMAC) learning framework, which enables the full exploitation of the context information from both RGB and depth data. Compared to existing RGB-D object detection frameworks, our approach has several appealing properties. First, it consists of an attention-based global context model for exploiting adaptive contextual information and incorporating this information into a region-based CNN (e.g., fast RCNN) framework to achieve improved object detection performance. Second, our CMAC framework further contains a fine-grained object part attention module to harness multiple discriminative object parts inside each possible object region for superior local feature representation. While greatly improving the accuracy of RGB-D object detection, the effective cross-modal information fusion as well as attentional context modeling in our proposed model provide an interpretable visualization scheme. Experimental results demonstrate that the proposed method significantly improves upon the state of the art on all public benchmarks. [ABSTRACT FROM AUTHOR]

Depth information is being widely used in many real-world applications. However, due to the limitation of depth sensing technology, the captured depth map in practice usually has much lower resolution than that of color image counterpart. In this paper, we propose to combine the internal smoothness prior and external gradient consistency constraint in graph domain for depth super-resolution. On one hand, a new graph Laplacian regularizer is proposed to preserve the inherent piecewise smooth characteristic of depth, which has desirable filtering properties. A specific weight matrix of the respect graph is defined to make full use of information of both depth and the corresponding guidance image. On the other hand, inspired by an observation that the gradient of depth is small except at edge separating regions, we introduce a graph gradient consistency constraint to enforce that the graph gradient of depth is close to the thresholded gradient of guidance. We reinterpret the gradient thresholding model as variational optimization with sparsity constraint. In this way, we remedy the problem of structure discrepancy between depth and guidance. Finally, the internal and external regularizations are casted into a unified optimization framework, which can be efficiently addressed by ADMM. Experimental results demonstrate that our method outperforms the state-of-the-art with respect to both objective and subjective quality evaluations. [ABSTRACT FROM AUTHOR]

Dilated convolutions support expanding receptive field without parameter exploration or resolution loss, which turn out to be suitable for pixel-level prediction problems. In this paper, we propose multiscale single image super-resolution (SR) based on dilated convolutions. We adopt dilated convolutions to expand the receptive field size without incurring additional computational complexity. We mix standard convolutions and dilated convolutions in each layer, called mixed convolutions, i.e., in the mixed convolutional layer, and the feature extracted by dilated convolutions and standard convolutions are concatenated. We theoretically analyze the receptive field and intensity of mixed convolutions to discover their role in SR. Mixed convolutions remove blind spots and capture the correlation between low-resolution (LR) and high-resolution (HR) image pairs successfully, thus achieving good generalization ability. We verify those properties of mixed convolutions by training 5-layer and 10-layer networks. We also train a 20-layer deep network to compare the performance of the proposed method with those of the state-of-the-art ones. Moreover, we jointly learn maps with different scales from a LR image to its HR one in a single network. Experimental results demonstrate that the proposed method outperforms the state-of-the-art ones in terms of PSNR and SSIM, especially for a large-scale factor. [ABSTRACT FROM AUTHOR]

In this paper, we propose a Weighted Local sparse representation based Depth Image Super-Resolution (WLDISR) schemes aiming at improving the Virtual View Image (VVI) quality of 3D video system. Different from color images, depth images are mainly used to provide geometrical information in synthesizing VVI. Due to the view synthesis characteristics difference between textural structures and smooth regions of depth images, we divide the depth images into edge and smooth patches and learn two local dictionaries, respectively. Meanwhile, the weight term is derived and incorporated explicitly in the cost function to denote different importance of edge structures and smooth regions to the VVI quality. Then, local sparse representation and weighted sparse representation are jointly used in both dictionary learning and reconstruction phases in depth image super-resolution. Based on different optimizations on learning and reconstruction modules, three WLDISR schemes, WLDISR-D, WLDISR-R, and WLDISR-ALL, are proposed. Experimental results on 3D sequences demonstrate that the proposed WLDISR-D, WLDISR-R, and WLDISR-ALL schemes can achieve more than 1.9-, 2.03-, and 2.16-dB gains on average, respectively, in terms of the VVIs’ quality, as compared with the state-of-the-art schemes. In addition, the visual quality of VVIs is also improved. [ABSTRACT FROM AUTHOR]

Many recent ultrasound image processing methods are based on patch comparison, such as filtering and segmentation. Identifying similar patches in noise-corrupted images is a key factor for the performance of these methods. While the Euclidean distance is ideal to handle the patch comparison under additive Gaussian noise, finding good measures to compare patches corrupted by multiplicative noise is still an open research. In this paper, we deduce several new geodesic distances, arising from parametric probabilistic spaces, and suggest them as similarity measures to process RF and log-compressed ultrasound images in patch-based methods. We provide practical examples using these measures in the fields of ultrasound image filtering and segmentation, with results that confirm the potential of the technique. [ABSTRACT FROM AUTHOR]

Recent advances in object detection have led to the development of segmentation by detection approaches that integrate top-down geometric priors for multiclass object segmentation. A key yet under-addressed issue in utilizing top-down cues for the problem of multiclass object segmentation by detection is efficiently generating robust and accurate geometric priors. In this paper, we propose a random geometric prior forest scheme to obtain object-adaptive geometric priors efficiently and robustly. In the scheme, a testing object first searches for training neighbors with similar geometries using the random geometric prior forest, and then the geometry of the testing object is reconstructed by linearly combining the geometries of its neighbors. Our scheme enjoys several favorable properties when compared with conventional methods. First, it is robust and very fast because its inference does not suffer from bad initializations, poor local minimums or complex optimization. Second, the figure/ground geometries of training samples are utilized in a multitask manner. Third, our scheme is object-adaptive but does not require the labeling of parts or poselets, and thus, it is quite easy to implement. To demonstrate the effectiveness of the proposed scheme, we integrate the obtained top-down geometric priors with conventional bottom-up color cues in the frame of graph cut. The proposed random geometric prior forest achieves the best segmentation results of all of the methods tested on VOC2010/2012 and is 90 times faster than the current state-of-the-art method. [ABSTRACT FROM PUBLISHER]

This paper advocates a novel multiscale, structure-sensitive saliency detection method, which can distinguish multilevel, reliable saliency from various natural pictures in a robust and versatile way. One key challenge for saliency detection is to guarantee the entire salient object being characterized differently from nonsalient background. To tackle this, our strategy is to design a structure-aware descriptor based on the intrinsic biharmonic distance metric. One benefit of introducing this descriptor is its ability to simultaneously integrate local and global structure information, which is extremely valuable for separating the salient object from nonsalient background in a multiscale sense. Upon devising such powerful shape descriptor, the remaining challenge is to capture the saliency to make sure that salient subparts actually stand out among all possible candidates. Toward this goal, we conduct multilevel low-rank and sparse analysis in the intrinsic feature space spanned by the shape descriptors defined on over-segmented super-pixels. Since the low-rank property emphasizes much more on stronger similarities among super-pixels, we naturally obtain a scale space along the rank dimension in this way. Multiscale saliency can be obtained by simply computing differences among the low-rank components across the rank scale. We conduct extensive experiments on some public benchmarks, and make comprehensive, quantitative evaluation between our method and existing state-of-the-art techniques. All the results demonstrate the superiority of our method in accuracy, reliability, robustness, and versatility. [ABSTRACT FROM AUTHOR]

Texts in natural scenes carry critical semantic clues for understanding images. When capturing natural scene images, especially by handheld cameras, a common artifact, i.e., blur, frequently happens. To improve the visual quality of such images, deblurring techniques are desired, which also play an important role in character recognition and image understanding. In this paper, we study the problem of recovering the clear scene text by exploiting the text field characteristics. A series of text-specific multiscale dictionaries (TMD) and a natural scene dictionary is learned for separately modeling the priors on the text and nontext fields. The TMD-based text field reconstruction helps to deal with the different scales of strings in a blurry image effectively. Furthermore, an adaptive version of nonuniform deblurring method is proposed to efficiently solve the real-world spatially varying problem. Dictionary learning allows more flexible modeling with respect to the text field property, and the combination with the nonuniform method is more appropriate in real situations where blur kernel sizes are depth dependent. Experimental results show that the proposed method achieves the deblurring results with better visual quality than the state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a object alignment method that detects the landmarks of an object in 2D images. In the regression forests (RFs) framework, observations (patches) that are extracted at several image locations cast votes for the localization of several landmarks. We propose to refine the votes before accumulating them into the Hough space, by sieving and/or aggregating. In order to filter out false positive votes, we pass them through several sieves, each associated with a discrete or continuous latent variable. The sieves filter out votes that are not consistent with the latent variable in question, something that implicitly enforces global constraints. In order to aggregate the votes when necessary, we adjusts on-the-fly a proximity threshold by applying a classifier on middle-level features extracted from voting maps for the object landmark in question. Moreover, our method is able to predict the unreliability of an individual object landmark. This information can be useful for subsequent object analysis like object recognition. Our contributions are validated for two object alignment tasks, face alignment and car alignment, on data sets with challenging images collected in the wild, i.e., the Labeled Face in the Wild, the Annotated Facial Landmarks in the Wild, and the street scene car data set. We show that with the proposed approach, and without explicitly introducing shape models, we obtain performance superior or close to the state of the art for both tasks. [ABSTRACT FROM AUTHOR]

In a setup where camera measurements are used to estimate 3D egomotion in an extended Kalman filter (EKF) framework, it is well-known that inertial sensors (i.e., accelerometers and gyroscopes) are especially useful when the camera undergoes fast motion. Inertial sensor data can be fused at the EKF with the camera measurements in either the correction stage (as measurement inputs) or the prediction stage (as control inputs). In general, only one type of inertial sensor is employed in the EKF in the literature, or when both are employed they are both fused in the same stage. In this paper, we provide an extensive performance comparison of every possible combination of fusing accelerometer and gyroscope data as control or measurement inputs using the same data set collected at different motion speeds. In particular, we compare the performances of different approaches based on 3D pose errors, in addition to camera reprojection errors commonly found in the literature, which provides further insight into the strengths and weaknesses of different approaches. We show using both simulated and real data that it is always better to fuse both sensors in the measurement stage and that in particular, accelerometer helps more with the 3D position tracking accuracy, whereas gyroscope helps more with the 3D orientation tracking accuracy. We also propose a simulated data generation method, which is beneficial for the design and validation of tracking algorithms involving both camera and inertial measurement unit measurements in general. [ABSTRACT FROM AUTHOR]

Multibiometrics can provide higher identification accuracy than single biometrics, so it is more suitable for some real-world personal identification applications that need high-standard security. Among various biometrics technologies, palmprint identification has received much attention because of its good performance. Combining the left and right palmprint images to perform multibiometrics is easy to implement and can obtain better results. However, previous studies did not explore this issue in depth. In this paper, we proposed a novel framework to perform multibiometrics by comprehensively combining the left and right palmprint images. This framework integrated three kinds of scores generated from the left and right palmprint images to perform matching score-level fusion. The first two kinds of scores were, respectively, generated from the left and right palmprint images and can be obtained by any palmprint identification method, whereas the third kind of score was obtained using a specialized algorithm proposed in this paper. As the proposed algorithm carefully takes the nature of the left and right palmprint images into account, it can properly exploit the similarity of the left and right palmprints of the same subject. Moreover, the proposed weighted fusion scheme allowed perfect identification performance to be obtained in comparison with previous palmprint identification methods. [ABSTRACT FROM AUTHOR]

Lighting is a key factor in creating impressive artistic portraits. In this paper, we propose to analyze portrait lighting by learning templates of lighting styles. Inspired by the experience of artists, we first define several novel features that describe the local contrasts in various face regions. The most informative features are then selected with a stepwise feature pursuit algorithm to derive the templates of various lighting styles. After that, the matching scores that measure the similarity between a testing portrait and those templates are calculated for lighting style classification. Furthermore, we train a regression model by the subjective scores and the feature responses of a template to predict the score of a portrait lighting quality. Based on the templates, a novel face illumination descriptor is defined to measure the difference between two portrait lightings. Experimental results show that the learned templates can well describe the lighting styles, whereas the proposed approach can assess the lighting quality of artistic portraits as human being does. [ABSTRACT FROM AUTHOR]

In this paper, we address the total variation (TV)-based nonlinear image restoration problems. In nonlinear image restoration problems, an original image is corrupted by a spatiallyinvariant blur, the build-in nonlinearity in imaging system, and the additive Gaussian white noise. We study the objective function consisting of the nonlinear least squares data-fitting term and the TV regularization term of the restored image. By making use of the structure of the objective function, an efficient alternating direction method of multipliers can be developed for solving the proposed model. The convergence of the numerical scheme is also studied. Numerical examples, including nonlinear image restoration and high-dynamic range imaging are reported to demonstrate the effectiveness of the proposed model and the efficiency of the proposed numerical scheme. [ABSTRACT FROM AUTHOR]

Salient contour detection can benefit from the integration of both contour cues and region cues. However, this task is difficult due to different nature of region representations and contour representations. To solve this problem, this paper proposes an energy minimization framework based on winding number constraints. In this framework, both region cues, such as color/texture homogeneity, and contour cues, such as local contrast and continuity, are represented in a joint objective function, which has both region and contour labels. The key problem is how to design constraints that ensure the topological consistency of the two kinds of labels. Our technique is based on the topological concept of winding number. Using a fast method for winding number computation, a small number of linear constraints are derived to ensure label consistency. Our method is instantiated by ratio-based energy functions. By successfully integrating both region and contour cues, our method shows advantages over competitive methods. Our method is extended to incorporate user interaction, which leads to further improvements. [ABSTRACT FROM AUTHOR]

This paper presents a fast and robust level set method for image segmentation. To enhance the robustness against noise, we embed a Markov random field (MRF) energy function to the conventional level set energy function. This MRF energy function builds the correlation of a pixel with its neighbors and encourages them to fall into the same region. To obtain a fast implementation of the MRF embedded level set model, we explore algebraic multigrid (AMG) and sparse field method (SFM) to increase the time step and decrease the computation domain, respectively. Both AMG and SFM can be conducted in a parallel fashion, which facilitates the processing of our method for big image databases. By comparing the proposed fast and robust level set method with the standard level set method and its popular variants on noisy synthetic images, synthetic aperture radar (SAR) images, medical images, and natural images, we comprehensively demonstrate the new method is robust against various kinds of noises. In particular, the new level set method can segment an image of size $500 \times 500$ within 3 s on MATLAB R2010b installed in a computer with 3.30-GHz CPU and 4-GB memory. [ABSTRACT FROM AUTHOR]

Example learning-based single image super-resolution (SR) is a promising method for reconstructing a high-resolution (HR) image from a single-input low-resolution (LR) image. Lots of popular SR approaches are more likely either time-or space-intensive, which limit their practical applications. Hence, some research has focused on a subspace view and delivered state-of-the-art results. In this paper, we utilize an effective way with mixture prior models to transform the large nonlinear feature space of LR images into a group of linear subspaces in the training phase. In particular, we first partition image patches into several groups by a novel selective patch processing method based on difference curvature of LR patches, and then learning the mixture prior models in each group. Moreover, different prior distributions have various effectiveness in SR, and in this case, we find that student-t prior shows stronger performance than the well-known Gaussian prior. In the testing phase, we adopt the learned multiple mixture prior models to map the input LR features into the appropriate subspace, and finally reconstruct the corresponding HR image in a novel mixed matching way. Experimental results indicate that the proposed approach is both quantitatively and qualitatively superior to some state-of-the-art SR methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose a multilayer surface albedo (MLSA) model to tackle face recognition in bad lighting conditions, especially with reference images in bad lighting conditions. Some previous researches conclude that illumination variations mainly lie in the large-scale features of an image and extract small-scale features in the surface albedo (or surface texture). However, this surface albedo is not robust enough, which still contains some detrimental sharp features. To improve robustness of the surface albedo, MLSA further decomposes it as a linear sum of several detailed layers, to separate and represent features of different scales in a more specific way. Then, the layers are adjusted by separate weights, which are global parameters and selected for only once. A criterion function is developed to select these layer weights with an independent training set. Despite controlled illumination variations, MLSA is also effective to uncontrolled illumination variations, even mixed with other complicated variations (expression, pose, occlusion, and so on). Extensive experiments on four benchmark data sets show that MLSA has good receiver operating characteristic curve and statistical discriminating capability. The refined albedo improves recognition performance, especially with reference images in bad lighting conditions. [ABSTRACT FROM AUTHOR]

Filtered backprojection, one of the most widely used reconstruction methods in tomography, requires a large number of low-noise projections to yield accurate reconstructions. In many applications of tomography, complete projection data of high quality cannot be obtained, because of practical considerations. Algebraic methods tend to handle such problems better, but are computationally more expensive. In this paper, we introduce a new method that improves the filtered backprojection method by using a custom data-dependent filter that minimizes the projection error of the resulting reconstruction. We show that the computational cost of the new method is significantly lower than that of algebraic methods. Experiments on both simulation and experimental data show that the method is able to produce more accurate reconstructions than filtered backprojection based on popular static filters when presented with data with a limited number of projections or statistical noise present. Furthermore, the results show that the method produces reconstructions with similar accuracy to algebraic methods, but is faster at producing them. Finally, we show that the method can be extended to exploit certain forms of prior knowledge, improving reconstruction accuracy in specific cases. [ABSTRACT FROM AUTHOR]

Recent investigations on human vision discover that the retinal image is a landscape or a geometric surface, consisting of features such as ridges and summits. However, most of existing popular local image descriptors in the literature, e.g., scale invariant feature transform (SIFT), histogram of oriented gradient (HOG), DAISY, local binary Patterns (LBP), and gradient location and orientation histogram, only employ the first-order gradient information related to the slope and the elasticity, i.e., length, area, and so on of a surface, and thereby partially characterize the geometric properties of a landscape. In this paper, we introduce a novel and powerful local image descriptor that extracts the histograms of second-order gradients (HSOGs) to capture the curvature related geometric properties of the neural landscape, i.e., cliffs, ridges, summits, valleys, basins, and so on. We conduct comprehensive experiments on three different applications, including the problem of local image matching, visual object categorization, and scene classification. The experimental results clearly evidence the discriminative power of HSOG as compared with its first-order gradient-based counterparts, e.g., SIFT, HOG, DAISY, and center-symmetric LBP, and the complementarity in terms of image representation, demonstrating the effectiveness of the proposed local descriptor. [ABSTRACT FROM AUTHOR]

Depth image compression is important for compact representation of 3D visual data in texture-plus-depth format, where texture and depth maps from one or more viewpoints are encoded and transmitted. A decoder can then synthesize a freely chosen virtual view via depth-image-based rendering using nearby coded texture and depth maps as reference. Further, depth information can be used in other image processing applications beyond view synthesis, such as object identification, segmentation, and so on. In this paper, we leverage on the observation that neighboring pixels of similar depth have similar motion to efficiently encode depth video. Specifically, we divide a depth block containing two zones of distinct values (e.g., foreground and background) into two arbitrarily shaped regions (sub-blocks) along the dividing boundary before performing separate motion prediction (MP). While such arbitrarily shaped sub-block MP can lead to very small prediction residuals (resulting in few bits required for residual coding), it incurs an overhead to transmit the dividing boundaries for sub-block identification at decoder. To minimize this overhead, we first devise a scheme called arithmetic edge coding (AEC) to efficiently code boundaries that divide blocks into sub-blocks. Specifically, we propose to incorporate the boundary geometrical correlation in an adaptive arithmetic coder in the form of a statistical model. Then, we propose two optimization procedures to further improve the edge coding performance of AEC for a given depth image. The first procedure operates within a code block, and allows lossy compression of the detected block boundary to lower the cost of AEC, with an option to augment boundary depth pixel values matching the new boundary, given the augmented pixels do not adversely affect synthesized view distortion. The second procedure operates across code blocks, and systematically identifies blocks along an object contour that should be coded using sub-block MP via a rate-distortion optimized trellis. Experimental results show an average overall bitrate reduction of up to 33% over classical H.264/AVC. [ABSTRACT FROM AUTHOR]

Constructing effective priors is critical to solving ill-posed inverse problems in image processing and computational imaging. Recent works focused on exploiting non-local similarity by grouping similar patches for image modeling, and demonstrated state-of-the-art results in many image restoration applications. However, compared to classic methods based on filtering or sparsity, non-local algorithms are more time-consuming, mainly due to the highly inefficient block matching step, i.e., distance between every pair of overlapping patches needs to be computed. In this work, we propose a novel Self-Convolution operator to exploit image non-local properties in a unified framework. We prove that the proposed Self-Convolution based formulation can generalize the commonly-used non-local modeling methods, as well as produce results equivalent to standard methods, but with much cheaper computation. Furthermore, by applying Self-Convolution, we propose an effective multi-modality image restoration scheme, which is much more efficient than conventional block matching for non-local modeling. Experimental results demonstrate that (1) Self-Convolution with fast Fourier transform implementation can significantly speed up most of the popular non-local image restoration algorithms, with two-fold to nine-fold faster block matching, and (2) the proposed online multi-modality image restoration scheme achieves superior denoising results than competing methods in both efficiency and effectiveness on RGB-NIR images. The code for this work is publicly available at https://github.com/GuoLanqing/Self-Convolution. [ABSTRACT FROM AUTHOR]

Fast and accurate motion estimation takes an important place in many fields of computer vision and image processing. Using Radon transform to compute projections of the images along specified directions is an effective way to show the relationship between the 2D image object and its projections and estimate motions between the images. All existing projection-based motion estimation methods without the use of iteration have a severe defect that only five of the six affine parameters can be estimated. There are some other methods that can estimate the six parameters, but most of them are usually based on a certain iterative framework, which is computationally intensive and sensitively dependent on the initial values. In this paper, a novel method based on Radon transform is proposed to estimate all the six affine parameters directly. The relationship in the projection domain between a pair of images connected by an affine motion is studied and a linear model is established, by which all the six affine parameters can be directively found. The employment of a hierarchical framework can produce more accurate results. The experimental results reveal that the proposed method has a much better performance than the state-of-the-art methods in this field. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization (NMF) has received considerable attention in image processing, computer vision, and patter recognition. An important variant of NMF is nonnegative graph embedding (NGE), which encodes the statistical or geometric information of data in the process of matrix factorization. The NGE offers a general framework for unsupervised/supervised settings. However, NGE-like algorithms often suffer from noisy data, unreliable graphs, and noisy labels, which are commonly encountered in real-world applications. To address these issues, in this paper, we first propose a robust nonnegative graph embedding (RNGE) framework, where the joint sparsity in both graph embedding and data reconstruction endues robustness to undesirable noises. Next, we present a robust seminonnegative graph embedding (RsNGE) framework, which only constrains the coefficient matrix to be nonnegative while places no constraint on the base matrix. This extends the applicable range of RNGE to data which are not nonnegative and endows more discriminative power of the learnt base matrix. The RNGE/RsNGE provides a general formulation such that all the algorithms unified within the graph embedding framework can be easily extended to obtain their robust nonnegative/seminonnegative solutions. Further, we develop elegant multiplicative updating solutions that can solve RNGE/RsNGE efficiently and offer a rigorous convergence analysis. We conduct extensive experiments on four real-world data sets and compare the proposed RNGE/RsNGE to other representative NMF variants and data factorization methods. The experimental results demonstrate the robustness and effectiveness of the proposed approaches. [ABSTRACT FROM AUTHOR]

This paper addresses the noise estimation in the digital domain and proposes a noise estimator based on the step signal model. It is efficient for any distribution of noise because it does not rely only on the smallest amplitudes in the signal or image. The proposed approach uses polarized/directional derivatives and a nonlinear combination of these derivatives to estimate the noise distribution (e.g., Gaussian, Poisson, speckle, etc.). The moments of this measured distribution can be computed and are also calculated theoretically on the basis of noise distribution models. The 1D performances are detailed, and as this paper is mostly dedicated to image processing, a 2D extension is proposed. The 2D performances for several noise distributions and noise models are presented and are compared with selected other methods. [ABSTRACT FROM AUTHOR]

The robust detection of small targets is one of the key techniques in infrared search and tracking applications. A novel small target detection method in a single infrared image is proposed in this paper. Initially, the traditional infrared image model is generalized to a new infrared patch-image model using local patch construction. Then, because of the non-local self-correlation property of the infrared background image, based on the new model small target detection is formulated as an optimization problem of recovering low-rank and sparse matrices, which is effectively solved using stable principle component pursuit. Finally, a simple adaptive segmentation method is used to segment the target image and the segmentation result can be refined by post-processing. Extensive synthetic and real data experiments show that under different clutter backgrounds the proposed method not only works more stably for different target sizes and signal-to-clutter ratio values, but also has better detection performance compared with conventional baseline methods. [ABSTRACT FROM AUTHOR]

This paper addresses the problem of combining multiple radar images of the same scene to produce a more informative composite image. The proposed approach for probabilistic fuzzy logic-based image fusion automatically forms fuzzy membership functions using the Gaussian-Rayleigh mixture distribution. It fuses the input pixel values directly without requiring fuzzification and defuzzification, thereby removing the subjective nature of the existing fuzzy logic methods. In this paper, the proposed approach is applied to through-the-wall radar imaging in urban sensing and evaluated on real multi-view and polarimetric data. Experimental results show that the proposed approach yields improved image contrast and enhances target detection. [ABSTRACT FROM AUTHOR]

This paper proposes a new super-resolution (SR) scheme for landmark images by retrieving correlated web images. Using correlated web images significantly improves the exemplar-based SR. Given a low-resolution (LR) image, we extract local descriptors from its up-sampled version and bundle the descriptors according to their spatial relationship to retrieve correlated high-resolution (HR) images from the web. Though similar in content, the retrieved images are usually taken with different illumination, focal lengths, and shot perspectives, resulting in uncertainty for the HR detail approximation. To solve this problem, we first propose aligning these images to the up-sampled LR image through a global registration, which identifies the corresponding regions in these images and reduces the mismatching. Second, we propose a structure-aware matching criterion and adaptive block sizes to improve the mapping accuracy between LR and HR patches. Finally, these matched HR patches are blended together by solving an energy minimization problem to recover the desired HR image. Experimental results demonstrate that our SR scheme achieves significant improvement compared with four state-of-the-art schemes in terms of both subjective and objective qualities. [ABSTRACT FROM AUTHOR]

Multilevel thresholding amounts to segmenting a gray-level image into several distinct regions. This paper presents a 2D histogram based multilevel thresholding approach to improve the separation between objects. Recent studies indicate that the results obtained with 2D histogram oriented approaches are superior to those obtained with 1D histogram based techniques in the context of bi-level thresholding. Here, a method to incorporate 2D histogram related information for generalized multilevel thresholding is proposed using the maximum Tsallis entropy. Differential evolution (DE), a simple yet efficient evolutionary algorithm of current interest, is employed to improve the computational efficiency of the proposed method. The performance of DE is investigated extensively through comparison with other well-known nature inspired global optimization techniques such as genetic algorithm, particle swarm optimization, artificial bee colony, and simulated annealing. In addition, the outcome of the proposed method is evaluated using a well known benchmark—the Berkley segmentation data set (BSDS300) with 300 distinct images. [ABSTRACT FROM AUTHOR]

In this paper, we present a syntactic string matching approach to solve the frontal face recognition problem. String matching is a powerful partial matching technique, but is not suitable for frontal face recognition due to its requirement of globally sequential representation and the complex nature of human faces, containing discontinuous and non-sequential features. Here, we build a compact syntactic Stringface representation, which is an ensemble of strings. A novel ensemble string matching approach that can perform non-sequential string matching between two Stringfaces is proposed. It is invariant to the sequential order of strings and the direction of each string. The embedded partial matching mechanism enables our method to automatically use every piece of non-occluded region, regardless of shape, in the recognition process. The encouraging results demonstrate the feasibility and effectiveness of using syntactic methods for face recognition from a single exemplar image per person, breaking the barrier that prevents string matching techniques from being used for addressing complex image recognition problems. The proposed method not only achieved significantly better performance in recognizing partially occluded faces, but also showed its ability to perform direct matching between sketch faces and photo faces. [ABSTRACT FROM AUTHOR]

This paper presents a new automatic image annotation algorithm. First, we introduce a new similarity measure between images: compactness. This uses low level visual descriptors for determining the similarity between two images. Compactness shows how close test image features lie to training image feature cluster centers. The measure provides the core for a k-nearest neighbor type image annotation method. Afterward, a formalism for defining different transfer techniques is devised and several label transfer techniques are provided. The method as whole is evaluated on four image annotation benchmarks. The results on these sets validate the accuracy of the approach, which outperforms many state-of-the-art annotation methods. The method presented here requires a simple training process, efficiently combines different feature types and performs better than complex learning algorithms, even in this incipient form. The main contributions of this paper are the usage of compactness as a similarity measure that enables efficient low level feature comparison and an annotation algorithm based on label transfer. [ABSTRACT FROM AUTHOR]

Exploiting the sparsity within representation models for images is critical for image denoising. The best currently available denoising methods take advantage of the sparsity from image self-similarity, pre-learned, and fixed representations. Most of these methods, however, still have difficulties in tackling high noise levels or noise models other than Gaussian. In this paper, the multiresolution structure and sparsity of wavelets are employed by nonlocal dictionary learning in each decomposition level of the wavelets. Experimental results show that our proposed method outperforms two state-of-the-art image denoising algorithms on higher noise levels. Furthermore, our approach is more adaptive to the less extensively researched uniform noise. [ABSTRACT FROM AUTHOR]

Block transform coded images usually suffer from annoying artifacts at low bit rates, caused by the coarse quantization of transform coefficients. In this paper, we propose a new method to reduce compression artifacts by the overlapped-block transform coefficient estimation from non-local blocks. In the proposed method, the discrete cosine transform coefficients of each block are estimated by adaptively fusing two prediction values based on their reliabilities. One prediction is the quantized values of coefficients decoded from the compressed bitstream, whose reliability is determined by quantization steps. The other prediction is the weighted average of the coefficients in nonlocal blocks, whose reliability depends on the variance of the coefficients in these blocks. The weights are used to distinguish the effectiveness of the coefficients in nonlocal blocks to predict original coefficients and are determined by block similarity in transform domain. To solve the optimization problem, the overlapped blocks are divided into several subsets. Each subset contains nonoverlapped blocks covering the whole image and is optimized independently. Therefore, the overall optimization is reduced to a set of sub-optimization problems, which can be easily solved. Finally, we provide a strategy for parameter selection based on the compression levels. Experimental results show that the proposed method can remarkably reduce compression artifacts and significantly improve both the subjective and objective qualities of block transform coded images. [ABSTRACT FROM AUTHOR]

This paper proposes a novel image-based framework to manipulate the illumination of human face through adaptive layer decomposition. According to our framework, only a single reference image, without any knowledge of the 3D geometry or material information of the input face, is needed. To transfer the illumination effects of a reference face image to a normal lighting face, we first decompose the lightness layers of the reference and the input images into large-scale and detail layers through weighted least squares (WLS) filter with adaptive smoothing parameters according to the gradient values of the face images. The large-scale layer of the reference image is filtered with the guidance of the input image by guided filter with adaptive smoothing parameters according to the face structures. The relit result is obtained by replacing the largescale layer of the input image with that of the reference image. To normalize the illumination effects of a non-normal lighting face (i.e., face delighting), we introduce similar reflectance prior to the layer decomposition stage by WLS filter, which make the normalized result less affected by the high contrast light and shadow effects of the input face. Through these two procedures, we can change the illumination effects of a non-normal lighting face by first normalizing the illumination and then transferring the illumination of another reference face to it. We acquire convincing relit results of both face relighting and delighting on numerous input and reference face images with various illumination effects and genders. Comparisons with previous papers show that our framework is less affected by geometry differences and can preserve better the identification structure and skin color of the input face. [ABSTRACT FROM AUTHOR]

Generating ensembles from multiple individual classifiers is a popular approach to raise the accuracy of the decision. As a rule for decision making, majority voting is a usually applied model. In this paper, we generalize classical majority voting by incorporating probability terms pn,k to constrain the basic framework. These terms control whether a correct or false decision is made if k correct votes are present among the total number of n. This generalization is motivated by object detection problems, where the members of the ensemble are image processing algorithms giving their votes as pixels in the image domain. In this scenario, the terms pn,k can be specialized by a geometric constraint. Namely, the votes should fall inside a region matching the size and shape of the object to vote together. We give several theoretical results in this new model for both dependent and independent classifiers, whose individual accuracies may also differ. As a real world example, we present our ensemble-based system developed for the detection of the optic disc in retinal images. For this problem, experimental results are shown to demonstrate the characterization capability of this system. We also investigate how the generalized model can help us to improve an ensemble with extending it by adding a new algorithm. [ABSTRACT FROM AUTHOR]

Advances in machine learning technology have made efficient image denoising possible. In this paper, we propose a new ridgelet support vector machine (RSVM) for image noise reduction. Multiscale ridgelet support vector filter (MRSVF) is first deduced from RSVM, to produce a multiscale, multidirection, undecimated, dyadic, aliasing, and shift-invariant geometric multiscale ridgelet support vector transform (GMRSVT). Then, multiscale dictionaries are learned from examples to reduce noises existed in GMRSVT coefficients. Compared with the available approaches, the proposed method has the following characteristics. The proposed MRSVF can extract the salient features associated with the linear singularities of images. Consequently, GMRSVT can well approximate edges, contours and textures in images, and avoid ringing effects suffered from sampling in the multiscale decomposition of images. Sparse coding is explored for noise reduction via the learned multiscale and overcomplete dictionaries. Some experiments are taken on natural images, and the results show the efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

The article offers information on the guidelines to be followed by authors who are interested to submit papers for the "IEEE Transactions on Image Processing."

The blind quality evaluation of screen content images (SCIs) and natural scene images (NSIs) has become an important, yet very challenging issue. In this paper, we present an effective blind quality evaluation technique for SCIs and NSIs based on a dictionary of learned local and global quality features. First, a local dictionary is constructed using local normalized image patches and conventional $K$ -means clustering. With this local dictionary, the learned local quality features can be obtained using a locality-constrained linear coding with max pooling. To extract the learned global quality features, the histogram representations of binary patterns are concatenated to form a global dictionary. The collaborative representation algorithm is used to efficiently code the learned global quality features of the distorted images using this dictionary. Finally, kernel-based support vector regression is used to integrate these features into an overall quality score. Extensive experiments involving the proposed evaluation technique demonstrate that in comparison with most relevant metrics, the proposed blind metric yields significantly higher consistency in line with subjective fidelity ratings. [ABSTRACT FROM AUTHOR]

Scene background initialization is the process by which a method tries to recover the background image of a video without foreground objects in it. Having a clear understanding about which approach is more robust and/or more suited to a given scenario is of great interest to many end users or practitioners. The aim of this paper is to provide an extensive survey of scene background initialization methods as well as a novel benchmarking framework. The proposed framework involves several evaluation metrics and state-of-the-art methods, as well as the largest video data set ever made for this purpose. The data set consists of several camera-captured videos that: 1) span categories focused on various background initialization challenges; 2) are obtained with different cameras of different lengths, frame rates, spatial resolutions, lighting conditions, and levels of compression; and 3) contain indoor and outdoor scenes. The wide variety of our data set prevents our analysis from favoring a certain family of background initialization methods over others. Our evaluation framework allows us to quantitatively identify solved and unsolved issues related to scene background initialization. We also identify scenarios for which state-of-the-art methods systematically fail. [ABSTRACT FROM PUBLISHER]

Increasing popularity of 3D videos calls for new methods to ease the conversion process of existing monocular video to stereoscopic or multi-view video. A popular way to convert video is given by depth image-based rendering methods, in which a depth map that is associated with an image frame is used to generate a virtual view. Because of the lack of knowledge about the 3D structure of a scene and its corresponding texture, the conversion of 2D video, inevitably, however, leads to holes in the resulting 3D image as a result of newly-exposed areas. The conversion process can be altered such that no holes become visible in the resulting 3D view by superimposing a regular grid over the depth map and deforming it. In this paper, an adaptive image warping approach as an improvement to the regular approach is proposed. The new algorithm exploits the smoothness of a typical depth map to reduce the complexity of the underlying optimization problem that is necessary to find the deformation, which is required to prevent holes. This is achieved by splitting a depth map into blocks of homogeneous depth using quadtrees and running the optimization on the resulting adaptive grid. The results show that this approach leads to a considerable reduction of the computational complexity while maintaining the visual quality of the synthesized views. [ABSTRACT FROM AUTHOR]

In this paper, we propose a depth-map merging based multiple view stereo method for large-scale scenes which takes both accuracy and efficiency into account. In the proposed method, an efficient patch-based stereo matching process is used to generate depth-map at each image with acceptable errors, followed by a depth-map refinement process to enforce consistency over neighboring views. Compared to state-of-the-art methods, the proposed method can reconstruct quite accurate and dense point clouds with high computational efficiency. Besides, the proposed method could be easily parallelized at image level, i.e., each depth-map is computed individually, which makes it suitable for large-scale scene reconstruction with high resolution images. The accuracy and efficiency of the proposed method are evaluated quantitatively on benchmark data and qualitatively on large data sets. [ABSTRACT FROM AUTHOR]

This paper presents a novel approach to edge-aware image manipulation. Our method processes a Gaussian pyramid from coarse to fine, and at each level, applies a nonlinear filter bank to the neighborhood of each pixel. Outputs of these spatially-varying filters are merged using global optimization. The optimization problem is solved using an explicit mixed-domain (real space and DCT transform space) solution, which is efficient, accurate, and easy-to-implement. We demonstrate applications of our method to a set of problems, including detail and contrast manipulation, HDR compression, nonphotorealistic rendering, and haze removal. [ABSTRACT FROM AUTHOR]