Search

Recent advances in deep learning have demonstrated excellent results for Facial Attribute Recognition (FAR), typically trained with large-scale labeled data. However, in many real-world FAR applications, only limited labeled data are available, leading to remarkable deterioration in performance for most existing deep learning-based FAR methods. To address this problem, here we propose a method termed Spatial-Semantic Patch Learning (SSPL). The training of SSPL involves two stages. First, three auxiliary tasks, consisting of a Patch Rotation Task (PRT), a Patch Segmentation Task (PST), and a Patch Classification Task (PCT), are jointly developed to learn the spatial-semantic relationship from large-scale unlabeled facial data. We thus obtain a powerful pre-trained model. In particular, PRT exploits the spatial information of facial images in a self-supervised learning manner. PST and PCT respectively capture the pixel-level and image-level semantic information of facial images based on a facial parsing model. Second, the spatial-semantic knowledge learned from auxiliary tasks is transferred to the FAR task. By doing so, it enables that only a limited number of labeled data are required to fine-tune the pre-trained model. We achieve superior performance compared with state-of-the-art methods, as substantiated by extensive experiments and studies.

State-of-the-art face recognition methods have achieved excellent performance on the clean datasets. However, in real-world applications, the captured face images are usually contaminated with noise, which significantly decreases the performance of these face recognition methods. In this paper, we propose a cascaded noise-robust deep convolutional neural network (CNR-CNN) method, consisting of two sub-networks, i.e., a denoising sub-network and a face recognition sub-network, for face recognition under noise. Instead of separately training the two sub-networks, we jointly train them in a cascaded manner. As a result, the images generated from the denoising sub-network are beneficial to the training of the face recognition sub-network. Furthermore, the dense connectivity is used to concatenate the feature maps layer-by-layer in the denoising sub-network, which can effectively exploit the shallow and deep features of CNN. Experimental results on public face datasets demonstrate the superior performance of the proposed method over several state-of-the-art methods.

In this paper, we propose a novel method, simple iterative clustering on graphs (SICG), to deal with robust model fitting problems. Specifically, we first construct a graph, where each vertex denotes a model hypothesis and each edge represents the similarity between two model hypotheses, for model fitting. We then propose a simple iterative clustering algorithm, which adapts the k-medoids clustering algorithm, to intuitively estimate model instances in data. The proposed SICG method is able to effectively fit and segment multiple-structure data contaminated with a large number of outliers and noises. Experimental results show that SICG achieves superior fitting results over several state-of-the-art model fitting methods on real images.

Recently, the correlation filter based trackers have achieved very good tracking performance. However, due to the boundary effects of the circulant matrix and the usage of cosine window, the lack of effective negative samples becomes a challenging problem for the correlation filter based trackers. This problem may cause overfitting so that these trackers become very sensitive to deformation and occlusion. In this paper, we propose a novel object tracker (i.e., STAPLE_HNM), which can effectively select hard negative samples and assign adaptive weights to these samples to train the correlation filter. Experimental results demonstrate that the proposed STAPLE_HNM tracker effectively improves the performance of the baseline STAPLE_CA tracker on the OTB-50 and OTB-100 datasets. Moreover, the proposed STAPLE_HNM tracker also achieves superior performance among several state-of-the-art trackers.

In this paper, we deal with geometric model fitting problems on graphs, where each vertex represents a model hypothesis, and each edge represents the similarity between two model hypotheses. Conventional median-shift methods are very efficient and they can automatically estimate the number of clusters. However, they assign the same weighting scores to all vertices of a graph, which can not show the discriminability on different vertices. Therefore, we propose a novel weighted median-shift on graphs method (WMSG) to fit and segment multiple-structure data. Specifically, we assign a weighting score to each vertex according to the distribution of the corresponding inliers. After that, we shift vertices towards the weighted median vertices iteratively to detect modes. The proposed method can adaptively estimate the number of model instances and deal with data contaminated with a large number of outliers. Experimental results on both synthetic data and real images show the advantages of the proposed method over several state-of-the-art model fitting methods.

In recent years, the triplet loss-based deep neural networks (DNN) are widely used in the task of face recognition and achieve the state-of-the-art performance. However, the complexity of training the triplet loss-based DNN is significantly high due to the difficulty in generating high-quality training samples. In this paper, we propose a novel DNN training framework to accelerate the training process of the triplet loss-based DNN and meanwhile to improve the performance of face recognition. More specifically, the proposed framework contains two stages: 1) The DNN initialization. A deep architecture based on the softmax loss function is designed to initialize the DNN. 2) The adaptive fine-tuning. Based on the trained model, a set of high-quality triplet samples is generated and used to fine-tune the network, where an adaptive triplet loss function is introduced to improve the discriminative ability of DNN. Experimental results show that, the model obtained by the proposed DNN training framework achieves 97.3% accuracy on the LFW benchmark with low training complexity, which verifies the efficiency and effectiveness of the proposed framework.

In this paper, we propose an efficient and robust gross outlier removal method, called the Conceptual Space based Gross Outlier Removal (CSGOR) method, to remove gross outliers for geometric model fitting. In the proposed method, each data point is mapped to a conceptual space by computing the preference of "good" model hypotheses. In the conceptual space, the distributions of inliers and gross outliers are significantly different. Specifically, inliers of each model instance are distributed in a subspace and they are far away from the origin of the conceptual space, while gross outliers are distributed near the origin. In this manner, the problem of densely gross outlier removal is formulated as a binary classification problem. The main advantage of the proposed method is that it can handle data with a large proportion of outliers and effectively remove gross outliers in data. Experimental results on both synthetic and real data have demonstrated the efficiency and effectiveness of the proposed method.

Metric learning technique learns a linear transformation of the given training data which can significantly promote the performance of a prediction task, such as kinship verification. However, many of the existing metric learning methods do not explicitly regularize for sparsity or low-rank, which in practice usually results in high-rank solutions that are not only time-consuming but also tend to overfitting. In addition, some methods simply neglect the positive semidefinite (PSD) constraint causing the learned metric to be potentially noisy. In this paper, we propose an effective sparse similarity metric learning (SSML) method which enforces both the group sparsity and the PSD constraints on the learned similarity matrix for kinship verification. In order to solve the proposed optimization problem efficiently, we successfully apply the alternating direction method of multipliers (ADMM) to obtain the optimal solution. Experimental results demonstrate that the proposed method achieves competitive results compared with other state-of-the-art metric learning methods on widely used kinship datasets.

In this paper, we propose a novel geometric model fitting method, called Mode-Seeking on Hypergraphs (MSH),to deal with multi-structure data even in the presence of severe outliers. The proposed method formulates geometric model fitting as a mode seeking problem on a hypergraph in which vertices represent model hypotheses and hyperedges denote data points. MSH intuitively detects model instances by a simple and effective mode seeking algorithm. In addition to the mode seeking algorithm, MSH includes a similarity measure between vertices on the hypergraph and a weight-aware sampling technique. The proposed method not only alleviates sensitivity to the data distribution, but also is scalable to large scale problems. Experimental results further demonstrate that the proposed method has significant superiority over the state-of-the-art fitting methods on both synthetic data and real images., Proceedings of the IEEE International Conference on Computer Vision, pp. 2902-2910, 2015

In this paper, we propose an effective scene text recognition method using sparse coding based features, called Histograms of Sparse Codes (HSC) features. For character detection, we use the HSC features instead of using the Histograms of Oriented Gradients (HOG) features. HSC features are extracted by computing sparse codes with dictionaries, which are learned from data using K-SVD, and aggregating perpixel sparse codes to form local histograms. For word recognition, we integrate multiple cues including character detection scores and geometric contexts in an objective function. The final recognition result is obtained by searching for the word which corresponds to the maximum value of the objective function. The parameters in the objective function are learned using the Minimum Classification Error (MCE) training method. Experiments on the ICDAR2003 and SVT datasets demonstrate that the HSC-based scene text recognition method outperforms the HOG-based method significantly and achieves the state-of-the-art performance.

In this paper, we propose a novel discriminative filter based regression learning (DFRL) method, which can effectively remove irrelevant information while preserving useful information for facial expression recognition. DFRL integrates the filter technique and the linear analysis techniques (i.e., Linear Discriminant Analysis-LDA and Linear Ridge Regression-LRR) to obtain an effective image representation. Two steps are involved in DFRL: 1) The discriminative filters corresponding to different facial expressions are separately trained by optimizing the cost function of the two-class LDA, 2) LRR is used to extract valuable expressional information with high discriminability from the combined filtered images. Experimental results on several challenging datasets demonstrate the superior effectiveness and generalization ability of the proposed DFRL compared with other competing methods.

Face recognition with occlusion is a challenging problem. Recently, the modular representation based method, i.e., modular linear regression based classification (MLRC) was proposed to deal with this problem. However, MLRC just simply combines the individual decision of each block within an image (based on the min rule) to make final decision. Therefore, the block distance information is not fully exploited. In this paper, we propose a robust modular linear regression based classification (RMLRC) method to overcome the above problem. RMLRC can effectively fuse the information provided by all the blocks and thus alleviate the limiations of the MLRC method. Experimental results show that the RMLRC method can achieve promising results for face recognition with occlusion.

In this paper, we propose a novel robust estimator, called ASEE (Adaptive Scale based Entropy-like Estimator) which minimizes the entropy of inliers. This estimator is based on IKOSE (Iterative Kth Ordered Scale Estimator) and LEL (Least Entropy-Like Estimator). Unlike LEL, ASEE only considers inliers' entropy while excluding outliers, which makes it very robust in parametric model estimation. Compared with other robust estimators, ASEE is simple and computationally efficient. From the experiments on both synthetic and real-image data, ASEE is more robust than several state-of-the-art robust estimators, especially in handling extreme outliers.

Segmentation-based tracking methods are popular in alleviating the model drift problem during online-learning of visual trackers. However, one of the limitations of those methods is that tracking results guide the process of segmentation. The model drift problem in tracking may have significant influence on segmentation. In this paper, we propose an online structured Hough Forests to address this limitation. The results of object tracking do not have significant influence on the process of segmentation. Our algorithm shows more robust results on several challenging sequences.

In this paper, a novel approach to single image super-resolution based on the multikernel regression is presented. This approach's core is to learn the map between the space of high-resolution image patches and the space of blurred high-resolution image patches which are the interpolation results generated from the corresponding low-resolution images. Kernel regression based super-resolution approach is promising, but the kernel selection is a critical problem. In order to avoid selecting the kernel via large amounts of cross-verifications, the multikernel regression is applied to learn the map function. This approach is efficient and the experimental results show that it manifests a high-quality performance in comparison with other super-resolution methods.

Visual tracking has been typically solved as a binary classification problem. Most existing trackers only consider the pairwise interactions between samples, and thereby ignore the higher-order contextual interactions, which may lead to the sensitivity to complicated factors such as noises, outliers, background clutters and so on. In this paper, we propose a visual tracker based on support vector machines (SVMs), for which a novel graph mode-based contextual kernel is designed to effectively capture the higher-order contextual information from samples. To do so, we first create a visual graph whose similarity matrix is determined by a baseline visual kernel. Second, a set of high-order contexts are discovered in the visual graph. The problem of discovering these high-order contexts is solved by seeking modes of the visual graph. Each graph mode corresponds to a vertex community termed as a high-order context. Third, we construct a contextual kernel that effectively captures the interaction information between the high-order contexts. Finally, this contextual kernel is embedded into SVMs for robust tracking. Experimental results on challenging videos demonstrate the effectiveness and robustness of the proposed tracker.

For plate detection, we found that when the plate is disturbed by complex upright borderlines, location can be inaccurate or even missed. On the basis of this, a new vehicle license plate location algorithm based on salient feature is introduced. It makes use of the texture feature, geometric characteristics and color information. By raw location and precise location, it can quickly locate the license plate position accurately and distinguishes the color type. Our experiment results show that this algorithm has a fast, efficient performance of locating vehicle license plate under complex background.

An effective method for retrieving 3D models is to represent and discriminate them with their 2D images projected from multiple viewpoints. Such view-based methods conform more closely to human visual recognition for 3D model retrieval, since the human retina essentially captures 2D images. However, most of the existing view-based methods do not take into account that different views have different importance even though they belong to the same object. To address this problem, we propose a novel view-based method for 3D CAD model retrieval. First, the PHOG descriptor is employed to describe the 2D images projected from a model. Then, Lagrange multipliers, vector quantization and a Support Vector Machine (SVM) are used to adaptively assign an optimal weight to each projected image. The similarity between a 3D query model and a 3D object in database is determined by the likeness of their corresponding 2D images associated with optimal weights. The effectiveness of the proposed method is shown in the experimental part.

Our goal is to fit the multiple instances (or structures) of a generic model existing in data. Here we propose a novel model selection scheme to estimate the number of genuine structures present. In contrast to conventional model selection approaches, our method is driven by kernel-based learning. The input data is first clustered based on their potential to have emerged from the same structure. However the number of clusters is deliberately overestimated to obtain a set of initial model fits onto the data. We then resolve the oversegmentation via a series of kernel optimisation conducted through multiple kernel learning, and the concept of kernel-target alignment is used as a model selection criterion. Experiments on synthetic and real data show that our method outperforms previous model selection schemes. We also focus on the application of multi-body motion segmentation. In particular we demonstrate success on estimating the number of motions on sequences with more than 3 unique motions.

This paper proposes an annealed particle swarm optimization based particle filter algorithm for articulated 3D human body tracking. In our algorithm, a sampling covariance and an annealing factor are incorporated into the velocity updating equation of particle swarm optimization (PSO). The sampling covariance and the annealing factor are initiated with appropriate values at the beginning of the PSO iteration, and ‘annealing’ is carried out at reasonable steps. Experiments with multi-camera walking sequences from the Brown dataset show that: 1) the proposed tracker can effectively alleviate the problem of inconsistency between the image likelihood and the true model; 2) the tracker is also robust to noise and body self-occlusion.

Spatio-temporal local feature based bag of visual words algorithm (BOVW) has shown promising results in complex human action classification. However, one key disadvantage of BOVW is geometrical unconstraint, which makes it impossible to recognize different actions with the same features but different spatial-temporal distribution of these features. In this paper, we exploit the spatio-temporal proximity distribution of local features in 3D space to characterize geometric context of action class. The obtained spatio-temporal proximity matrix models both the appearance and geometrics of human actions. Moreover, a spatio-temporal proximity distribution kernel (ST-PDK) is proposed to measure the similarity of videos, which satisfies Mercer condition and is directly incorporated into the kernel function of the SVM classifier. Our algorithm achieves the highest classification accuracy on KTH dataset, one of the most challenging and popular action datasets.

In this paper, we propose a novel method to localize (or track) a foreground object and segment the foreground object from the surrounding background with occlusions for a moving camera. We measure the likelihood of a target position by using a combination of a generative model and a discriminative model, considering not only the foreground similarity to the target model but also the dissimilarity between the foreground and the background appearances. Object segmentation is treated as a binary labeling problem. A Markov Random Field (MRF) is employed to add a spatial smooth prior on the foreground/background patterns. We demonstrate the advantages of the proposed method on several challenging videos and compare our results with the results of several other popular methods. The proposed method has achieved good results.

We propose an unconventional but highly effective approach to robust fitting of multiple structures by using statistical learning concepts. We design a novel Mercer kernel for the robust estimation problem which elicits the potential of two points to have emerged from the same underlying structure. The Mercer kernel permits the application of well-grounded statistical learning methods, among which nonlinear dimensionality reduction, principal component analysis and spectral clustering are applied for robust fitting. Our method can remove gross outliers and in parallel discover the multiple structures present. It functions well under severe outliers (more than 90% of the data) and considerable inlier noise without requiring elaborate manual tuning or unrealistic prior information. Experiments on synthetic and real problems illustrate the superiority of the proposed idea over previous methods.

Filtering feature selection method (filtering method, for short) is a well-known feature selection strategy in pattern recognition and data mining. Filtering method outperforms other feature selection methods in many cases when the dimension of features is large. There are so many filtering methods proposed in previous work leading to the “selection trouble” that how to select an appropriate filtering method for a given text data set. Since to find the best filtering method is usually intractable in real application, this paper takes an alternative path. We propose a feature selection framework that fuses the results obtained by different filtering methods. In fact, deriving a better rank list from different rank lists, known as rank aggregation, is a hot topic studied in many disciplines. Based on the proposed framework and Markov chains rank aggregation techniques, in this paper, we present two new feature selection methods: FR-MC1 and FR-MC4. We also introduce a perturbation algorithm to alleviate the drawbacks of Markov chains rank aggregation techniques. Empirical evaluation on two public text data sets shows that the two new feature selection methods achieve better or comparable results than classical filtering methods, which also demonstrate the effectiveness of our framework.

Initializing a background model requires robust statistical methods as the task should be robust against random occurrences of foreground objects, as well as against general image noise. The median has been employed for the problem of background initialization. However, the median has only a breakdown point of 50%. In this paper, we propose a new robust method which can tolerate more than 50% of noise and foreground pixels in the background initialization process. We compare our new method with five others and give quantitative evaluations on background initialization. Experiments show that the proposed method achieves very promising results in background initialization.

It has been shown that integrating multiple cues will increase the reliability and robustness of a vision system in situations that no single cue is reliable. In this paper, we propose a method by fusing multiple cues (i.e., the color cue and the edge cue). In contrast to previous work, we propose a novel shape similarity measure which includes the spatial distribution of, the number of, and the gradient intensity of the edge points. We integrate this shape similarity measure with our recently proposed SMOG-based color similarity measure in the framework of particle filter (PF). Experimental results demonstrate the high robustness and effectiveness of our method in handling appearance changes, cluttered background, moving camera, and occlusions.

Statistical background modeling is a fundamental and important part of many visual tracking systems and of other computer vision applications. In this paper, we presents an effective and adaptive background modeling method for detecting foreground objects in both static and dynamic scenes. The proposed method computes SAmple CONsensus (SACON) of the background samples and estimates a statistical model per pixel. Numerous experiments on both indoor and outdoor video sequences show that the proposed method, compared with several state-of-the-art methods, can achieve very promising performance

One of the most difficult issues in visual tracking is to track people in groups, especially under occlusions. In this paper, we present a novel sample consensus based method, which utilizes both color and spatial information of human bodies, to model the appearance of people. We use this appearance model to segment and track people through occlusions. We show experimental results in several video sequences to validate the effectiveness of the proposed method.