Search

To hallucinate super-resolution (super-res) face from a real low-quality face, a super-resolution technique based on definition-scalable inference (SRDSI) is proposed in this paper. In the proposed strategy, all high-res labeled faces are first decomposed into basic faces and enhanced faces to train a basic face and an enhanced face inferring model, and then two inferring models are used to hallucinate super-res basic face with low-definition and enhanced faces with high-frequency information from a single low-res face. Finally, the basic face is merged with its enhanced face into a super-res face with high-definition. In addition, this paper employs SIFT key-points to evaluate the similarity between the super-res face and its high-res labeled face. Experimental results show that SRDSI can effectively recover more structural information as well as SIFT key-points from real low-res faces and achieves better performance than state-of-the-art super-resolution techniques in terms of both visual and objective quality.

This paper gives a comprehensive study on gait biometrics via a joint CNN-based method. Gait is a kind of behavioral biometric feature with unique advantages, e.g., long-distance, cross-view and non-cooperative perception and analysis. In this paper, the definition of gait analysis includes gait recognition and gait-based soft biometrics such as gender and age prediction. We propose to investigate these two problems in a joint CNN-based framework which has been seldom reported in the recent literature. The proposed method is efficient in terms of training time, testing time and storage. We achieve the state-of-the-art performance on several gait recognition and soft biometrics benchmarks. Also, we discuss which part of the human body is important and informative for a specific task by network visualization.

Nowadays, graph-based dimensionality reduction approaches have become more and more popular due to their successful utilization for classification and clustering tasks. In these approaches, how to establish an appropriate graph is critical. To address this issue, a novel graph-based dimensionality reduction framework termed joint graph optimization and projection learning (JGOPL) is proposed in this paper. Compared with existing dimensionality reduction approaches, there are three main advantages of JGOPL. First, through performing the graph optimization and low-dimensional feature learning simultaneously, our proposed approach can accomplish the tasks of graph construction and dimensionality reduction jointly. Second, the l21-norm based distance measurement is adopted in the loss function of our JGOPL so that its robustness to the negative influence caused by the outliers or variations of data can be improved. Third, in order to well exploit and preserve the local structure information of high-dimensional data, a locality constraint is introduced into the proposed JGOPL to discourage a sample from connecting with the distant samples during graph optimization. Extensive classification and clustering experiments are carried out on seven publicly available databases to demonstrate the effectiveness of our approach. At last, the locality constraint and graph optimization strategy proposed in this paper is not only limited to dimensionality reduction, but also can be incorporated into other relevant graph-based tasks (such as spectral clustering).

A major issue in the classification of class imbalanced datasets involves the determination of the most suitable performance metrics to be used. In previous work using several examples, it has been shown that imbalance can exert a major impact on the value and meaning of accuracy and on certain other well-known performance metrics. In this paper, our approach goes beyond simply studying case studies and develops a systematic analysis of this impact by simulating the results obtained using binary classifiers. A set of functions and numerical indicators are attained which enables the comparison of the behaviour of several performance metrics based on the binary confusion matrix when they are faced with imbalanced datasets. Throughout the paper, a new way to measure the imbalance is defined which surpasses the Imbalance Ratio used in previous studies. From the simulation results, several clusters of performance metrics have been identified that involve the use of Geometric Mean or Bookmaker Informedness as the best null-biased metrics if their focus on classification successes (dismissing the errors) presents no limitation for the specific application where they are used. However, if classification errors must also be considered, then the Matthews Correlation Coefficient arises as the best choice. Finally, a set of null-biased multi-perspective Class Balance Metrics is proposed which extends the concept of Class Balance Accuracy to other performance metrics.

Cross-modality face recognition aims to identify faces across different modalities, such as matching sketches with photos, low resolution face images with high resolution images, and near infrared images with visual lighting images, which is challenging because of the modality gap caused by texture, resolution, and illumination variations. Existing approaches either utilized hand-crafted approaches which ignore inherent data distribution characteristic, or applied deep learning-based algorithms on holistic face images with facial local information ignored. In this paper, we propose a deep local descriptor learning framework for cross-modality face recognition, which aims to learn discriminant and compact local information directly from raw facial patches. A novel cross-modality enumeration loss is proposed to eliminate the modality gap on local patch level, which is then integrated into a convolutional neural networks for deep local descriptor extraction. The proposed deep local descriptor can be easily applied to any traditional face recognition systems, and we use Fisherface as an example in the paper. Extensive experiments on six widely used cross-modality face recognition datasets demonstrate the superiority of proposed method over state-of-the-art methods.

In this paper, we propose a linear classifier design method in the weight space. A linear classifier is completely determined by a weight vector. To design a linear classifier is equivalent to finding a weight vector. When there are a number of training samples, each training sample represents a plane in the weight space. On one side of the plane, the training sample is correctly classified while it is incorrectly classified on the other side. Thus, finding the optimal linear classifier can be formulated as finding a subspace that provides the best classification accuracy. In this paper, we have developed an algorithm to find such optimal subspaces in the weight space. Although we could not find the optimal linear classifier due to prohibitive computational complexity, the proposed design method produced noticeable improvements in some cases. Experimental results show that the proposed linear classifier performed better than or equivalent to existing linear classifiers.

This paper presents a fast and accurate method for matching oblique aerial image pairs. In order to achieve accurate matching results, we must consider viewpoint differences between the input images in addition to rotation and scaling. Existing methods that match aerial image pairs with viewpoint differences undergo heavy computation and have difficulty finding correspondences. In this paper, we propose a homography matrix evaluation method based on a geometric approach to increase the accuracy of image matching results. In addition, we achieve faster matching through an iterative transform simulation that reduces computational complexity. Experimental results show that the proposed method improves aerial image matching in terms of computational efficiency while achieving successful matching results.

SLAM is generally addressed using natural landmarks such as keypoints or texture, but it poses some limitations, such as the need for enough textured environments and high computational demands. In some cases, it is preferable sacrificing the flexibility of such methods for an increase in speed and robustness by using artificial landmarks. The recent work [1] proposes an off-line method to obtain a map of squared planar markers in large indoor environments. By freely distributing a set of markers printed on a piece of paper, the method estimates the marker poses from a set of images, given that at least two markers are visible in each image. Afterwards, camera localization can be done, in the correct scale. However, an off-line process has several limitations. First, errors can not be detected until the whole process is finished, e.g., an insufficient number of markers in the scene or markers not properly spotted in the capture stage. Second, the method is not incremental, so, in case of requiring the expansion of the map, it is necessary to repeat the whole process from start. Finally, the method can not be employed in real-time systems with limited computational resources such as mobile robots or UAVs. To solve these limitations, this work proposes a real-time solution to the problems of simultaneously localizing the camera and building a map of planar markers. This paper contributes with a number of solutions to the problems arising when solving SLAM from squared planar markers, coining the term SPM-SLAM. The experiments carried out show that our method can be more robust, precise and fast, than visual SLAM methods based on keypoints or texture.

Completely automated iris recognition has emerged as an integral part of e-business and e-governance infrastructure which has acquired billions of iris images under near-infrared illumination to establish the identity of individuals. A range of e-business and surveillance applications can provide iris images that are acquired under visible illumination. Therefore, development of accurate cross-spectral iris matching capabilities is highly desirable. This paper investigates cross-spectral iris recognition using a range of deep learning architectures. Our experimental results on two publicly available cross-spectral iris databases, from 209 and 120 different subjects respectively, indicate outperforming results and validate our approach for the cross-spectral iris matching. Our observations indicate that the self-learned features generated from the convolution neural networks (CNN) are generally sparse and offer great potential for template compression. Therefore, this paper also introduces the iris recognition with supervised discrete hashing that can not only achieve more accurate performance but also offer a significant reduction in the size of iris templates. Most accurate cross-spectral matching performance is achieved by incorporating supervised discrete hashing on the features learned from the trained CNN with softmax cross-entropy loss. The proposed approach not only achieves outperforming results over other considered CNN architecture but also offers significantly reduced template size as compared with the other iris recognition methods available in the literature.

In this paper a new nonparametric scene parsing approach is proposed which has three steps: image retrieval, label transferring and label gathering. In our approach, to incorporate the contextual knowledge in scene parsing, we propose to integrate both parametric and nonparametric context models into a unified framework. We adopt a co-occurrence graph to be our parametric context model to learn the co-occurrence frequency of objects. To consider different preferences of the co-occurring of one object with the other objects, the concept of co-occurring priority is introduced in this paper for the first time. Next, by using the learned co-occurrence graph and the context knowledge of the set of retrieved images, we propose new ways to incorporate contextual information in all three steps of nonparametric scene parsing approach. To evaluate our proposed approach, it is applied on MSRC-21 and SiftFlow datasets. The results show that our approach outperforms its competitors.

This paper presents a robust method for rotation estimation of planar point sets using the Angular Radon Spectrum (ARS). Given a Gaussian Mixture Model (GMM) representing the point distribution, the ARS is a continuous function derived from the Radon Transform of such distribution. The ARS characterizes the orientation of a point distribution by measuring its alignment w.r.t. a pencil of parallel lines. By exploting its translation and angular-shift invariance, the rotation angle between two point sets can be estimated through the correlation of the corresponding spectra. Beside its definition, the novel contributions of this paper include the efficient computation of the ARS and of the correlation function through their Fourier expansion, and a new algorithm for assessing the rotation between two point sets. Moreover, experiments with standard benchmark datasets assess the performance of the proposed algorithm and other state-of-the-art methods in presence of noisy and incomplete data.

In extreme learning machines (ELM), the hidden node parameters are randomly generated and the output weights can be analytically computed. To overcome the bad feature extraction ability of the shallow architecture of ELM, the hierarchical ELM has been extensively studied as a deep architecture with multilayer neural network. However, the commonly used mean square error (MSE) criterion is very sensitive to outliers and impulsive noises, generally existing in real world data. In this paper, we investigate the correntropy to improve the robustness of the multilayer ELM and provide sparser representation. The correntropy, as a nonlinear measure of similarity, is robust to outliers and can approximate different norms (from l0 to l2). A new full correntropy based multilayer extreme learning machine (FC-MELM) algorithm is proposed to handle the classification of datasets which are corrupted by impulsive noises or outliers. The contributions of this paper are three-folds: (1) The MSE based reconstruction loss is replaced by the correntropy based loss function; In this way, the robustness of the ELM based multilayer algorithms is enhanced. (2) The traditional l1-based sparsity penalty term is also replaced by a correntropy-based sparsity penalty term, which can further improve the performance of the proposed algorithm with a sparser representation of the data. The combination of (1) and (2) provides the correntropy-based ELM autoencoder. (3) The FC-MELM is proposed by using the correntropy-based ELM autoencoder as a building block. It is notable that the FC-MELM is trained in a forward manner, which means fine-tuning procedure is not required. Thus, the FC-MELM has great advantage in learning efficiently when compared with traditional deep learning algorithms. The good property of the proposed algorithm is confirmed by the experiments on well-known benchmark datasets, including the MNIST datasets, the NYU Object Recognition Benchmark dataset, and the Moore network traffic dataset. Finally, the proposed FC-MELM algorithm is applied to address Computer Aided Cancer Diagnosis. Experiments conducted on the well-known Wisconsin Breast Cancer Data (Diagnostic) dataset are presented and show that the proposed FC-MELM outperforms state-of-the-art methods in solving computer aided cancer diagnosis problems.

Contactless 3D fingerprint identification has gained significant attentions in recent years as it can offer more hygienic, accurate and ubiquitous personal identification. Despite such advantages, contactless 3D imaging often results in partial 3D fingerprints as it requires relatively higher cooperation from users during the contactless 3D imaging. Such contactless 3D fingerprint images significantly degrade matching accuracy due to partial 3D fingerprint imaging. This paper proposes an end-to-end contactless 3D fingerprint representation learning model based on convolutional neural network (CNN). The proposed model includes one fully convolutional network for fingerprint segmentation and three Siamese networks to learn multi-view 3D fingerprint feature representation. Contactless partial 3D fingerprint identification is a more challenging problem due to its high degree of freedom during contactless 3D fingerprint acquisition and is also addressed by using proposed model. We therefore investigate multi-view 3D fingerprint recognition and partial 3D fingerprint using proposed approach. Comparative experimental results, presented in this paper using state-of-the-art 3D fingerprint recognition method, demonstrate the effectiveness of the proposed multi-view approach and illustrate a significant improvement of state-of-the-art 3D fingerprint recognition methods.

This paper explores the problem of circle fitting for incomplete (partial arc) laser scanning point cloud data in the presence of outliers. In mobile laser scanning, data are commonly incomplete because of the orientation of the scanning unit to the surveying objects and the limited street-based positions. Also, multiple structures in the built environment often produce clustered outliers. To address these problems, this paper combines robust Principal Component Analysis (PCA) and robust regression with an efficient algebraic circle fitting method to develop two algorithms for circle fitting. Experimental efforts show that the proposed algorithms are statistically robust and can tolerate a high-percentage (exceeding 44%) of clustered outliers with insignificant error levels, while still achieving better shape recognition compared to existing competitive methods. For example, for a simulation of 1000 quarter circle datasets including 20% clustered outliers, RANSAC estimated the circle radius with a Mean Squared Error (MSE) of 172.10, whereas the proposed algorithms fit circles with an MSE of less than 0.42. The algorithms have potential in many areas including building information modeling, particle tracking, product quality control, arboreal assessment, and road asset monitoring.

This paper proposes a new method of personal identification that introduces the analysis of lip prints. In spite of its important role in forensic and biometric applications, the results of previous investigations into lip prints are scanty. This is mainly due to the difficulties that accompany any analysis of the lips: lips are very flexible and pliable, and successive lip print impressions – even those obtained from the same person – may significantly differ from one other. Our article’s principal contribution is a proposal for a new personal identification methodology and application that uses, for the first time, a strategy for biometric and forensic analysis of lip print structure. As a result of our analysis we propose a lip print pattern for each individual. This pattern contains only such furrows that occur on the greatest number of lip prints obtained from the same person, where these furrows’ locations and inclinations remain similar across the lip prints obtained. It should be noted that in our approach, instead of lip photos we employ lip prints, such as can be obtained at a crime scene. It is worth noticing also that we propose a new method of personal identification where, instead of popular machine learning methods,the furrow-analysis of lip prints is introduced. Thus, no classifier learning phase is required. According to the authors’ convictions, based on reports in the literature, the proposed approach describes for the first time a strategy as to how lip print structures could be analyzed in biometric applications. The main novelty of this paper is its use of multiple lip print furrows from the same person to determine the most common lip furrow distribution in four different directions. On the basis of the pattern recognition of the lip furrows, the identity of the person will be determined. The proposed method’s effectiveness is 92.73%, determined using a database containing 350 lip prints.

Local ternary pattern (LTP) is a simple, yet high-discriminative texture feature model. In order to further promote the performance of LTP in noisy texture classification, this paper proposes a multi-scale median LTP (MLTP) framework. It firstly designs a non-overlapped median sampling scheme to resist against noise, and then re-defines central descriptor, radial descriptor and magnitude descriptor for MLTP. Moreover, it also proposes the pattern mapping solution named rotation-invariant uniform three (riu3) so as to project original MLTP codes from high-dimensional patterns to low-dimensional ones. In our multi-scale algorithm, every image is sampled and encoded separately by three MLTP descriptors, and then original patterns are mapped by a pre-stored riu3 pattern lookup table. Finally, the frequency histogram of joint distribution on three mapped MLTP code images is computed to generate the feature vector of given sampling scale. The different vectors under different scales are concatenated together to produce the objective vectors for training classifier or texture classification. The available classifiers for our algorithm include nearest neighbor classifier (NNC), and support vector machine (SVM), etc. The experiments on five publicly-available texture databases show that the multi-scale MLTP method proposed in this paper could obtain an average accuracy of 99.48% on dataset OTC12, 96.97% on UIUC, 98.71% on KTH-TIPS2b, 98.55% on Brodatz and 97.49% on ALOT. Compared with other state-of-the-art approaches, under the low-intensity conditions of Gaussian, Salt & Pepper and random pixel corruption noise, multi-scale MLTP + SVM could still keep higher accuracy and have better noise robustness than others.

In this paper we further develop the recent concept of multi-component shapes, which is applicable to image processing and image analysis tasks. The domain of multi-component shapes is very diverse and includes shapes that correspond to a group of objects that act together (e.g. a fish shoal), natural components of a segmented object (e.g. cells in embryonic tissues), a set of shapes corresponding to the same object appearing at different times (e.g. human gait in an image sequence), and many more. So far, there are few methods for numerically evaluating multi-component shapes. In this paper we introduce one such method: a disconnectedness measure, that naturally corresponds to multi-component shapes, and has no analogue in single-component shape measures. The new measure depends on the number of shape components, the whole shape but also the shape of its components, on the relative size of the shape’s components and their mutual position. All these are natural requirements for a “disconnectedness” multi-component shape measure. In addition, the new measure is invariant with respect to translation, rotation and scaling transformations. The measure is simple and fast to compute. The disconnectedness measure introduced here is a generic image analysis tool. It has not been developed for a specific application. As such, it can be applied to a variety of applications. Several of them are provided in the paper, as well as synthetic examples that support a better understanding of the behavior of the new measure.

This paper advocates the local steering kernel to encode the image patch.A novel local adaptive contrast measure using local steering kernel is proposed.This paper presents a patch-based infrared small target detection approach.The proposed infrared small target detection method achieves promising results. Because infrared small target detection plays a crucial role in infrared monitoring and early warning systems, it has been the subject of considerable research. Although many infrared small target detection approaches have been proposed, how to robustly detect small targets in poor quality infrared images remains a challenge. Since existing feature descriptors are often sensitive to the quality of infrared images, this paper advocates the use of a local steering kernel (LSK) to encode the infrared image patch because the LSK method can provide robust estimation of local intrinsic structure, even for poor quality images. Furthermore, this paper proposes a novel local adaptive contrast measure based on LSK reconstruction (LACM-LSK) for infrared small target detection. To demonstrate the effectiveness of the proposed approach, a diverse test dataset, including six infrared image sequences with different backgrounds, was collected. Extensive experiments on the test dataset confirm that the proposed infrared small target detection approach can achieve better detection performance than state-of-the-art approaches.

Recently, deep learning has achieved great success in visual tracking. The goal of this paper is to review the state-of-the-art tracking methods based on deep learning. First, we introduce the background of deep visual tracking, including the fundamental concepts of visual tracking and related deep learning algorithms. Second, we categorize the existing deep-learning-based trackers into three classes according to network structure, network function and network training. For each categorize, we explain its analysis of the network perspective and analyze papers in different categories. Then, we conduct extensive experiments to compare the representative methods on the popular OTB-100, TC-128 and VOT2015 benchmarks. Based on our observations, we conclude that: (1) The usage of the convolutional neural network (CNN) model could significantly improve the tracking performance. (2) The trackers using the convolutional neural network (CNN) model to distinguish the tracked object from its surrounding background could get more accurate results, while using the CNN model for template matching is usually faster. (3) The trackers with deep features perform much better than those with low-level hand-crafted features. (4) Deep features from different convolutional layers have different characteristics and the effective combination of them usually results in a more robust tracker. (5) The deep visual trackers using end-to-end networks usually perform better than the trackers merely using feature extraction networks. (6) For visual tracking, the most suitable network training method is to per-train networks with video information and online fine-tune them with subsequent observations. Finally, we summarize our manuscript and highlight our insights, and point out the further trends for deep visual tracking.

The classification of dynamical data streams is among the most complex problems encountered in classification. This is, firstly, because the distribution of the data streams is non-stationary, and it changes without any prior “warning”. Secondly, the manner in which it changes is also unknown. Thirdly, and more interestingly, the model operates with the assumption that the correct classes of previously-classified patterns become available at a juncture after their appearance. This paper pioneers the use of unreported novel schemes that can classify such dynamical data streams by invoking the recently-introduced “Anti-Bayesian” (AB) techniques. Contrary to the Bayesian paradigm, that compare the testing sample with the distribution’s central points, AB techniques are based on the information in the distant-from-the-mean samples. Most Bayesian approaches can be naturally extended to dynamical systems by dynamically tracking the mean of each class using, for example, the exponential moving average based estimator, or a sliding window estimator. The AB schemes introduced by Oommen et al.., on the other hand, work with a radically different approach and with the non-central quantiles of the distributions. Surprisingly and counter-intuitively, the reported AB methods work equally or close-to-equally well to an optimal supervised Bayesian scheme on a host of accepted Pattern Recognition problems. This thus begs its natural extension to the unexplored arena of classification for dynamical data streams. Naturally, for such an AB classification approach, we need to track the non-stationarity of the quantiles of the classes. To achieve this, in this paper, we develop an AB approach for the online classification of data streams by applying the efficient and robust quantile estimators developed by Yazidi and Hammer [12,37]. Apart from the methodology itself, in this paper, we compare the Bayesian and AB approaches using both real-life and synthetic data. The results demonstrate the intriguing and counter-intuitive results that the AB approach, sometimes, actually outperforms the Bayesian approach for this application both with respect to the peak performance obtained, and the robustness of the choice of the respective tuning parameters. Furthermore, the AB approach is much more robust against outliers, which is an inherent property of quantile estimators [12,37], which is a property that the Bayesian approach cannot match, since it rather tracks the mean.

Four novel multilayer network embedding algorithms are proposed with the consideration on four network structural properties.Four objective functions based on NMF are specially designed and optimized to preserve the target structural properties.Extensive experiments are conducted to evaluate the performances of the proposed embedding algorithms on different tasks. Multilayer network is a structure commonly used to describe and model the complex interaction between sets of entities/nodes. A three-layer example is the author-paper-word structure in which authors are linked by co-author relation, papers are linked by citation relation, and words are linked by semantic relation. Network embedding, which aims to project the nodes in the network into a relatively low-dimensional space for latent factor analysis, has recently emerged as an effective method for a variety of network-based tasks, such as collaborative filtering and link prediction. However, existing studies of network embedding both focus on the single-layer network and overlook the structural properties of the network, e.g., the degree distribution and communities, which are significant for node characterization, such as the preferences of users in a social network. In this paper, we propose four multilayer network embedding algorithms based on Nonnegative Matrix Factorization (NMF) with consideration given to four structural properties: whole network (NNMF), community (CNMF), degree distribution (DNMF), and max spanning tree (TNMF). Experiments on synthetic data show that the proposed algorithms are able to preserve the desired structural properties as designed. Experiments on real-world data show that multilayer network embedding improves the accuracy of document clustering and recommendation, and the four embedding algorithms corresponding to the four structural properties demonstrate the differences in performance on these two tasks. These results can be directly used in document clustering and recommendation systems.

A novel approach in the field of classifier ensembles is proposed.The approach uses an ensemble of random decision trees.Each decision tree is trained on a different area of the input space.Areas can overlap and a good coverage of the input space is ensured.Experimental results confirm the validity of the approach. Ensemble methods have shown to be more effective than monolithic classifiers, in particular when diversity holds among their components. How to enforce diversity in classifier ensembles has received much attention from machine learning researchers, yielding a variety of different techniques and algorithms. In this paper, a novel algorithm for ensemble classifiers is proposed, in which ensemble components are trained with focus on different regions of the sample space. In so doing, diversity is mainly a consequence of the intention to limit the scope of base classifiers. The algorithm proposed in this paper shares roots with several ensemble paradigms, in particular with random forests, as it generates forests of decision trees as well. As decision trees are trained with focus on specific subsets of the sample space, the resulting ensemble is in fact a forest of local trees. Comparative experimental results highlight that, on average, these ensembles perform better than other relevant kinds of ensemble classifiers, including random forests.

In this paper, a locality constraint distance metric learning is proposed for traffic congestion detection. First of all, an accurate and unified definition of congestion is proposed and the congestion level analysis is treated as a regression problem in the paper. Based on that definition, a dataset consists of 20 different scenes is constructed for the first time since the existing dataset is not diverse for real applications. To characterize the congestion level in different scenes, the low-level texture feature and kernel regression is utilized to detect traffic congestion level. To reduce the influence among different scenes, a Locality Constraint Distance Metric Learning (LCML) which ensured the local smoothness and preserved the correlations between samples is proposed. The extensive experiments confirm the effectiveness of the proposed method.

In this paper, we investigate the problem of video-based kinship verification via human face analysis. While several attempts have been made on facial kinship verification from still images, to our knowledge, the problem of video-based kinship verification has not been formally addressed in the literature. In this paper, we make the two contributions to video-based kinship verification. On one hand, we present a new video face dataset called Kinship Face Videos in the Wild (KFVW) which were captured in wild conditions for the video-based kinship verification study, as well as the standard benchmark. On the other hand, we employ our benchmark to evaluate and compare the performance of several state-of-the-art metric learning based kinship verification methods. Experimental results are presented to demonstrate the efficacy of our proposed dataset and the effectiveness of existing metric learning methods for video-based kinship verification. Lastly, we also evaluate human ability on kinship verification from facial videos and experimental results show that metric learning based computational methods are not as good as that of human observers.

Multi-label classification associates an unseen instance with multiple relevant labels. In recent years, a variety of methods have been proposed to handle the multi-label problems. Classifier chains is one of the most popular multi-label methods because of its efficiency and simplicity. In this paper, we consider to optimize classifier chains from the viewpoint of conditional likelihood maximization. In the proposed unified framework, classifier chains can be optimized in either or both of two aspects: label correlation modeling and multi-label feature selection. In this paper we show that previous classifier chains algorithms are specified in the unified framework. In addition, previous information theoretic multi-label feature selection algorithms are specified with different assumptions on the feature and label spaces. Based on these analyses, we propose a novel multi-label method, k-dependence classifier chains with label-specific features, and demonstrate the effectiveness of the method.

Squared planar markers are a popular tool for fast, accurate and robust camera localization, but its use is frequently limited to a single marker, or at most, to a small set of them for which their relative pose is known beforehand. Mapping and localization from a large set of planar markers is yet a scarcely treated problem in favour of keypoint-based approaches. However, while keypoint detectors are not robust to rapid motion, large changes in viewpoint, or significant changes in appearance, fiducial markers can be robustly detected under a wider range of conditions. This paper proposes a novel method to simultaneously solve the problems of mapping and localization from a set of squared planar markers. First, a quiver of pairwise relative marker poses is created, from which an initial pose graph is obtained. The pose graph may contain small pairwise pose errors, that when propagated, leads to large errors. Thus, we distribute the rotational and translational error along the basis cycles of the graph so as to obtain a corrected pose graph. Finally, we perform a global pose optimization by minimizing the reprojection errors of the planar markers in all observed frames. The experiments conducted show that our method performs better than Structure from Motion and visual SLAM techniques., Paper submitted to journal. Code available. See webpage http://www.uco.es/investiga/grupos/ava/node/57/

In word spotting literature, many approaches have considered word images as temporal signals that could be matched by classical Dynamic Time Warping algorithm. Consequently, DTW has been widely used as a on the shelf tool. However there exists many other improved versions of DTW, along with other robust sequence matching techniques. Very few of them have been studied extensively in the context of word spotting whereas it has been well explored in other application domains such as speech processing, data mining etc. The motivation of this paper is to investigate such area in order to extract significant and useful information for users of such techniques. More precisely, this paper has presented a comparative study of classical Dynamic Time Warping (DTW) technique and many of its improved modifications, as well as other sequence matching techniques in the context of word spotting, considering both theoretical properties as well as experimental ones. The experimental study is performed on historical documents, both handwritten and printed, at word or line segmentation level and with a limited or extended set of queries. The comparative analysis is showing that classical DTW remains a good choice when there is no segmentation problems for word extraction. Its constrained version (e.g. Itakura Parallelogram) seems better on handwritten data, as well as Hilbert transform also shows promising performances on handwritten and printed datasets. In case of printed data and low level features (pixel’s column based), the aggregation of features (e.g. Piecewise-DTW) seems also very important. Finally, when there are important word segmentation errors or when we are considering line segmentation level, Continuous Dynamic Programming (CDP) seems to be the best choice.

Deep learning has recently achieved great success in many visual recognition tasks. However, the deep neural networks (DNNs) are often perceived as black-boxes, making their decision less understandable to humans and prohibiting their usage in safety-critical applications. This guest editorial introduces the thirty papers accepted for the Special Issue on Explainable Deep Learning for Efficient and Robust Pat- tern Recognition. They are grouped into three main categories: explainable deep learning methods, effi- cient deep learning via model compression and acceleration, as well as robustness and stability in deep learning. For each of the three topics, a survey of the representative works and latest developments is presented, followed by the brief introduction of the accepted papers belonging to this topic. The special issue should be of high relevance to the reader interested in explainable deep learning methods for ef- ficient and robust pattern recognition applications and it helps promoting the future research directions in this field.

This paper focuses on the re-assembly of an archaeological artifact, given images of its fragments. This problem can be considered as a special challenging case of puzzle solving. The restricted case of re-assembly of a natural image from square pieces has been investigated extensively and was shown to be a difficult problem in its own right. Likewise, the case of matching “clean” 2D polygons/splines based solely on their geometric properties has been studied. But what if these ideal conditions do not hold? This is the problem addressed in the paper. Three unique characteristics of archaeological fragments make puzzle solving extremely difficult: (1) The fragments are of general shape; (2) They are abraded, especially at the boundaries (where the strongest cues for matching should exist); and (3) The domain of valid transformations between the pieces is continuous. The key contribution of this paper is a fully-automatic and general algorithm that addresses puzzle solving in this intriguing domain. We show that our approach manages to correctly reassemble dozens of broken artifacts and frescoes.

The paper is devoted to two problems: (1) reinforcement of SVM algorithms, and (2) justification of memorization mechanisms for generalization. (1) Current SVM algorithm was designed for the case when the risk for the set of nonnegative slack variables is defined by l 1 norm. In this paper, along with that classical l 1 norm, we consider risks defined by l 2 norm and l ∞ norm. Using these norms, we formulate several modifications of the existing SVM algorithm and show that the resulting modified SVM algorithms can improve (sometimes significantly) the classification performance. (2) Generalization ability of existing learning algorithms is usually explained by arguments involving uniform convergence of empirical losses to the corresponding expected losses over a given set of functions. However, along with bounds for uniform convergence of empirical losses to the expected losses, the VC theory also provides bounds for relative uniform convergence. These bounds lead to a more accurate estimate of the expected loss. Advanced methods of estimating of expected risk of error have to leverage these bounds, which also support mechanisms of training data memorization, which, as the paper demonstrates, can improve classification performance.

In this paper, we address a machine learning task using encrypted training data. Our basic scenario has three parties: Data Owners, who own private data; an Application, which wants to train and use an arbitrary machine learning model on the Users’ data; and an Authorization Server, which provides Data Owners with public and secret keys of a partial homomorphic cryptosystem (that protects the privacy of their data), authorizes the Application to get access to the encrypted data, and assists it in those computations not supported by the partial homomorphism. As machine learning model, we have selected the Support Vector Machine (SVM) due to its excellent performance in supervised classification tasks. We evaluate two well known SVM algorithms, and we also propose a new semiparametric SVM scheme better suited for the privacy-protected scenario. At the end of the paper, a performance analysis regarding the accuracy and the complexity of the developed algorithms and protocols is presented.

International audience; Graph edit distance (GED) is a powerful and flexible graph matching paradigm that can be used to address different tasks in structural pattern recognition, machine learning, and data mining. In this paper, some new binary linear programming formulations for computing the exact GED between two graphs are proposed. A major strength of the formulations lies in their genericity since the GED can be computed between directed or undirected fully attributed graphs (i.e. with attributes on both vertices and edges). Moreover, a relaxation of the domain constraints in the formulations provides efficient lower bound approximations of the GED. A complete experimental study comparing the proposed formulations with 4 state-of-the-art algorithms for exact and approximate graph edit distances is provided. By considering both the quality of the proposed solution and the efficiency of the algorithms as performance criteria, the results show that none of the compared methods dominates the others in the Pareto sense. As a consequence, faced to a given real-world problem, a trade-off between quality and efficiency has to be chosen w.r.t. the application constraints. In this context, this paper provides a guide that can be used to choose the appropriate method.

In this paper we exploit the use of synthetic data for on-line handwritten gesture commands recognition with an emphasis on the problem of forgetting unused classes. For on-line learning, one of the most crucial moments of the processing is the initialization. In some applications the data is available and these can be fed to the learning model. However, in applications such as user-friendly handwritten gesture recognition, this scenario is not possible. Since from the user perspective it is better to let the user define his own symbols, the learning model is lacking in the amount of data at the initialization. Some strategies have been proposed to acquire synthetic handwritten gesture commands and use these for on-line learning. In this paper we exploit this technique further and focus on the forgetting of unused classes by applying a random buffer and Elastic Memory Learning (EML) to avoid this from happening. In the experiments we search for the proper amount of synthetic data produced for each sample as well as exploit the most appropriate time to stop the generation of synthetic data for learning purposes. We also investigate the influence of synthetic data on forgetting when using the proposed EML. We base the generation of synthetic gesture commands on Kinematic Theory.

Fine-grained visual categorization (FGVC) has attracted extensive attention in recent years. The general pipeline of current FGVC techniques is to 1) locate the discriminative regions; 2) extract features from each region independently; and 3) feed the integrated features to a classifier. In this paper, we re-investigate the pipeline from the view of human visual recognition mechanisms. The perceiving of discriminative regions is a temporal processing by the human visual system (HVS) via the attention-shift mechanism. However, the existing independent feature extracting and one-pass feeding strategy ignore the inherent semantic relationships among discriminative regions, and thus is improper to model the attention-shift process properly. Therefore, in this paper, we propose a novel end-to-end FGVC network structure named Attention-Shift based Deep Neural Network (AS-DNN) to locate the discriminative regions automatically and encode the semantic correlations iteratively. AS-DNN consists of two channels: 1) the global perception channel C glb and 2) the attention-shift channel C sft , simulating the global perception and the attention-shift mechanism, respectively. Experimental results show that AS-DNN achieves state-of-the-art performances by outperforming both the CNN-based weakly or strongly-supervised FGVC algorithms on several widely-used fine-grained datasets, and the visualization of attention regions exhibit that the proposed method can locate the discriminative regions robustly in complex backgrounds and postures.

The Euclidean Minimum Sum-of-Squares Clustering( MSSC ) is one of the most important models for the clustering problem. Due to its NP-hardness, the problem continues to receive much attention in the scientific literature and several heuristic procedures have been proposed. Recent research has been devoted to the improvement of the classical K-MEANS algorithm, either by suitably selecting its starting configuration or by using it as a local search method within a global optimization algorithm. This paper follows this last approach by proposing a new implementation of a Memetic Differential Evolution ( MDE ) algorithm specifically designed for the MSSC problem and based on the repeated execution of K-MEANS from selected configurations. In this paper we describe how to adapt MDE to the clustering problem and we show, through a vast set of numerical experiments, that the proposed method has very good quality, measured in terms of the minimization of the objective function, as well as a very good efficiency, measured in the number of calls to the local optimization routine, with respect to state of the art methods.

Recently softmax based loss functions have surged to advance image classification and face verification. Most efforts boost discrimination of the softmax loss by using novel angular margins in varying ways, but few analyze where the discrimination truly comes from whilst considering the power of relieving the overfitting to enhance the softmax loss. In this paper, we firstly delve into such mainstream of softmax based loss functions in theory, and recognize the importance of easing overfitting to the softmax loss. In terms of such analysis, this paper intends to bring the softmax loss up to the competitive level with current well-behaved loss functions. We do this in two ways: (1) regularizing the softmax to relieve the overfitting by learning the log-probability centers, and (2) rescaling deep embeddings of the softmax with a constant scale to further enhance inter-class separability in Euclidean space. We call the resulting loss function rLogCenter loss for short. Simple and interpretable as our loss is, it guides CNNs to induce performance gains in the experiments of both image classification and face verification.

Highlights • In response to COVID-19, a novel visual SLAM method is proposed to serve healthcare. • The proposed method enables healthcare robot navigation in complex environments. • Evaluation with real-world data demonstrates improved accuracy and robustness., The COVID-19 pandemic has affected many countries, posing a threat to human health and safety, and putting tremendous pressure on the medical system. This paper proposes a novel SLAM technology using RGB and depth images to improve hospital operation efficiency, reduce the risk of doctor-patient cross-infection, and curb the spread of the COVID-19. Most current visual SLAM researches assume that the environment is stationary, which makes handling real-world scenarios such as hospitals a challenge. This paper proposes a method that effectively deals with SLAM problems for scenarios with dynamic objects, e.g., people and movable objects, based on the semantic descriptor extracted from images with help of a knowledge graph. Specifically, our method leverages a knowledge graph to construct a priori movement relationship between entities and establishes high-level semantic information. Built upon this knowledge graph, a semantic descriptor is constructed to describe the semantic information around key points, which is rotation-invariant and robust to illumination. The seamless integration of the knowledge graph and semantic descriptor helps eliminate the dynamic objects and improves the accuracy of tracking and positioning of robots in dynamic environments. Experiments are conducted using data acquired from healthcare facilities, and semantic maps are established to meet the needs of robots for delivering medical services. In addition, to compare with the state-of-the-art methods, a publicly available dataset is used in our evaluation. Compared with the state-of-the-art methods, our proposed method demonstrated great improvement with respect to both accuracy and robustness in dynamic environments. The computational efficiency is also competitive.

In this paper, we address one specific video retrieval problem in terms of human face. Given one query in forms of either a frame or a sequence from a person, we search the database and return the most relevant face videos, i.e., ones have the same class label with the query. Such problem is very challenging due to the large intra-class variations and the high request on the efficiency of video representations in terms of both time and space. To handle such challenges, this paper proposes a novel Deep Video Code (DVC) method which encodes video faces into compact binary codes. Specifically, we devise an end-to-end convolutional neural network (CNN) framework that takes face videos as training inputs, models each of them as a unified representation by temporal feature pooling operation, and finally projects the high-dimensional representations of both frames and videos into Hamming space to generate binary codes. In such Hamming space, distance of dissimilar pairs is larger than that of similar pairs by a margin. To this end, a novel bounded triplet hashing loss is elaborately designed, which takes all dissimilar pairs into consideration for each anchor point in a mini-batch, and the optimization of the loss function is smoother and more stable. Extensive experiments on challenging video face databases and general image/video datasets with comparison to the state-of-the-arts verify the effectiveness of our method in different kinds of retrieval scenarios.

Visual place recognition has attracted widespread research interest in multiple fields such as computer vision and robotics. Recently, researchers have employed advanced deep learning techniques to tackle this problem. While an increasing number of studies have proposed novel place recognition methods based on deep learning, few of them has provided a whole picture about how and to what extent deep learning has been utilized for this issue. In this paper, by delving into over 200 references, we present a comprehensive survey that covers various aspects of place recognition from deep learning perspective. We first present a brief introduction of deep learning and discuss its opportunities for recognizing places. After that, we focus on existing approaches built upon convolutional neural networks, including off-the-shelf and specifically designed models as well as novel image representations. We also discuss challenging problems in place recognition and present an extensive review of the corresponding datasets. To explore the future directions, we describe open issues and some new tools, for instance, generative adversarial networks, semantic scene understanding and multi-modality feature learning for this research topic. Finally, a conclusion is drawn for this paper.

Under big data, large number of features as well as their complex data types makes traditional feature selection and knowledge reasoning in CBR system not adapt to new condition. To solve these problems, first, this paper proposes Weighted Relative Probability Change of Solution Parameters (WRPCSP) algorithm to execute feature selection. Then, this paper integrates Bayesian network (BN) with CBR system for knowledge reasoning. Based on probability calculation and reasoning, WRPCSP algorithm together with BN allows the proposed CBR system to well work under big data. In addition, to overcome the efficiency problem caused by large number of features, this paper also proposes Group-Outside (GO) algorithm to assign the computing task of big data for parallel data processing. GO algorithm can make the computing capacity of Hadoop fully utilized to gain the least time costing for parallel data processing. Finally, lots of experiments are performed to validate the proposed method.

The refractive distortion and epipolar errors are analyzed in detail using simulations.A new hybrid optimization framework for two-view underwater calibration with local sliding (LS) procedure based on bundle adjustment.A new fully automatic method for two-view underwater structure and motion estimation.The accuracy of 3D reconstruction is evaluated quantitatively by comparing the results to ground truth 3D models.The influence of non-zero thickness and non-parallel interface on 3D reconstruction is studied experimentally. In an underwater imaging system, a refractive interface is introduced when a camera looks into the water-based environment, resulting in distorted images due to the refraction of light. Simply ignoring the refraction effect or using the lens radial distortion model causes erroneous 3D reconstruction. This paper deals with a general underwater imaging setup using two cameras, of which each camera is placed in a separate waterproof housing with a flat glass window. In order to handle refraction properly, a simplified refractive camera model is used in this paper. Based on two new concepts, namely the Ellipse of Refrax (EoR) and the Refractive Depth (RD) of a scene point, we derive two new formulations of the underwater known rotation structure and motion (SaM) problem. One gives a globally optimal solution and the other is robust to outliers. The constraint of known rotation is further relaxed by incorporating the robust known rotation SaM into a new hybrid optimization framework. Our method is able to simultaneously perform underwater camera calibration and 3D reconstruction automatically without using any calibration object or additional calibration device. In order to evaluate the performance and practical applicability of our method, extensive experiments using synthetic data, synthetically rendered images and real underwater images were carried out. The experimental results demonstrate that the proposed method can significantly improve the accuracy of the reconstructed 3D structure (within 0.78 mm for an object of dimension over 200 mm compared with the ground truth model captured by a land-based system) and of the system parameters for underwater applications. Compared with bundle adjustment using the refractive camera model initialized with traditional 3D reconstruction methods, our proposed optimization method has significantly better completeness and accuracy and lower 3D errors in the reconstructed models.

Features of G-lets Edge Operator: Geometry of objects in the image are preserved.Edges are continuous.Tracking outlines of small objects in cluttered images are possible.Edges in noisy images without losing continuity is shown to be possible.Computations are light due to sparse operations. A new edge detection technique using transformation groups based G-lets filters is proposed in this paper. Discretizing gradients seem to produce discontinuity in classic edge detectors. No particular filter is capable of identifying meaningful edges at all scales and it increases computations with a multiscale approach. It is a challenge to get localized edges without spurious ones due to noise and integrate the obtained edges into meaningful object boundaries. Without breaking edge continuity and strictly localizing edges requires that filters do not blur the image during preprocessing. G-lets filters are found to be capable of performing well in most type of images including natural, noisy, low resolution and synthetic. In this paper, an edge detection algorithm using G-lets filters which are built by direct factorization of linear transformation matrices using irreducible representations is proposed. A multiresolution approach is shown to enhance the possibility of detecting faint edges. An edge tracing algorithm is presented to produce the edge image. The computational cost involved is comparatively lesser than existing filters. It is found that the geometries in the original image are preserved in the edge image. The edge tracing algorithm is capable of constructing object boundaries without the inner textures in a way that is not completely dependent on intensity thresholding. G-lets filters and the edge operator is found to be a promising algorithm for drastically bringing down the computations needed for realtime applications. The results are compared with BSDS500 boundary detection dataset using pb and global pb detectors.

In this paper, a method to improve texture analysis performance in biometrics by adjusting image sharpness is presented. Images of high sharpness are usually considered as high quality data in texture analysis. Therefore, the imaging sensor and lens are carefully selected and calibrated in an image acquisition system in order to capture clear images. However, the results of our experiments show that the performance of texture analysis in biometrics can be improved by filtering clear images to lower sharpness. The experiments were conducted on the PolyU Palmprint Database using two algorithms (CompCode and POC), as well as on the CASIA Iris Database using IrisCode. In this paper, a filtering method using Gaussian filters is adopted to the images during the pre-processing stage to adjust the image sharpness. Results indicate that there is an optimal range of image sharpness and if all the images are filtered to this range, the performance of texture analysis on the whole dataset will be optimized. A scheme is also proposed to find the optimal range and to filter an image to the optimal range. HighlightsImprove texture analysis performance in biometrics by adjusting image sharpness.We propose a hypothesis that there is an optimal range [E1,E2].When image sharpness is tuned to the range, recognition performance can be improved.Find the range on PolyU Palmprint Database and CASIA Iris Database.EER of palmprint recognition improved by 19.2% and iris recognition improved by 11.5%.

Color image segmentation is a crucial step in many computer vision and pattern recognition applications. This paper introduces an adaptive and unsupervised approach based on Voronoi regions to solve the color image segmentation problem. The proposed method uses a hybrid of spatial and feature space Dirichlet tessellation followed by inter-Voronoi region proximal cluster merging to automatically find the number of clusters and cluster centroids in an image. Since, the Voronoi regions are much smaller compared to the whole image, Voronoi region-wise clustering improves the efficiency and accuracy of the number of clusters and cluster centroid estimation process. The proposed method was compared with four other adaptive unsupervised cluster-based image segmentation algorithms on three image segmentation evaluation benchmarks. The experimental results reported in this paper confirm that the proposed method outperforms the existing algorithms in terms of the image segmentation quality and results in much lower average execution time per image. HighlightsWe propose a hybrid of Dirichlet tessellation to automatically segment an image.Spatial Dirichlet tessellation adaptively divides an image into Voronoi regions.Feature space Dirichlet tessellation adaptively clusters pixels in Voronoi regions.Inter-Voronoi region proximal cluster merging automatically finds the final clusters.

This paper introduces a deep model called Deep Sparse-Coding Network (DeepSCNet) to combine the advantages of Convolutional Neural Network (CNN) and sparse-coding techniques for image feature representation. DeepSCNet consists of four type of basic layers: The sparse-coding layer performs generalized linear coding for local patch within the receptive field by replacing the convolution operation in CNN into sparse-coding. The Pooling layer and the Normalization layer perform identical operations as that in CNN. And finally the Map reduction layer reduces CPU/memory consumption by reducing the number of feature maps before stacking with the following layers. These four type of layers can be easily stacked to construct a deep model for image feature learning. The paper further discusses the multi-scale, multi-locality extension to the basic DeepSCNet, and the overall approach is fully unsupervised. Compared to CNN, training DeepSCNet is relatively easier even with training set of moderate size. Experiments show that DeepSCNet can automatically discover highly discriminative feature directly from raw image pixels. HighlightsWe propose a novel deep model combining the advantages of CNN and sprase-coding technique.Image representation can be obtained directly from raw pixels.Multi-scale and Multi-Locality extensions are proposed to boost the recognition accuracy.The classification accuracy is very promising among traditional unsupervised methods.

The single sample per person (SSPP) problem is a great challenge for real-world face recognition systems. In an SSPP scenario, there is always a large gap between a normal sample enrolled in the gallery set and the non-ideal probe sample. In this paper, we propose a new face recognition method, called decision pyramid classifier (DPC), to solve SSPP problems with large appearance variations (e.g., illumination, expression and partly occlusions). Unlike the conventional image partitioning methods, the proposed DPC is a nonparametric method which does not require a training process. In the data preprocessing phase of DPC, we divide each training image into multiple non-overlapping local blocks and respectively extract features from each block to generate the training feature set. For an unseen image, DPC requires obtaining its features using the exactly same preprocessing. By constructing a decision pyramid, we predict the final category of the unseen face image. Experimental results show that DPC possesses higher recognition rate than other related face recognition methods. This paper proposes a decision pyramid classifier (DPC) for single sample face recognition.DPC is a multilayer structure and a non-parametric method.Experimental results show that DPC has good performance on face recognition.

Improving the safety of transportation systems attracts lots of attention. Researchers introduced their methods to detect and analyze the vehicle and the pedestrian on the road to accomplish this goal. However, the bicycle is also a significant factor of the safety on a road. In this paper, a bicycle detector for side-view image is proposed based on the observation that a bicycle consists of two wheels in the form of ellipse shapes and a frame in the form of two triangles. Through the proposed triangle detection algorithm, the bicycle model and the geometric constraints on the relationship between the triangles and ellipses, the computation is fast according to the sample implementation and the evaluation of the reduced data amount. Besides, the training process is unnecessary and only single image is required for our algorithm. The experimental results are also given in this paper to show the practicability and the performance of the proposed bicycle model and bicycle detection algorithm.

As a principled method for partial correlation estimation, sparse inverse covariance estimation (SICE) has been employed to model brain connectivity networks, which holds great promise for brain disease diagnosis. For each subject, the SICE method naturally leads to a set of connectivity networks with various sparsity. However, existing methods usually select a single network from them for classification and the discriminative power of this set of networks has not been fully exploited. This paper argues that the connectivity networks at different sparsity levels present complementary connectivity patterns and therefore they should be jointly considered to achieve high classification performance. In this paper, we propose a subject-adaptive method to integrate multiple SICE networks as a unified representation for classification. The integration weight is learned adaptively for each subject in order to endow the method with the flexibility in dealing with subject variations. Furthermore, to respect the manifold geometry of SICE networks, Stein kernel is employed to embed the manifold structure into a kernel-induced feature space, which allows a linear integration of SICE networks to be designed. The optimization of the integration weight and the classification of the integrated networks are performed via a sparse representation framework. Through our method, we provide a unified and effective network representation that is transparent to the sparsity level of SICE networks, and can be readily utilized for further medical analysis. Experimental study on ADHD and ADNI data sets demonstrates that the proposed integration method achieves notable improvement of classification performance in comparison with methods using a single sparsity level of SICE networks and other commonly used integration methods, such as Multiple Kernel Learning.

Recently, many researchers have attempted to classify Facial Attributes (FAs) by representing characteristics of FAs such as attractiveness, age, smiling and so on. In this context, recent studies have demonstrated that visual FAs are a strong background for many applications such as face verification, face search and so on. However, Facial Attribute Classification (FAC) in a wide range of attributes based on the regression representation -predicting of FAs as real-valued labels- is still a significant challenge in computer vision and psychology. In this paper, a regression model formulation is proposed for FAC in a wide range of FAs (e.g. 73 FAs). The proposed method accommodates real-valued scores to the probability of what percentage of the given FAs is present in the input image. To this end, two simultaneous dictionary learning methods are proposed to learn the regression and identity feature dictionaries simultaneously. Accordingly, a multi-level feature extraction is proposed for FAC. Then, four regression classification methods are proposed using a regression model formulated based on dictionary learning, SRC and CRC. Convincing results are acquired to handle a wide range of FAs and represent the probability of FAs on the PubFig, LFW, Groups and 10k US Adult Faces databases compared to several state-of-the-art methods. This paper proposes a method for regression facial attributes classification.We propose two simultaneous optimization problems for Facial Attribute Classification.A multilevel feature extraction method was proposed to discriminate the facial features.Promising results were obtained to classify facial attributes on the PubFig, Groups, 10k US adult and LFW databases.

Local Binary Patterns (LBP) have emerged as one of the most prominent and widely studied local texture descriptors. Truly a large number of LBP variants has been proposed, to the point that it can become overwhelming to grasp their respective strengths and weaknesses, and there is a need for a comprehensive study regarding the prominent LBP-related strategies. New types of descriptors based on multistage convolutional networks and deep learning have also emerged. In different papers the performance comparison of the proposed methods to earlier approaches is mainly done with some well-known texture datasets, with differing classifiers and testing protocols, and often not using the best sets of parameter values and multiple scales for the comparative methods. Very important aspects such as computational complexity and effects of poor image quality are often neglected.In this paper, we provide a systematic review of current LBP variants and propose a taxonomy to more clearly group the prominent alternatives. Merits and demerits of the various LBP features and their underlying connections are also analyzed. We perform a large scale performance evaluation for texture classification, empirically assessing forty texture features including thirty two recent most promising LBP variants and eight non-LBP descriptors based on deep convolutional networks on thirteen widely-used texture datasets. The experiments are designed to measure their robustness against different classification challenges, including changes in rotation, scale, illumination, viewpoint, number of classes, different types of image degradation, and computational complexity. The best overall performance is obtained for the Median Robust Extended Local Binary Pattern (MRELBP) feature. For textures with very large appearance variations, Fisher vector pooling of deep Convolutional Neural Networks is clearly the best, but at the cost of very high computational complexity. The sensitivity to image degradations and computational complexity are among the key problems for most of the methods considered. HighlightsA taxonomy and comprehensive survey of LBP variants.Characteristics of, and connections between LBP variants are provided.A comprehensive experimental evaluation of 32 LBP methods.Comparison of 32 LBP variants with 8 deep ConvNets features.Evaluation of robustness to rotation, illumination, scale and noise changes.Comparison of computational complexity of forty variants.