Search

Visual recognition under adverse conditions is a very important and challenging problem of high practical value, due to the ubiquitous existence of quality distortions during image acquisition, transmission, or storage. While deep neural networks have been extensively exploited in the techniques of low-quality image restoration and high-quality image recognition tasks, respectively, few studies have been done on the important problem of recognition from very low-quality images. This paper proposes a deep learning-based framework for improving the performance of image and video recognition models under adverse conditions, using robust adverse pre-training or its aggressive variant. The robust adverse pre-training algorithms leverage the power of pre-training and generalize the conventional unsupervised pre-training and data augmentation methods. We further develop a transfer learning approach to cope with real-world datasets of unknown adverse conditions. The proposed framework is comprehensively evaluated on a number of image and video recognition benchmarks, and obtains significant performance improvements under various single or mixed adverse conditions. Our visualization and analysis further add to the explainability of the results. [ABSTRACT FROM AUTHOR]

Presents architectures for M4R shape encoding and decoding. Information on the M4R method; Description of pipelined architectures for encoding and decoding; Identity of the three coding modes used by intra and intershape coding; Details on the development of architectures for M4R encoding and decoding.

We explain that the task of multiple rigid motion segmentation and estimation from image feature point correspondences demands an estimator of high robustness. We show that a heuristics-based partial modeling approach can be used to develop a highly robust estimator called the MF estimator for general regression, where ‘MF’ represents an abbreviation of Model Fitting. Finally, we provide experimental results in estimating single rigid motion from a mixture of 2D-2D (or image-image), 3D-2D (or range-image), and 3D-3D (or range-range) corresponding point data by using the proposed MF estimator. As will be seen. only four well matched corresponding point pairs are needed to get a good estimate of motion parameters no matter how m(my mismatched corresponding point pairs (or outliers) occur. The article represents an initial effort towards robust image-motion analysis. [ABSTRACT FROM AUTHOR]

The estimation of the three-dimensional (3-D) motion parameters of a rigid body is a very important subject in scene analysis and trajectory prediction. Motion parameters can be estimated from two sets of object feature points before and after the motion In general, the matching correspondences of the feature points are available, and the motion parameters can be estimated by solving equations associated with the correspondences. In this paper, we present a new method for motion estimation from 3-D data without requiring the knowledge of matching correspondences. In the noise-free case. this approach identifies four candidates for the rotation matrix. The rotation matrix giving the best match of the point features can then be selected from the four candidates and the matching correspondences are subsequently established. Possible ambiguities due to symmetrical feature points are also discussed in this paper. In the presence of noise, this method provides an initial estimate of the motion, which is then used to establish the matching correspondences. Subsequently, a new estimate of the motion parameters can be obtained with the established matching correspondence information. The effects of random zero-mean noise are studied. Simulated results are shown to demonstrate the effectiveness and accuracy of this technique. [ABSTRACT FROM AUTHOR]

Recent advances in multimedia ontology have resulted in a number of concept models, e.g., large-scale concept for multimedia and Mediamill 101, which are accessible and public to other researchers. However, most current research effort still focuses on building new concepts from scratch, very few work explores the appropriate method to construct new concepts upon the existing models already in the warehouse. To address this issue, we propose a new framework in this paper, termed LEarning Structured Model by Probabilistic loGic Ontology Tree for MultiM edia (LEGO 1 -MM), which can seamlessly integrate both the new target training examples and the existing primitive concept models to infer the more complex concept models. LEGO-MM treats the primitive concept models as the lego toy to potentially construct an unlimited vocabulary of new concepts. Specifically, we first formulate the logic operations to be the lego connectors to combine the existing concept models hierarchically in probabilistic logic ontology trees. Then, we incorporate new target training information simultaneously to efficiently disambiguate the underlying logic tree and correct the error propagation. Extensive experiments are conducted on a large vehicle domain data set from ImageNet. The results demonstrate that LEGO-MM has significantly superior performance over the existing state-of-the-art methods, which build new concept models from scratch. [ABSTRACT FROM AUTHOR]

Introduces articles about graphical models in computer vision published in the July 2003 issue of the periodical 'IEEE Transactions on Pattern Analysis and Machine Intelligence.' Stereo matching using belief propagation; Unsupervised learning of human motion; Decision making and uncertainty management in a three-dimensional reconstruction system.

Abstract: In this paper, we propose a multiple-metric learning algorithm to learn jointly a set of optimal homogenous/heterogeneous metrics in order to fuse the data collected from multiple sensors for joint classification. The learned metrics have the potential to perform better than the conventional Euclidean metric for classification. Moreover, in the case of heterogenous sensors, the learned multiple metrics can be quite different, which are adapted to each type of sensor. By learning the multiple metrics jointly within a single unified optimization framework, we can learn better metrics to fuse the multi-sensor data for a joint classification. Furthermore, we also exploit multi-metric learning in a kernel induced feature space to capture the non-linearity in the original feature space via kernel mapping. [Copyright &y& Elsevier]

This paper investigates the joint-structured-sparsity-based methods for transient acoustic signal classification with multiple measurements. By joint structured sparsity, we not only use the sparsity prior for each measurement but we also exploit the structural information across the sparse representation vectors of multiple measurements. Several different sparse prior models are investigated in this paper to exploit the correlations among the multiple measurements with the notion of the joint structured sparsity for improving the classification accuracy. Specifically, we propose models with the joint structured sparsity under different assumptions: same sparse code model, common sparse pattern model, and a newly proposed joint dynamic sparse model. For the joint dynamic sparse model, we also develop an efficient greedy algorithm to solve it. Extensive experiments are carried out on real acoustic data sets, and the results are compared with the conventional discriminative classifiers in order to verify the effectiveness of the proposed method. [ABSTRACT FROM PUBLISHER]

Abstract: Random walk was first introduced by Karl Pearson in 1905 and has inspired many research works in different fields. In recent years, random walk has been adopted in information network research, for example, ranking and similarity estimation. In this paper, we introduce a new model called RankCompete, which allows multiple random walkers in the same network (existing work mostly focus on random walks of a single category). By introducing the “competition” concept into the random walk framework, our method can fulfill both clustering and ranking tasks simultaneously and thus provide an effective analysis tool for networks. Compared with the traditional network ranking approaches, our new method focuses more on the structure of specialized clusters. Compared with the traditional graph clustering approaches, our new method provides a faster and more intuitive way to group the network nodes. We validate our approach on both bibliography networks and visual information networks, and the results show that our approach can obtain 100% perfect clustering results in clustering the DBLP 20 conferences and outperform the state-of-the-art on the Cora dataset. Furthermore, we show that our method can be effectively used to summarize personal photo collections. [Copyright &y& Elsevier]

This paper details the authors' efforts to push the baseline of emotion recognition performance on the Geneva Multimodal Emotion Portrayals (GEMEP) Facial Expression Recognition and Analysis database. Both subject-dependent and subject-independent emotion recognition scenarios are addressed in this paper. The approach toward solving this problem involves face detection, followed by key-point identification, then feature generation, and then, finally, classification. An ensemble of features consisting of hierarchical Gaussianization, scale-invariant feature transform, and some coarse motion features have been used. In the classification stage, we used support vector machines. The classification task has been divided into person-specific and person-independent emotion recognitions using face recognition with either manual labels or automatic algorithms. We achieve 100% performance for the person-specific one, 66% performance for the person-independent one, and 80% performance for overall results, in terms of classification rate, for emotion recognition with manual identification of subjects. [ABSTRACT FROM AUTHOR]

Abstract: We consider the problem of automatically recognizing a human face from its multi-view images with unconstrained poses. We formulate the multi-view face recognition task as a joint sparse representation model and take advantage of the correlations among the multiple views for face recognition using a novel joint dynamic sparsity prior. The proposed joint dynamic sparsity prior promotes shared joint sparsity patterns among the multiple sparse representation vectors at class-level, while allowing distinct sparsity patterns at atom-level within each class to facilitate a flexible representation. Extensive experiments on the CMU Multi-PIE face database are conducted to verify the efficacy of the proposed method. [Copyright &y& Elsevier]

An introduction is presented in which the editor discusses various reports within the issue on topics including algorithm for manifold regularization, generalization of TensorFace, and human action recognition.

This paper presents a novel time varying dynamic Bayesian network (TVDBN) model for the analysis of nonstationary sequences which are of interest in many fields. The changing network structure and parameter in TVDBN are treated as random processes whose values at each time epoch determine a stationary DBN model; this DBN model is then used to specify the distribution of data sequence at the time epoch. Under such a hierarchical formulation, the changing state of network can be incorporated into the Bayesian framework straightforwardly. The network state is assumed to transit smoothly in the joint space of numerical parameter and graphical topology so that we can achieve robust online network learning even without abundant observations. Particle filtering is employed to dynamically update current network state as well as infer hidden data values. We implement our time varying model for data sequences of multinomial and Gaussian distributions, while the general model framework can be used for any other distribution. Simulations on synthetic data and evaluations on video sequences both demonstrate that the proposed TVDBN is effective in modeling nonstationary sequences. Comprehensive comparisons have been made against existing nonstationary models, and our proposed model is shown to be the top performer. [ABSTRACT FROM AUTHOR]

Detecting text and caption from videos is important and in great demand for video retrieval, annotation, indexing, and content analysis. In this paper, we present a corner based approach to detect text and caption from videos. This approach is inspired by the observation that there exist dense and orderly presences of corner points in characters, especially in text and caption. We use several discriminative features to describe the text regions formed by the corner points. The usage of these features is in a flexible manner, thus, can be adapted to different applications. Language independence is an important advantage of the proposed method. Moreover, based upon the text features, we further develop a novel algorithm to detect moving captions in videos. In the algorithm, the motion features, extracted by optical flow, are combined with text features to detect the moving caption patterns. The decision tree is adopted to learn the classification criteria. Experiments conducted on a large volume of real video shots demonstrate the efficiency and robustness of our proposed approaches and the real-world system. Our text and caption detection system was recently highlighted in a worldwide multimedia retrieval competition, Star Challenge, by achieving the superior performance with the top ranking. [ABSTRACT FROM AUTHOR]

Abstract: Acoustic Event Detection (AED) aims to identify both timestamps and types of events in an audio stream. This becomes very challenging when going beyond restricted highlight events and well controlled recordings. We propose extracting discriminative features for AED using a boosting approach, which outperform classical speech perceptual features, such as Mel-frequency Cepstral Coefficients and log frequency filterbank parameters. We propose leveraging statistical models better fitting the task. First, a tandem connectionist-HMM approach combines the sequence modeling capabilities of the HMM with the high-accuracy context-dependent discriminative capabilities of an artificial neural network trained using the minimum cross entropy criterion. Second, an SVM–GMM-supervector approach uses noise-adaptive kernels better approximating the KL divergence between feature distributions in different audio segments. Experiments on the CLEAR 2007 AED Evaluation set-up demonstrate that the presented features and models lead to over 45% relative performance improvement, and also outperform the best system in the CLEAR AED Evaluation, on detection of twelve general acoustic events in a real seminar environment. [ABSTRACT FROM AUTHOR]

We consider the problem of estimating 3-D human body pose from visual signals within a discriminative framework. It is challenging because there is a wide gap between complex 3-D human motion and planar visual observation, which makes this a severely ill-conditioned problem. In this paper, we focus on three critical factors to tackle human body pose estimation, namely, feature extraction, learning algorithm, and camera utilization. On the feature level, we describe images using the salient interest points represented by scale-invariant feature transform (SIFT)-like descriptors, in which the position, appearance, and local structural information are encoded simultaneously. On the learning algorithm level, we propose to use Gaussian processes and multiple linear (ML) regression to model the mapping between poses and features. Fusing image information from multiple cameras in different views is of great interest to us on the camera level. We make a comprehensive evaluation on the HumanEva database and get two meaningful insights into the three crucial aspects for human pose estimation: 1) although the choice of feature is very important to the problem, once the learning algorithm becomes efficient, the choice of feature is no longer critical, and 2) the impact of information combination from multiple cameras on pose estimation is closely related to not only the quantity of image information, but also its quality. In most cases, it is true that the more information is involved, the better results can be achieved. But when the information quantity is the same, the differences in quality will lead to totally different performance. Furthermore, dense evaluations demonstrate that our approach is an accurate and robust solution to the human body pose estimation problem. [ABSTRACT FROM AUTHOR]

Abstract: We present a novel Gaussianized vector representation for scene images by an unsupervised approach. Each image is first encoded as an ensemble of orderless bag of features. A global Gaussian Mixture Model (GMM) learned from all images is then used to randomly distribute each feature into one Gaussian component by a multinomial trial. The posteriors of the feature on all the Gaussian components serve as the parameters of the multinomial distribution. Finally, the normalized means of the features distributed in every Gaussian component are concatenated to form a supervector, which is a compact representation for each scene image. We prove that these supervectors observe the standard normal distribution. The Gaussianized vector representation is a more generalized form of the widely used histogram representation. Our experiments on scene categorization tasks using this vector representation show significantly improved performance compared with the histogram-of-features representation. This paper is an extended version of our work that won the IBM Best Student Paper Award at the 2008 International Conference on Pattern Recognition (ICPR 2008) (). [Copyright &y& Elsevier]

Subspace learning techniques for face recognition have been widely studied in the past three decades. In this paper, we study the problem of general subspace-based face recognition under the scenarios with spatial misalignments and/or image occlusions. For a given subspace derived from training data in a supervised, unsupervised, or semi-supervised manner, the embedding of a new datum and its underlying spatial misalignment parameters are simultaneously inferred by solving a constrained l1 norm optimization problem, which minimizes the l1 error between the misalignment-amended image and the image reconstructed from the given subspace along with its principal complementary subspace. A byproduct of this formulation is the capability to detect the underlying image occlusions. Extensive experiments on spatial misalignment estimation, image occlusion detection, and face recognition with spatial misalignments and/or image occlusions all validate the effectiveness of our proposed general formulation for misalignment-robust face recognition. [ABSTRACT FROM AUTHOR]

The graph construction procedure essentially determines the potentials of those graph-oriented learning algorithms for image analysis. In this paper, we propose a process to build the so-called directed l¹-graph, in which the vertices involve all the samples and the in-going edge weights to each vertex describe its l¹-norm driven reconstruction from the remaining samples and the noise. Then, a series of new algorithms for various machine learning tasks, e.g., data clustering, subspace learning, and semisupervised learning, are derived upon the l¹-graphs. Compared with the conventional k-nearest-neighbor graph and ε-ball graph, the l¹-graph possesses the advantages: 1) greater robustness to data noise, 2) automatic sparsity, and 3) adaptive neighborhood for individual datum. Extensive experiments on three real-world datasets show the consistent superiority of l¹-graph over those classic graphs in data clustering, subspace learning, and semi-supervised learning tasks. [ABSTRACT FROM AUTHOR]

Abstract: In recent years, the spectral clustering method has gained attentions because of its superior performance. To the best of our knowledge, the existing spectral clustering algorithms cannot incrementally update the clustering results given a small change of the data set. However, the capability of incrementally updating is essential to some applications such as websphere or blogsphere. Unlike the traditional stream data, these applications require incremental algorithms to handle not only insertion/deletion of data points but also similarity changes between existing points. In this paper, we extend the standard spectral clustering to such evolving data, by introducing the incidence vector/matrix to represent two kinds of dynamics in the same framework and by incrementally updating the eigen-system. Our incremental algorithm, initialized by a standard spectral clustering, continuously and efficiently updates the eigenvalue system and generates instant cluster labels, as the data set is evolving. The algorithm is applied to a blog data set. Compared with recomputation of the solution by the standard spectral clustering, it achieves similar accuracy but with much lower computational cost. It can discover not only the stable blog communities but also the evolution of the individual multi-topic blogs. The core technique of incrementally updating the eigenvalue system is a general algorithm and has a wide range of applications—as well as incremental spectral clustering—where dynamic graphs are involved. This demonstrates the wide applicability of our incremental algorithm. [Copyright &y& Elsevier]

The success of bilinear subspace learning heavily depends on reducing correlations among features along rows and columns of the data matrices. In this work, we study the problem of rearranging elements within a matrix in order to maximize these correlations so that information redundancy in matrix data can be more extensively removed by existing bilinear subspace learning algorithms. An efficient iterative algorithm is proposed to tackle this essentially integer programming problem. In each step, the matrix structure is refined with a constrained Earth Mover's Distance procedure that incrementally rearranges matrices to become more similar to their low-rank approximations, which have high correlation among features along rows and columns. In addition, we present two extensions of the algorithm for conducting supervised bilinear subspace learning. Experiments in both unsupervised and supervised bilinear subspace learning demonstrate the effectiveness of our proposed algorithms in improving data compression performance and classification accuracy. [ABSTRACT FROM AUTHOR]

The article discusses various reports published within the issue including one on the introduction of an integrated system for recognizing actions and objects, another on a probabilistic framework for three-dimensional (3D) visual object representation and one on the detection of repetitive patterns within deformed 2D lattices.

In this correspondence, we study the extra-factor estimation problem with the assumption that the training image ensemble is expressed as an nth-order tensor with the nth-dimension characterizing all features for an image and other dimensions for different extra factors, such as illuminations, poses, and identities. To overcome the local minimum issue of conventional algorithms designed for this problem, we present a novel statistical learning framework called mode-kn Factor Analysis for obtaining a closed-form solution to estimating the extra factors of any test image. In the learning stage, for the kth (k ≠ n) dimension of the data tensor, the mode-kn patterns are constructed by concatenating the feature dimension and the kth extra-factor dimension, and then a mode-kn factor analysis model is learnt based on the mode-kn patterns unfolded from the original data tensor. In the inference stage, for a test image, the mode classification of the kth dimension is performed within a probabilistic framework. The advantages of mode-kn factor analysis over conventional tensor analysis algorithms are twofold: 1) a closed-form solution, instead of iterative sub-optimal solution as conventionally, is derived for estimating the extra-factor mode of any test image; and 2) the classification capability is enhanced by interacting with the process of synthesizing data of all other modes in the kth dimension. Experiments on the Pointing'04 and CMU PIE databases for pose and illumination estimation both validate the superiority of the proposed algorithm over conventional algorithms for extra-factor estimation. [ABSTRACT FROM AUTHOR]

We present a supervised learning approach for object-category specific restoration, recognition, and segmentation of images which are blurred using an unknown kernel. The novelty of this work is a multilayer graphical model which unifies the lowlevel vision task of restoration and the high-level vision task of recognition in a cooperative framework. The graphical model is an interconnected two-layer Markov random field. The restoration layer accounts for the compatibility between sharp and blurred images and models the association between adjacent patches in the sharp image. The recognition layer encodes the entity class and its location in the underlying scene. The potentials are represented using nonparametric kernel densities and are learnt from training data. Inference is performed using nonparametric belief propagation. Experiments demonstrate the effectiveness of our model for the restoration and recognition of blurred license plates as well as face images. [ABSTRACT FROM AUTHOR]

Precise 3-D head pose estimation plays a significant role in developing human-computer interfaces and practical face recognition systems. This task is challenging due to the particular appearance variations caused by pose changes for a certain subject. In this paper, the pose data space is considered as a union of submanifolds which characterize different subjects, instead of a single continuous manifold as conventionally regarded. A novel manifold embedding algorithm dually supervised by both identity and pose information, called snchronized submanifold embedding (SSE), is proposed for person-independent precise 3-D pose estimation, which means that the testing subject may not appear in the model training stage. First, the submanifold of a certain subject is approximated as a set of simplexes constructed using neighboring samples. Then, these simplexized submanifolds from different subjects are embedded by synchronizing the locally propagated poses within the simplexes and at the same time maximizing the intrasubmanifold variances. Finally, the pose of a new datum is estimated as the propagated pose of the nearest point within the simplex constructed by its nearest neighbors in the dimensionality reduced feature space. The experiments on the 3-D pose estimation database, CHIL data for CLEAR07 evaluation, and the extended application for age estimation on FG-NET aging database, demonstrate the superiority of SSE over conventional regression algorithms as well as unsupervised manifold learning algorithms. [ABSTRACT FROM AUTHOR]

For the problem of image registration, the top few reliable correspondences are often relatively easy to obtain, while the overall matching accuracy may fall drastically as the desired correspondence number increases. In this paper, we present an efficient feature matching algorithm to employ sparse reliable correspondence priors for piloting the feature matching process. First, the feature geometric relationship within individual image is encoded as a spatial graph, and the pairwise feature similarity is expressed as a bipartite similarity graph between two feature sets; then the geometric neighborhood of the pairwise assignment is represented by a categorical product graph, along which the reliable correspondences are propagated; and finally a closed-form solution for feature matching is deduced by ensuring the feature geometric coherency as well as pairwise feature agreements. Furthermore, our algorithm is naturally applicable for incorporating manual correspondence priors for semi-supervised feature matching. Extensive experiments on both toy examples and real-world applications demonstrate the superiority of our algorithm over the state-of-the-art feature matching techniques. [ABSTRACT FROM AUTHOR]

Automated analysis of human affective behavior has attracted increasing attention from researchers in psychology, computer science, linguistics, neuroscience, and related disciplines. However, the existing methods typically handle only deliberately displayed and exaggerated expressions of prototypical emotions, despite the tact that deliberate behavior differs in visual appearance, audio profile, and timing from spontaneously occurring behavior. To address this problem, efforts to develop algorithms that can process naturally occurring human affective behavior have recently emerged. Moreover, an increasing number of efforts are reported toward multimodal fusion for human affect analysis, including audiovisual fusion, linguistic and paralinguistic fusion, and multicue visual fusion based on facial expressions, head movements, and body gestures. This paper introduces and surveys these recent advances. We first discuss human emotion perception from a psychological perspective. Next, we examine available approaches for solving the problem of machine understanding of human affective behavior and discuss important issues like the collection and availability of training and test data. We finally outline some of the scientific and engineering challenges to advancing human affect sensing technology. [ABSTRACT FROM AUTHOR]

Recent research has demonstrated the success of supervised dimensionality reduction algorithms 2DLDA and 2DMFA, which are based on the image-as-matrix representation, in small sample size cases. To solve the convergence problem in 2DLDA and 2DMFA, we propose in this work two new schemes, called Null Space based 2DLDA (NS2DLDA) and Null Space based 2DMFA (NS2DMFA), and apply them to the challenging multi-view face recognition task. First, we convert each 2-D face image (matrix) into a vector and compute the first projection matrix P1 from the null space of the intra-class scatter matrix, such that the samples from the same class are projected to the same point. Then the data are projected and reconstructed with P1. Finally, we re-organize the reconstructed datum into a matrix and then compute the second projection direction P2, in the form of a Kronecker product of two matrices, by maximizing the inter-class scatter. A proof of algorithmic convergence is provided. The experiments on two benchmark multi-view face databases, the CMU PIE and FERET databases, demonstrate that NS2DLDA outperforms Fisherface, Null Space LDA (NSLDA) and 2DLDA. Additionally, NS2DMFA is also demonstrated to be more accurate than MFA and 2DMFA for face recognition. [ABSTRACT FROM AUTHOR]

Conventional subspace learning or recent feature extraction methods consider globality as the key criterion to design discriminative algorithms for image classification. We demonstrate in this paper that applying the local manner in sample space, feature space, and learning space via linear sub-space learning can sufficiently boost the discriminating power, as measured by discriminating power coefficient (DPC). The proposed solution achieves good classification accuracy gains and shows computationally efficient. Particularly, we approximate the global nonlinearity through a multimodal localized piecewise subspace learning framework, in which three locality criteria can work individually or jointly for any new subspace learning algorithm design. It turns out that most existing subspace learning methods can be unified in such a common framework embodying either the global or local learning manner. On the other hand, we address the problem of numerical difficulty in the large-size pattern classification case, where many local variations cannot be adequately handled by a single global model. By localizing the modeling, the classification error rate estimation is also localized and thus it appears to be more robust and flexible for the model selection among different model candidates. As a new algorithm design based on the proposed framework, the query-driven locally adaptive (QDLA) mixture-of-experts model for robust face recognition and head pose estimation is presented. Experiments demonstrate the local approach to be effective, robust, and fast for large size, multiclass, and multivariance data sets. [ABSTRACT FROM AUTHOR]

Factor analysis (FA) seeks to reveal the relationship between an observed vector variable and a latent variable of reduced dimension. It has been widely used in many applications involving high-dimensional data, such as image representation and face recognition. An intrinsic limitation of FA lies in its potentially poor performance when the data dimension is high, a problem known as curse of dimensionality. Motivated by the fact that images are inherently matrices, we develop, in this brief, an FA model for matrix-variate variables and present an efficient parameter estimation algorithm. Experiments on both toy and real-world image data demonstrate that the proposed matrix-variant FA model is more efficient and accurate than the classical FA approach, especially when the observed variable is high-dimensional and the samples available are limited. [ABSTRACT FROM AUTHOR]

Estimating human age automatically via facial image analysis has lots of potential real-world applications, such as human computer interaction and multimedia communication. However, it is still a challenging problem for the existing computer vision systems to automatically and effectively estimate human ages. The aging process is determined by not only the person's gene, but also many external factors, such as health, having style, living location, and weather conditions. Males and females may' also age differently. The current age estimation performance is still not good enough for practical use and more effort has to be put into this research direction. In this paper, we introduce the age manifold learning scheme for extracting face aging features and design a locally adjusted robust regressor for learning and prediction of human ages. The novel approach improves the age estimation accuracy significantly over all previous methods. The merit of the proposed approaches for image-based age estimation is shown by extensive experiments on a large internal age database and the public available FG-NET database. [ABSTRACT FROM AUTHOR]

Principal Components Analysis (PCA) has traditionally been utilized with data expressed in the form of 1-D vectors, but there exists much data such as gray-level images, video sequences, Gabor-filtered images and so on, that are intrinsically in the form of second or higher order tensors. For representations of image objects in their intrinsic form and order rather than concatenating all the object data into a single vector, we propose in this paper a new optimal object reconstruction criterion with which the information of a high-dimensional tensor is represented as a much lower dimensional tensor computed from projections to multiple concurrent subspaces. In each of these subspaces, correlations with respect to one of the tensor dimensions are reduced, enabling better object reconstruction performance. Concurrent subspaces analysis (CSA) is presented to efficiently learn these subspaces in an iterative manner. In contrast to techniques such as PCA which vectorize tensor data, CSA's direct use of data in tensor form brings an enhanced ability to learn a representative subspace and an increased number of available projection directions. These properties enable CSA to outperform traditional algorithms in the common case of small sample sizes, where CSA can be effective even with only a single sample per class. Extensive experiments on images of faces and digital numbers encoded as second or third order tensors demonstrate that the proposed CSA outperforms PCA-based algorithms in object reconstruction and object recognition. [ABSTRACT FROM AUTHOR]

This paper presents a unified solution to three unsolved problems existing in face verification with subspace learning techniques: selection of verification threshold, automatic determination of subspace dimension, and deducing feature fusing weights. In contrast to previous algorithms which search for the projection matrix directly, our new algorithm investigates a similarity metric matrix (SMM). With a certain verification threshold, this matrix is learned by a semidefinite programming approach, along with the constraints of the kindred pairs with similarity larger than the threshold, and inhomogeneous pairs with similarity smaller than the threshold. Then, the subspace dimension and the feature fusing weights are simultaneously inferred from the singular value decomposition of the derived SMM. In addition, the weighted and tensor extensions are proposed to further improve the algorithmic effectiveness and efficiency, respectively. Essentially, the verification is conducted within an affine subspace in this new algorithm and is, hence, called the affine subspace for verification (ASV). Extensive experiments show that the ASV can achieve encouraging face verification accuracy in comparison to other subspace algorithms, even without the need to explore any parameters. [ABSTRACT FROM AUTHOR]

The authors make a case for human centered computing (HCC), which aims to focus a computer revolution on human needs and abilities. As digitized information and functionality evolve into a digital ecosystem upon which all humans increasingly rely, computing must also grow more reflective of and responsive to the ordinary lives of the diverse individuals who make up the world's population. HCC requires considering cognitive and social systems as well as physical locations and cultural environments. The authors distinguish between HCC and human-computer interaction, user-centered design, human computation, and other fields. They mention contributions from neuroscience, domain knowledge, data mining, sensor fusion, and multimodal action. An IEEE task force on HCC has been formed.

In this paper, a novel hybrid dimension reduction technique for classification is proposed based on the hybrid analysis of principal component analysis (PCA) and linear discriminant analysis (LDA). LDA is known for capturing the most discriminant features of the data in the projected space while PCA is known for preserving the most descriptive ones after projection. Our hybrid technique integrates discriminant and descriptive information and finds a richer set of alternatives beyond LDA and PCA in a 2-D parametric space, which fits a specific classification task and data distribution better. Theoretical study shows that our technique also alleviates the singularity problem of scatter matrix, which is caused by small training set, and increases the effective dimension of the projected subspace. In order to find the hybrid features adaptively and avoid exhaustive parameter searching, we further propose a boosted hybrid analysis method that incorporates a nonlinear boosting process to enhance a set of hybrid classifiers and combine them into a more accurate one. Compared with the other techniques that aim at combining PCA and LDA, our approaches are novel because our method finds alternatives to LDA and PCA in a 2-D parameter space and the boosting process provides enhancement and robust combination of the classifiers. Extensive experiments are conducted on benchmark and real image databases to compare our proposed methods with the state-of-the-art linear and nonlinear discriminant analysis techniques. The results show the superior performance of our hybrid analysis methods. [ABSTRACT FROM AUTHOR]

In this paper, we present the logarithmic total variation (LTV) model for face recognition under varying illumination, including natural lighting conditions, where we rarely know the strength, direction, or number of light sources. The proposed LTV model has the ability to factorize a single face image and obtain the illumination invariant facial structure, which is then used for face recognition. Our model is inspired by the SQI model but has better edge-preserving ability and simpler parameter selection. The merit of this model is that neither does it require any lighting assumption nor does it need any training. The LTV model reaches very high recognition rates in the tests using both Yale and CMU PIE face databases as well as a face database containing 765 subjects under outdoor lighting conditions. [ABSTRACT FROM AUTHOR]

Abstract: For learning-based tasks such as image classification, the feature dimension is usually very high. The learning is afflicted by the curse of dimensionality as the search space grows exponentially with the dimension. Discriminant-EM (DEM) proposed a framework by applying self-supervised learning in a discriminating subspace. This paper extends the linear DEM to a nonlinear kernel algorithm, Kernel DEM (KDEM), and evaluates KDEM extensively on benchmark image databases and synthetic data. Various comparisons with other state-of-the-art learning techniques are investigated for several tasks of image classification. Extensive results show the effectiveness of our approach. [Copyright &y& Elsevier]

Automatic classification is one of the basic tasks required in any pattern recognition and human computer interaction application. In this paper, we discuss training probabilistic classifiers with labeled and unlabeled data. We provide a new analysis that shows under what conditions unlabeled data can be used in learning to improve classification performance. We also show that, if the conditions are violated, using unlabeled data can be detrimental to classification performance. We discuss the implications of this analysis to a specific type of probabilistic classifiers, Bayesian networks, and propose a new structure learning algorithm that can utilize unlabeled data to improve classification. Finally, we show how the resulting algorithms are successfully employed in two applications related to human-Computer interaction and pattern recognition: facial expression recognition and face detection. [ABSTRACT FROM AUTHOR]

We present a user-centric system for visualization and layout for content-based image retrieval. Image features (visual and/or semantic) are used to display retrievals as thumbnails in a 2-D spatial layout or “configuration” which conveys all pair-wise mutual similarities. A graphical optimization technique is used to provide maximally uncluttered and informative layouts. Moreover, a novel subspace feature weighting technique can be used to modify 2-D layouts in a variety of context-dependent ways. An efficient computational technique for subspace weighting and re-estimation leads to a simple user-modeling framework whereby the system can learn to display query results based on layout examples (or relevance feedback) provided by the user. The resulting retrieval, browsing and visualization can adapt to the user's (time-varying) notions of content, context and preferences in style and interactive navigation. Monte Carlo simulations with machine-generated layouts as well as pilot user studies have demonstrated the ability of this framework to model or “mimic” users, by automatically generating layouts according to their preferences. [ABSTRACT FROM AUTHOR]

We present computer vision techniques for building dressed human models using images. We develop an algorithm for three-dimensional body reconstruction and texture mapping using contour, stereo, and texture information from several images and deformable superquadrics as the model parts. We demonstrate a novel vision technique for analysis of cloth draping behavior. This technique allows for estimation of cloth model parameters, such as bending properties, but can also be used to estimate the contact points between the body and clothing in the range data of dressed humans. Combined with our body reconstruction algorithm and additional constraints on the articulation model, the detection of the garment-body contact points allows construction of a dressed human model in which even the geometry that was covered by clothing in the available data is reasonably well estimated [ABSTRACT FROM PUBLISHER]

The emerging MPEG-4 standard supports the transmission and composition of facial animation with natural video. The new standard will include a facial animation parameter (FAP) set that is defined based on the study of minimal facial actions and is closely related to muscle actions. The FAP set enables model-based representation of natural or synthetic talking-head sequences and allows intelligible visual reproduction of facial expressions, emotions, and speech pronunciations at the receiver. This paper addresses the data-compression issue of talking heads and presents three methods for bit-rate reduction of FAPs. Compression efficiency is achieved by way of transform coding, principal component analysis, and FAP interpolation. These methods are independent of each other in nature and thus can be applied in combination to lower the bit-rate demand of FAPs, making possible the transmission of multiple talking heads over band-limited channels. The basic methods described here have been adopted into the MPEG-4 Visual Committee Draft and are readily applicable to other articulation data such as body animation parameters. The efficacy of the methods is demonstrated by both subjective and objective results [ABSTRACT FROM PUBLISHER]

Human age, as an important personal trait, can be directly inferred by distinct patterns emerging from the facial appearance. Derived from rapid advances in computer graphics and machine vision, computer-based age synthesis and estimation via faces have become particularly prevalent topics recently because of their explosively emerging real-world applications, such as forensic art, electronic customer relationship management, security control and surveillance monitoring, biometrics, entertainment, and cosmetology. Age synthesis is defined to rerender a face image aesthetically with natural aging and rejuvenating effects on the individual face. Age estimation is defined to label a face image automatically with the exact age (year) or the age group (year range) of the individual face. Because of their particularity and complexity, both problems are attractive yet challenging to computer-based application system designers. Large efforts from both academia and industry have been devoted in the last a few decades. In this paper, we survey the complete state-of-the-art techniques in the face image-based age synthesis and estimation topics. Existing models, popular algorithms, system performances, technical difficulties, popular face aging databases, evaluation protocols, and promising future directions are also provided with systematic discussions. [ABSTRACT FROM PUBLISHER]

This article presents a robust algorithm for automated registration of images related by rigid-body transformations. This algorithm uses a new region-based similarity metric, which enables accurate registration of images of large contrast differences. Region segmentation required by the metric is accomplished using a multiscale segmentation algorithm, and minimization of this metric is done using the Powell direction set method. Experimental results are presented to demonstrate that the algorithm is effective for aligning images from single or multiple imaging modalities without the use of any fiducial markers. © 1997 John Wiley & Sons, Inc. Int J Imaging Syst Technol, 8, 513–518, 1997 [ABSTRACT FROM AUTHOR]

We describe a curvature-based approach for estimating nonrigid motion of moving surfaces. We deal with conformal motion, which can be characterized by stretching of the surface. At each point, this stretching is equal in all directions but different for different points. The stretching function can be defined as additional (with global translation and rotation) motion parameter. We present a new algorithm for local stretching recovery from Gaussian curvature, based on polynomial (linear and quadratic) approximations of the stretching function. It requires point correspondences between time frames but not the complete knowledge of nonrigid transformation. Experiments on simulated and real data are performed to illustrate performance and accuracy of derived algorithms. Noise sensitivity of the algorithm is also evaluated using Gaussian noise on simulated data. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization (NMF) is a popular technique for finding parts-based, linear representations of nonnegative data. It has been successfully applied in a wide range of applications such as pattern recognition, information retrieval, and computer vision. However, NMF is essentially an unsupervised method and cannot make use of label information. In this paper, we propose a novel semi-supervised matrix decomposition method, called Constrained Nonnegative Matrix Factorization (CNMF), which incorporates the label information as additional constraints. Specifically, we show how explicitly combining label information improves the discriminating power of the resulting matrix decomposition. We explore the proposed CNMF method with two cost function formulations and provide the corresponding update solutions for the optimization problems. Empirical experiments demonstrate the effectiveness of our novel algorithm in comparison to the state-of-the-art approaches through a set of evaluations based on real-world applications. [ABSTRACT FROM PUBLISHER]

We consider the active learning problem, which aims to select the most representative points. Out of many existing active learning techniques, optimum experimental design (OED) has received considerable attention recently. The typical OED criteria minimize the variance of the parameter estimates or predicted value. However, these methods see only global euclidean structure, while the local manifold structure is ignored. For example, I-optimal design selects those data points such that other data points can be best approximated by linear combinations of all the selected points. In this paper, we propose a novel active learning algorithm which takes into account the local structure of the data space. That is, each data point should be approximated by the linear combination of only its neighbors. Given the local reconstruction coefficients for every data point and the coordinates of the selected points, a transductive learning algorithm called Locally Linear Reconstruction (LLR) is proposed to reconstruct every other point. The most representative points are thus defined as those whose coordinates can be used to best reconstruct the whole data set. The sequential and convex optimization schemes are also introduced to solve the optimization problem. The experimental results have demonstrated the effectiveness of our proposed method. [ABSTRACT FROM PUBLISHER]

Matrix factorization techniques have been frequently applied in information retrieval, computer vision, and pattern recognition. Among them, Nonnegative Matrix Factorization (NMF) has received considerable attention due to its psychological and physiological interpretation of naturally occurring data whose representation may be parts based in the human brain. On the other hand, from the geometric perspective, the data is usually sampled from a low-dimensional manifold embedded in a high-dimensional ambient space. One then hopes to find a compact representation,which uncovers the hidden semantics and simultaneously respects the intrinsic geometric structure. In this paper, we propose a novel algorithm, called Graph Regularized Nonnegative Matrix Factorization (GNMF), for this purpose. In GNMF, an affinity graph is constructed to encode the geometrical information and we seek a matrix factorization, which respects the graph structure. Our empirical study shows encouraging results of the proposed algorithm in comparison to the state-of-the-art algorithms on real-world problems. [ABSTRACT FROM AUTHOR]