Search

Since each visual feature only reflects a unique characteristic about a 3-dimensional (3D) model and different visual features have diverse discriminative power in model representation, it would be beneficial to fuse multiple visual features in 3D model retrieval. To this end, we propose a fast view-based 3D model retrieval framework in this article. This framework comprises two parts: the first one is an Unsupervised Multiple Feature Fusion algorithm (UMFF), which is used to learn a compact yet discriminative feature representation from the original multiple visual features; and the second one is an efficient Online Projection Learning algorithm (OPL), which is designed to fast transfer the input multiple visual features of a newcome model into its corresponding low-dimensional feature representation. In this framework, many existing ranking algorithms such as the simple distance-based ranking method can be directly adopted for sorting all 3D models in the database using the learned new feature representation and returning the top ranked models to the user. Extensive experiments on two public 3D model databases demonstrate the efficiency and the effectiveness of the proposed approach over its competitors. The proposed framework cannot only dramatically improve the retrieval performance but also reduce the computational cost in dealing with the newcome models. HighlightsAn Unsupervised Multiple Feature Fusion (UMFF) algorithm is proposed to fuse multiple features for 3D model representation.The ? 1 -graph method is used to learn roust and datum-adaptive graph to preserve local geometric structure information.An efficient Online Projection Learning (OPL) algorithm is designed to solve the out-of-sample problem.

Owing to the well interpretation ability, Non-Negative Matrix Factorization (NMF) has attracted much attention from computer vision and machine learning communities. However, the standard NMF adopts a least square error function as the empirical likelihood term in the model, which is sensitive to the noise and outliers. So, it is not robust in practice. To overcome this problem, we propose a noise robust NMF method named as Locally Weighted Sparse Graph regularized Non-negative Matrix Factorization (LWSG_NMF). Since many real-world noises can be broadly decomposed into the dense Gaussian random noise and the sparse block noise, we propose a sparse noise assumption. Based on this assumption, we reformulate the empirical likelihood term of the standard NMF by explicitly imposing a sparse noise term. Meanwhile, a locally weighted sparse graph regularization term is also incorporated in our model to exploit the local geometric structure information of data. Different from the other existing graph-based methods, we take the effect of noise into account in learning our graph regularization term. An iterative optimization method is also proposed to solve the objective function of LWSG_NMF. Extensive experiments on three public benchmark datasets demonstrate the robustness and the effectiveness of our proposed method for human face recognition and handwritten digital recognition in the presence of noise.

Multiple visual features have been proposed and used in 3-dimensional (3D) model retrieval in recent years. Since each visual feature reflects a unique characteristic about the model, they have unequal discriminative power with respect to a specific category of 3D model, and they are complementary to each other in model representation. Thus, it would be beneficial to combine multiple visual features together in 3D model retrieval. In light of this, we propose an efficient Semi-supervised Multiple Feature Fusion (SMFF) method for view-based 3D model retrieval in this paper. Specifically, We first extract multiple visual features to describe both the local and global appearance characteristics of multiple 2D projected images that are generated from 3D models. Then, SMFF is adopted to learn a more compact and discriminative low-dimensional feature representation via multiple feature fusion using both the labeled and unlabeled 3D models. Once the low-dimensional features have been learned, many existing methods such as SVM and KNN can be used in the subsequent retrieval phase. Moreover, an out-of-sample extension of SMFF is provided to calculate the low-dimensional features for the newly added 3D models in linear time. Experiments on two public 3D model datasets demonstrate that using such a learned feature representation can significantly improve the performance of 3D model retrieval and the proposed method outperforms the other competitors.

Using motion capture to create naturally looking motion sequences for virtual character animation has become a standard procedure in the games and visual effects industry. With the fast growth of motion data, the task of automatically annotating new motions is gaining an importance. In this paper, we present a novel statistic feature to represent each motion according to the pre-labeled categories of key-poses. A probabilistic model is trained with semi-supervised learning of the Gaussian mixture model (GMM). Each pose in a given motion could then be described by a feature vector of a series of probabilities by GMM. A motion feature descriptor is proposed based on the statistics of all pose features. The experimental results and comparison with existing work show that our method performs more accurately and efficiently in motion retrieval and annotation.

In computer vision and multimedia analysis, it is common to use multiple features (or multimodal features) to represent an object. For example, to well characterize a natural scene image, we typically extract a set of visual features to represent its color, texture, and shape. However, it is challenging to integrate multimodal features optimally. Since they are usually high-order correlated, e.g., the histogram of gradient (HOG), bag of scale invariant feature transform descriptors, and wavelets are closely related because they collaboratively reflect the image texture. Nevertheless, the existing algorithms fail to capture the high-order correlation among multimodal features. To solve this problem, we present a new multimodal feature integration framework. Particularly, we first define a new measure to capture the high-order correlation among the multimodal features, which can be deemed as a direct extension of the previous binary correlation. Therefore, we construct a feature correlation hypergraph (FCH) to model the high-order relations among multimodal features. Finally, a clustering algorithm is performed on FCH to group the original multimodal features into a set of partitions. Moreover, a multiclass boosting strategy is developed to obtain a strong classifier by combining the weak classifiers learned from each partition. The experimental results on seven popular datasets show the effectiveness of our approach.

With the explosive growth of motion capture data, it becomes very imperative in animation production to have an efficient search engine to retrieve motions from large motion repository. However, because of the high dimension of data space and complexity of matching methods, most of the existing approaches cannot return the result in real time. This paper proposes a high level semantic feature in a low dimensional space to represent the essential characteristic of different motion classes. On the basis of the statistic training of Gauss Mixture Model, this feature can effectively achieve motion matching on both global clip level and local frame level. Experiment results show that our approach can retrieve similar motions with rankings from large motion database in real-time and also can make motion annotation automatically on the fly. Copyright © 2013 John Wiley & Sons, Ltd.

Motion capture (MOCAP) is an important technique that is widely used in many areas such as computer animation, film industry, physical training and so on. Even with professional MOCAP system, the missing marker problems always occur. Motion refinement is an essential preprocessing step for MOCAP data based applications. Although many existing approaches for motion refinement have been developed, it is still a challenging task due to the complexity and diversity of human motion. A data driven based motion refinement method is proposed in this paper, which modifies the traditional sparse coding process for special task of motion recovery from missing parts. Meanwhile, the objective function is derived by taking both statistical and kinematical property of motion data into account. Poselet model and moving window grouping are applied in the proposed method to achieve a fine-grained feature representation, which preserves the embedded spatial-temporal kinematic information. 5 motion dictionaries are learnt for each kind of poselet from training data in parallel. The motion refine problem is finally solved as an e1-minimization problem. Compared with several state-of-art motion refine methods, the experimental result shows that our approach outperforms the competitors.

Human motion retrieval plays an important role in many motion data based applications. In the past, many researchers tended to use a single type of visual feature as data representation. Because different visual feature describes different aspects about motion data, and they have dissimilar discriminative power with respect to one particular class of human motion, it led to poor retrieval performance. Thus, it would be beneficial to combine multiple visual features together for motion data representation. In this article, we present an Adaptive Multi-view Feature Selection (AMFS) method for human motion retrieval. Specifically, we first use a local linear regression model to automatically learn multiple view-based Laplacian graphs for preserving the local geometric structure of motion data. Then, these graphs are combined together with a non-negative view-weight vector to exploit the complementary information between different features. Finally, in order to discard the redundant and irrelevant feature components from the original high-dimensional feature representation, we formulate the objective function of AMFS as a general trace ratio optimization problem, and design an effective algorithm to solve the corresponding optimization problem. Extensive experiments on two public human motion database, i.e., HDM05 and MSR Action3D, demonstrate the effectiveness of the proposed AMFS over the state-of-art methods for motion data retrieval. The scalability with large motion dataset, and insensitivity with the algorithm parameters, make our method can be widely used in real-world applications. Display Omitted An Adaptive Multi-view Feature Selection (AMFS) algorithm is proposed to fuse multiple features formotion data retrieval.The local regression model isused to learn a datum-adaptive graph for each feature to preserve local structure information.The selection matrix is learnt from all local graphs by exploiting the complementary information between different features.An efficient iterative optimization approach is designed to solve objective function represented in trace ratio form.

Missing marker problem is very common in human motion capture. In contrast to most current methods which handle this problem based on trying to learn a reliable predictor from the observations, we consider it from the perspective of sparse representation and propose a novel method which is named l1-sparse representation of missing markers prediction (L1-SRMMP). We assume that the incomplete pose can be represented by a linear combination of a few poses from the training set and the representation is sparse. Therefore, we cast the predicting missing markers as finding a sparse representation of the observable data of the incomplete pose, and then we use it to predict the missing data. In order to get a sparse representation, we employ l1-norm in our objective function. Moreover, we propose presentation coefficient weighted update (PCWU) algorithm to mitigate the limited capacity problem of the training set. Experimental results demonstrate the effectiveness and efficiency of our method to predict the missing markers in human motion capture. Copyright © 2011 John Wiley & Sons, Ltd.

Human motion denoising is an indispensable step of data preprocessing for many motion data based applications. In this paper, we propose a data-driven based human motion denoising method that sparsely selects the most correlated subset of motion bases for clean motion reconstruction. Meanwhile, it takes the statistic property of two common noises, i.e., Gaussian noise and outliers, into account in deriving the objective functions. In particular, our method firstly divides each human pose into five partitions termed as poselets to gain a much fine-grained pose representation. Then, these poselets are reorganized into multiple overlapped poselet groups using a lagged window moving across the entire motion sequence to preserve the embedded spatial-temporal motion patterns. Afterward, five compacted and representative motion dictionaries are constructed in parallel by means of fast K-SVD in the training phase; they are used to remove the noise and outliers from noisy motion sequences in the testing phase by solving ?1-minimization problems. Extensive experiments show that our method outperforms its competitors. More importantly, compared with other data-driven based method, our method does not need to specifically choose the training data, it can be more easily applied to real-world applications. HighlightsA fine-grained pose representation model is proposed to boost the performance.We present a data-driven based motion denoising method by solving ?1-minimization problems.The proposed model selects the most correlated motion bases for motion reconstruction.Our method does not need to choose the training data and can be implemented in parallel mode.

Motion capture is an important technique with a wide range of applications in areas such as computer vision, computer animation, film production, and medical rehabilitation. Even with the professional motion capture systems, the acquired raw data mostly contain inevitable noises and outliers. To denoise the data, numerous methods have been developed, while this problem still remains a challenge due to the high complexity of human motion and the diversity of real-life situations. In this paper, we propose a data-driven-based robust human motion denoising approach by mining the spatial-temporal patterns and the structural sparsity embedded in motion data. We first replace the regularly used entire pose model with a much fine-grained partlet model as feature representation to exploit the abundant local body part posture and movement similarities. Then, a robust dictionary learning algorithm is proposed to learn multiple compact and representative motion dictionaries from the training data in parallel. Finally, we reformulate the human motion denoising problem as a robust structured sparse coding problem in which both the noise distribution information and the temporal smoothness property of human motion have been jointly taken into account. Compared with several state-of-the-art motion denoising methods on both the synthetic and real noisy motion data, our method consistently yields better performance than its counterparts. The outputs of our approach are much more stable than that of the others. In addition, it is much easier to setup the training dataset of our method than that of the other data-driven-based methods.

Even though there is an explosive growth of motion capture data, there is still a lack of efficient and reliable methods to automatically annotate all the motions in a database. Moreover, because of the popularity of mocap devices in home entertainment systems, real-time human motion annotation or recognition becomes more and more imperative. This paper presents a new motion annotation method that achieves both the aforementioned two targets at the same time. It uses a probabilistic pose feature based on the Gaussian Mixture Model to represent each pose. After training a clustered pose feature model, a motion clip could be represented as an action string. Then, a dynamic programming-based string matching method is introduced to compare the differences between action strings. Finally, in order to achieve the real-time target, we construct a hierarchical action string structure to quickly label each given action string. The experimental results demonstrate the efficacy and efficiency of our method. Copyright © 2014 John Wiley & Sons, Ltd.

In this paper, we present an integrated framework of human motion retrieval based on freehand sketch. With some simple rules, the user can acquire a desired motion by sketching several key postures. To retrieve efficiently and accurately by sketch, the 3D postures are projected onto several 2D planes. The limb direction feature is proposed to represent the input sketch and the projected-postures. Furthermore, a novel index structure based on k-d tree is constructed to index the motions in the database, which speeds up the retrieval process. With our posture-by-posture retrieval algorithm, a continuous motion can be got directly or generated by using a pre-computed graph structure. What's more, our system provides an intuitive user interface. The experimental results demonstrate the effectiveness of our method. Copyright © 2014 John Wiley & Sons, Ltd.

To reveal and leverage the correlated and complemental information between different views, a great amount of multi-view learning algorithms have been proposed in recent years. However, unsupervised feature selection in multi-view learning is still a challenge due to lack of data labels that could be utilized to select the discriminative features. Moreover, most of the traditional feature selection methods are developed for the single-view data, and are not directly applicable to the multi-view data. Therefore, we propose an unsupervised learning method called Adaptive Unsupervised Multi-view Feature Selection (AUMFS) in this paper. AUMFS attempts to jointly utilize three kinds of vital information, i.e., data cluster structure, data similarity and the correlations between different views, contained in the original data together for feature selection. To achieve this goal, a robust sparse regression model with the l2,1-norm penalty is introduced to predict data cluster labels, and at the same time, multiple view-dependent visual similar graphs are constructed to flexibly model the visual similarity in each view. Then, AUMFS integrates data cluster labels prediction and adaptive multi-view visual similar graph learning into a unified framework. To solve the objective function of AUMFS, a simple yet efficient iterative method is proposed. We apply AUMFS to three visual concept recognition applications (i.e., social image concept recognition, object recognition and video-based human action recognition) on four benchmark datasets. Experimental results show the proposed method significantly outperforms several state-of-the-art feature selection methods. More importantly, our method is not very sensitive to the parameters and the optimization method converges very fast.

In this paper, we propose a novel active learning algorithm, called Locally Regressive Optimal Design (LROD), to improve the effectiveness of relevance feedback-based social image retrieval. Our algorithm assumes that for each data point, the label values of both this data point and its neighbors can be well estimated using a locally regressive function. Specifically, we adopt a local linear regression model to predict the label value of each data point in a local patch. The regularized local model predication error of the local patch is defined as our local loss function. Then, a unified objective function is proposed to minimize the summation of these local loss functions over all the data points, so that an optimal predicated label value can be assigned to each data point. Finally, we embed it into a semi-supervised learning framework to construct the final objective function. Experiment results on MSRA-MM2.0 database demonstrate the efficiency and effectiveness of the proposed algorithm for relevance feedback-based social image retrieval. © 2012 Elsevier B.V..

Sketch-based human motion retrieval is a hot topic in computer animation in recent years. In this paper, we present a novel sketch-based human motion retrieval method via selected 2-dimensional (2D) Geometric Posture Descriptor (2GPD). Specially, we firstly propose a rich 2D pose feature call 2D Geometric Posture Descriptor (2GPD), which is effective in encoding the 2D posture similarity by exploiting the geometric relationships among different human body parts. Since the original 2GPD is of high dimension and redundant, a semi-supervised feature selection algorithm derived from Laplacian Score is then adopted to select the most discriminative feature component of 2GPD as feature representation, and we call it as selected 2GPD. Finally, a posture-by-posture motion retrieval algorithm is used to retrieve a motion sequence by sketching several key postures. Experimental results on CMU human motion database demonstrate the effectiveness of our proposed approach.