Search

In this paper, we develop a novel method for view-based recognition of human action/activity from videos. By observing just a few frames, we can identify the activity that takes place in a video sequence. The basic idea of our method is that activities can be positively identified from a sparsely sampled sequence of a few body poses acquired from videos. In our approach, an activity is represented by a set of pose and velocity vectors for the major body parts (hands, legs, and torso) and stored in a set of multidimensional hash tables. We develop a theoretical foundation that shows that robust recognition of a sequence of body pose vectors can be achieved by a method of indexing and sequencing and it requires only a few pose vectors (i.e., sampled body poses in video frames). We find that the probability of false alarm drops exponentially with the increased number of sampled body poses. So, matching only a few body poses guarantees high probability for correct recognition. Our approach is parallel, i.e., all possible model activities are examined at one indexing operation. In addition, our method is robust to partial occlusion since each body part is indexed separately. We use a sequence-based voting approach to recognize the activity invariant to the activity speed.

We present a supervised learning approach for object-category specific restoration, recognition, and segmentation of images which are blurred using an unknown kernel. The novelty of this work is a multilayer graphical model which unifies the low-level vision task of restoration and the high-level vision task of recognition in a cooperative framework. The graphical model is an interconnected two-layer Markov random field. The restoration layer accounts for the compatibility between sharp and blurred images and models the association between adjacent patches in the sharp image. The recognition layer encodes the entity class and its location in the underlying scene. The potentials are represented using nonparametric kernel densities and are learnt from training data. Inference is performed using nonparametric belief propagation. Experiments demonstrate the effectiveness of our model for the restoration and recognition of blurred license plates as well as face images.

In this paper we present a supervised learning approach for object-category specific restoration, recognition and segmentation of images which are blurred using an unknown kernel. The feature of this work is a multi layer graphical model which unifies the low level vision task of restoration, and the high level vision task of recognition in a cooperative framework. Proposed graphical model is an interconnected two layer Markov Random Field. The restoration layer accounts for the compatibility between sharp and blurred patches, and models the association between adjacent patches in the sharp image. The recognition layer encodes the patch location and class. The potentials are represented using non-parametric kernel densities and are learnt from the training data. Inference is performed using nonparametric belief propagation. We propose a similar model for super-resolution from multiple frames, and suggest the use of ordinal regression for sub-pixel shift estimation to address the registration issues. Experiments demonstrate the effectiveness of proposed models for the restoration and recognition of blurred license plate and face images.

Incrementally learning from a large number of unlabeled examples continues to be an active area of research in pattern recognition. Active Learning has made great strides in recent years to address this problem, taking advantage of SVMs to develop robust learning systems. Recently, diversity sampling for SVM active learning has garnered much attention. In this work we propose a fundamentally motivated view of diversity for SVM active learning based on an information-theoretic diversity measure. Comparative testing on a database from the small-sample learning problem of image retrieval is done and thoughts for future work are presented.

In this paper, we present a supervised learning based approach for sub-pixel motion estimation. The novelty of this work is the learning based method itself which tries to learn the shifts from a large training database. Integer pixel shift is sub-divided and discretized to levels in both the horizontal and vertical direction. We pose the problem of motion estimation in a polar coordinate system. Shift estimation in the x and y direction has been posed as a problem of estimating r and thetas. The ordinal property of r has been used, and consequently, we employ a ranking based approach for estimating r. For thetas estimation we employ multi-class classification techniques. We demonstrate how very simplistic features can be used to differentiate between different sub-pixel shifts

A novel method for view-based recognition of human activity is presented. The basic idea of our method is that activities can be positively identified from a sparsely sampled sequence of few body poses acquired from videos. In our approach, an activity is represented by a set of pose and velocity vectors for the major body parts (hands, legs and torso) and stored in a set of multidimensional hash tables. We show that robust recognition of a sequence of body pose vectors can be achieved by a method of indexing and sequencing and it requires only few vectors (i.e. sampled body poses in video frames). We find that the probability of false alarm drops exponentially with the increased number of sampled body poses. We also achieve speed invariant recognition by eliminating the time factor and replacing it with sequence information. Experiments performed with videos having 8 different activities show robust recognition even for different viewing directions.

Small-sample learning in image retrieval is a pertinent and interesting problem. Relevance feedback is an active area of research that seeks to find algorithms that are robust with only a small number of examples. Much work has been done in both the machine learning and pattern recognition communities to develop algorithms that learn a high-level semantic concept in a low-level image feature space. In this paper we seek to leverage techniques from both these communities to explore a hybrid relevance feedback system which combines the insight gained from discriminant analysis and active learning. Our technique uses a diversity-based pool-query technique along with biased discriminant analysis to improve the query refinement process. Comparative results are observed and thoughts for future work are presented.

In this paper we present a novel learning based method for restoring and recognizing images of digits that have been blurred using an unknown kernel. The novelty of our work is an iterative loop that alternates between recognition and restoration stages. In the restoration stage we model the image as an undirected graphical model over the image patches with the compatibility functions represented as non-parametric kernel densities. Compatibility functions are initially learned using uniform random samples from the training data. We solve the inference problem by an extended version of the non-parametric belief propagation algorithm in which we introduce the notion of partial messages. We close the loop by using the confidence scores of the recognition to non-uniformly sample from the training set in order to retrain the compatibility functions. We show experimental results on synthetic and license plate images.

A novel method for human body parts labeling, given a set of body parts that are represented by bars in the image is developed. First, we assume that in addition to the head and torso, the human body has 8 major body parts namely, the upper and lower arms (left and right) and the upper and lower legs (left and right) and all these parts can be approximated by cylinders that appear as rectangular bars in the image. One of the original contributions of this paper is the novel use of the spatial distributions of the location and orientation of the bars employed to find a correct role assignment for each bar by maximizing a joint probability function. Thus, we formulate the problem of labeling human body parts as an optimal hypothesis selection problem where each hypothesis corresponds to a particular role assignment. Next, we develop an algorithm which identifies the optimal hypothesis. We test our method with 80 images that depict human poses and obtain correct labeling in 98.7% of the trials. We also test our method in cases of bar detection errors, occlusion and noisy images and obtain quite good results.

We present a novel learning-based framework for zooming and recognizing images of digits obtained from vehicle registration plates, which have been blurred using an unknown kernel. We model the image as an undirected graphical model over image patches in which the compatibility functions are represented as nonparametric kernel densities. The crucial feature of this work is an iterative loop that alternates between super-resolution and restoration stages. A machine-learning-based framework has been used for restoration which also models spatial zooming. Image segmentation is done by a column-variance estimation-based "dissection" algorithm. Initially, the compatibility functions are learned by nonparametric kernel density estimation, using random samples from the training data. Next, we solve the inference problem by using an extended version of the nonparametric belief propagation algorithm, in which we introduce the notion of partial messages. Finally, we recognize the super-resolved and restored images. The resulting confidence scores are used to sample from the training set to better learn the compatibility functions.