Search

This thesis is to develop new facial expression recognition techniques based on 2D/3D images or videos, with the purpose to improve the recognition efficiency and accuracy of the current state-of-art. A fully automatic facial expression recognition system is designed, including real-time landmark detection, spatio-temporal feature extraction, hierarchical classification, and most discriminant facial regions identification for expression recognition. In general, the proposed system improved the facial expression recognition state-of-art.

Facial expressions form a significant part of our nonverbal communications and understanding them is essential for effective human computer interaction. Due to the diversity of facial geometry and expressions, automatic expression recognition is a challenging task. This paper deals with the problem of person-independent facial expression recognition from a single 3D scan. We consider only the 3D shape because facial expressions are mostly encoded in facial geometry deformations rather than textures. Unlike the majority of existing works, our method is fully automatic including the detection of landmarks. We detect the four eye corners and nose tip in real time on the depth image and its gradients using Haar-like features and AdaBoost classifier. From these five points, another 25 heuristic points are defined to extract local depth features for representing facial expressions. The depth features are projected to a lower dimensional linear subspace where feature selection is performed by maximizing their relevance and minimizing their redundancy. The selected features are then used to train a multi-class SVM for the final classification. Experiments on the benchmark BU-3DFE database show that the proposed method outperforms existing automatic techniques, and is comparable even to the approaches using manual landmarks.

Automatically recognizing facial expressions presents an active and challenging problem in computer vision and pattern classification. The person-independent case is even more challenging. In this paper, we propose a hierarchical approach to achieve person-independent facial expression recognition. Specifically, the expressions that are easily confused together are merged into one class and join the remaining prototypic expressions in the first tier classification; the expressions in the merged class are then separated in the second tier. Support Vector Machine is adopted as the classifier in both tiers, with the LBP and displacement features in the first tier as well as mouth and eyebrows features in the second tier. The proposed method is tested on the Cohn-Kanade Extended (CK+) dataset and evaluated in terms of a confusion matrix. The person-independent experiments demonstrate the effectiveness of the proposed hierarchical classifier in improving recognition accuracy and eliminating confusions.