Multi-view laplacian eigenmaps based on bag-of-neighbors for RGB-D human emotion recognition.

Human emotion recognition is an important direction in the fields of human-computer interaction and computer vision. However, most existing human emotion researches just focus on one view of the study objects. In this paper, we first introduce a RGB-D video-emotion dataset and a RGB-D face-emotion dataset for research, both of which are collected under psychological principles and methods. Then we propose a new supervised nonlinear multi-view laplacian eigenmaps (MvLE) approach and a multi-hidden-layer out-of-sample network (MHON) that can make full use of RGB view and Depth view of the two datasets. MvLE is employed to map the samples of both views from original spaces into a common subspace. As samples of RGB view and Depth view lie on different spaces, a new distance metric bag of neighbors (BON) introduced in MvLE can capture their similar distributions. Moreover, to adapt to large-scale applications, MHON is developed to get the low-dimensional representations of additional samples and predict their labels. MvLE and MHON can deal with the cases that RGB view and Depth view have different dimensions of original spaces, even different number of samples or categories. The experiment results indicate that the proposed methods achieve considerable improvement over some state-of-art methods. [ABSTRACT FROM AUTHOR]