Two-pathway attention network for real-time facial expression recognition

Many scholars are committed to using deep learning methods to study facial expression recognition (FER). In recent years, FER has gradually been confined to psychology research in the early days to now involves knowledge of many disciplines such as physiology, psychology, cognition and medicine. With the extreme achievement of computer version techniques, various convolutional neural network structures were developed for real-time and accurate FER. There are two main problems in the existing convolutional neural network for handling FER problems: insufficient training data caused over-fitting and expression-unrelated intra-class differences. In this paper, we propose a two-pathway attention network to solve these two problems better. We suppress the intra-class differences efficiently by extracting facial regions based on facial muscle movements driven by facial expressions. We prevent deep networks from insufficient training data by extensively extracting global structures and local facial regions as the training dataset to feed a two-pathway ensemble model. Further more, we weight the whole feature maps from the global image and local regions by introducing an attention mechanism module to reweighs each part according to its contribution to FER. We adopt real-time facial region extraction and multi-layer feature data compression to ensure the real-time performance of the algorithm and reduce the amount of parameters in ensemble model. Experiments on public datasets suggest that our method certifies its effectiveness, reaches human-level performance, and outperforms current state-of-the-art methods with 92.8% on CK+ and 87.0% on FERPLUS.