Search

Facial expression is related to facial muscle contractions and different muscle movements correspond to different emotional states. For micro-expression recognition, the muscle movements are usually subtle, which has a negative impact on the performance of current facial emotion recognition algorithms. Most existing methods use self-attention mechanisms to capture relationships between tokens in a sequence, but they do not take into account the inherent spatial relationships between facial landmarks. This can result in sub-optimal performance on micro-expression recognition tasks. Therefore, learning to recognize facial muscle movements is a key challenge in the area of micro-expression recognition. In this paper, we propose a Hierarchical Transformer Network (HTNet) to identify critical areas of facial muscle movement. HTNet includes two major components: a transformer layer that leverages the local temporal features and an aggregation layer that extracts local and global semantical facial features. Specifically, HTNet divides the face into four different facial areas: left lip area, left eye area, right eye area and right lip area. The transformer layer is used to focus on representing local minor muscle movement with local self-attention in each area. The aggregation layer is used to learn the interactions between eye areas and lip areas. The experiments on four publicly available micro-expression datasets show that the proposed approach outperforms previous methods by a large margin. The codes and models are available at: https://github.com/wangzhifengharrison/HTNet. [ABSTRACT FROM AUTHOR]

Automatic hand-drawn sketch recognition is an important task in computer vision. However, the vast majority of prior works focus on exploring the power of deep learning to achieve better accuracy on complete and clean sketch images, and thus fail to achieve satisfactory performance when applied to incomplete or destroyed sketch images. To address this problem, we first develop two datasets that contain different levels of scrawl and incomplete sketches. Then, we propose an angular-driven feedback restoration network (ADFRNet), which first detects the imperfect parts of a sketch and then refines them into high quality images, to boost the performance of sketch recognition. By introducing a novel “feedback restoration loop” to deliver information between the middle stages, the proposed model can improve the quality of generated sketch images while avoiding the extra memory cost associated with popular cascading generation schemes. In addition, we also employ a novel angular-based loss function to guide the refinement of sketch images and learn a powerful discriminator in the angular space. Extensive experiments conducted on the proposed imperfect sketch datasets demonstrate that the proposed model is able to efficiently improve the quality of sketch images and achieve superior performance over the current state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Facial expression recognition has a wide range of applications in the real world. Although many existing deep learning methods have achieved remarkable success, weak expression recognition remains a challenging task because of the significant domain gap between a weak expression and its peak expression counterpart. One idea to solve this problem is to find an effective way to bridge the gap between the two domains by either transfer learning or cross-domain image synthesis. In this paper, we propose a Group Generative Adversarial Network (GroupGAN) that recognizes weak facial expression by magnifying the expressions to stronger or peak ones. Different from the traditional GAN which typically has only one generator and one discriminator, the proposed GroupGAN has one generator, one extractor and two discriminators. Similar to the "two-player game" analogy of the traditional GAN, in our setting the generator along with feature extractor act as one group to compete with the other group of the two distinct discriminators. Extensive experiments show that the proposed GroupGAN significantly improves the performance of weak expression recognition, and is able to magnify weak expressions, thus facilitating many expression-related vision tasks like sketch recognition. [ABSTRACT FROM AUTHOR]