Face Reenactment Based on Unsupervised Motion Transfer and Video Correction.

Face reenactment aims to transfer the upper body motions from a driving actor to a target actor. Current methods either cannot transfer motion adequately or cannot synthesize high-quality video. This paper proposes a novel face reenactment method via unsupervised motion transfer and deep learning-based correction. Firstly, the motion of the driving actor is largely transferred to the target via an unsupervised motion model and a rough synthetic target video can be obtained. Then, a generative neural network with spatial-temporal structure is designed to correct the rough video to a realistic and smooth video. To synthesize smooth and detailed video, 3D convolution and attention mechanism are introduced into the network to process temporal information and guide the video correction. To avoid synthesizing background with artifacts, the background information is embedded into the network as fixed parameters. To improve the realism of the teeth, a mouth enhancement loss is designed. The network is trained in an adversarial manner, ensuring the realism of the generated images. Experiments show that this method can synthesize high-quality target videos and the performance is better than the current state-of-the-art face reenactment methods. [ABSTRACT FROM AUTHOR]