Evaluating the effectiveness of a multivariate autoregressive model in predicting the time course of dynamic facial behaviours

Faces convey critical information about people, such as cues to their identity and emotional state. In the real world, facial behaviours evolve dynamically and encapsulate a range of biological motion signals. Furthermore, behavioural and neuroimaging studies have demonstrated that human observers are sensitive to this temporal information. The presence of systematic temporal changes in the face implies the possibility of predicting the evolution of dynamic facial behaviours. We video recorded subjects delivering positive or negative phrases, and used a PCA-based active appearance model to capture critical dimensions of facial variation over time. We applied multivariate autoregressive models to predict PCA scores of future frames from the frames immediately preceding them, up to a lag of 200ms prior to the target frame. These models did successfully predict future frames, but they did not benefit from extending the temporal support, suggesting they relied primarily on image similarity between consecutive frames. We next used hidden Markov models to segment videos into shorter sequences comprising more consistent facial behaviours. The Markov models successfully extracted distinct facial basis states, however segmenting the data by state did not yield any predictive benefit to autoregressive models fit within those states. We conclude that autoregressive models have only limited predictive power in the context of facial expression analysis.