Automatically Predicting Peer Satisfaction during Collaborative Learning with Linguistic, Acoustic, and Visual Features

Collaborative learning has numerous benefits such as enhancing learners' critical thinking, developing social skills, and improving learning gains. While engaging in this interactive process, learners' satisfaction toward their partners plays a crucial role in defining the success of the collaboration. However, detecting learners' satisfaction during an ongoing collaboration remains challenging, and there are no automatic techniques to predict learners' satisfaction. In this paper, we propose a multimodal approach to automatically predict peer satisfaction for co-located collaboration with features extracted from 44 middle school learners' collaborative dialogues. We investigated three types of features extracted from learners' dialogues: 1) linguistic features indicating semantics and sentiment; 2) acoustic-prosodic features including energy and pitch; and 3) visual features including eye gaze, head pose, facial action units, and body pose. We then trained several regression models with each of those features to predict the peer satisfaction scores that learners received from their partners. The results revealed that head position and body location were significant indicators of peer satisfaction: lower head and body distances between partners were associated with more positive peer satisfaction. Next, we investigated the influence of multimodal feature fusion methods on peer satisfaction prediction accuracy: early fusion versus late fusion. We report the comparison results between models trained with (1) best-performing unimodal features, (2) multimodal features combined by early fusion, and (3) multimodal features combined by late fusion. This line of research reveals how multimodal features from collaborative dialogues are associated with peer satisfaction, and represents a step toward the development of real-time intelligent systems that support collaborative learning.