Performance Effects of Structural and Behavioural Debriefings in Transparent Simulations

Simulation-based learning environments are used extensively to support learning in complex business systems. Nevertheless, studies show that subjects have great difficulty in understanding and managing dynamic systems, mainly due to cognitive processing difficulties. Previous research has addressed some aspects of model transparency and instructional strategy and produced inconclusive results about their impact on learning effectiveness. This study explores the learning and performance effects of using transparent simulations (that is, showing users the internal structure of models) and applying structural and behavioural debriefings focussed respectively on the critical variables and relations and on the relation between model structure, patterns of actions, and system behaviour. We present a set of hypotheses about the influence of the model transparency and debriefing conditions on participants' performance and comprehension of the system dynamics. In a simulation experiment, participants interacted with a system dynamics model, representing the growth of a business venture. Participants provided with the more transparent strategy and exposed to structure and behavioural debriefings achieved higher performance and demonstrated better comprehension of the model dynamics. However, our findings indicate that while these conditions are beneficial for understanding the business system dynamics, making only the simulator transparent, even combined with structural debriefing, is more limited in its effect. Three out of the six hypotheses were confirmed. Structural debriefing (in conjunction with model transparency) positively influences performance, behavioural debriefing (combined with model transparency) positively influences both model dynamics comprehension and performance.