Objective Assessment of Pedagogical Effectiveness and the Human Rights Foreign Policy Simulation Game

Literature on simulation games stresses both the need for objective assessment of pedagogical effectiveness and the concern that such assessment may not be happening because it is too difficult. This article speaks to both points by presenting an approach to objectively assess the effectiveness of a simulation called the Human Rights Foreign Policy Game (HRFPG). After describing the HRFPG and its learning objectives, the article explains this approach to assessment. This approach applies and extends upon insights from earlier approaches, especially Shaw, by using a pretest/posttest design and open-ended questions. By performing a thematic analysis of students' posttest descriptions of what they learned, we identify incidents of learning related to the game's learning objectives and then increase our confidence in this assessment by conducting a student-specific turnover analysis to eliminate any claimed learning that had already been expressed on the pretest. We then demonstrate how the resulting data can be used in basic quantitative analyses to ask further interesting questions. Three main conclusions are reached. First, the HRFPG generates objectively assessable learning. Second, the HRFPG does not systematically disadvantage students according to gender or attitudes toward competitive games or group work. Finally, the assessment methodology, while time-consuming, is practicable.