How to Evaluate Serious Games Concepts? A Systematic Prototyping and Testing Approach.

The challenge in developing a serious game is to find the perfect balance between learning and playing. The development process should include an appropriate involvement of the target group and enable a systematic evaluation of this balance through prototyping and testing. The goal is to create an entertaining and purposeful learning experience and thus enable knowledge growth. This paper presents the evaluation results of the serious game E.F.A. with the target group - managers in the social service sector. The first prototype was tested in an early phase as a paper prototype by experts in media didactics and subject experts. Early stage testing is a decisive factor for the development of serious games. However, the accessibility of the target group is not always given for fast testing and iterative improvement. After collecting expert feedback and incorporating it into the game, the high-fidelity prototype was created and tested by the target group. Those test runs were followed by group interviews. Their results are the focus of this paper which aims at answering the following research questions: How did the target group experience the serious game and their increase in knowledge? To what extent can the evaluation results with the target group be linked to the early tests with the paper prototype? How does the feedback vary and what conclusions can be drawn from this? The results of the paper show that the serious game was rated very differently among the target group. Some generally praised the playful approach. Others criticized the game as childish and unsuitable for the target group. The feedback obtained from different user groups with the help of different prototypes varied for a set of evaluation criteria such as playing time, remembered knowledge and dialogs. For each evaluation criteria recommendations are given regarding the test group and type of prototype. [ABSTRACT FROM AUTHOR]