Paper-based vs. Digital Prototyping: How to Evaluate Serious Game Concepts at Different Stages of Development.

During the development process of a serious game elements such as storyline, feedback and learning tasks need to be designed in order to create an entertaining but also goal-oriented learning experience. Thus, a decisive challenge is to master the balance between learning and playing. Prototyping and testing can support a systematic evaluation of this balance. This paper describes how the methods of paper and digital prototyping were applied in the project E.F.A. which aims at developing a serious game on occupational health and safety. Paper prototyping was used at an early stage of the development process. Work on the digital prototype only began after the testing and refinement of the paper prototype was completed. This paper aims at answering the following research questions: How do evaluation results of a serious game concept differ between testing with a paper-based and a digital prototype? For which evaluation criteria is either a paper prototype or a digital prototype an appropriate medium to efficiently test and refine the concept of a serious game? To collect data on these aspects, eight test runs with the paper-based and seven with the digital prototype were conducted. Testers were asked to think aloud while playing and were interviewed before and after testing. A comparison of paper-based and digital testing shows that a paper prototype is rather useful for evaluating baseline elements such as the complexity of learning tasks. In the specific case of E.F.A. the implementation of learning contents was criticized and participants perceived only a small grow in knowledge. For testing aspects of usability digital testing, though, turns out to be a better suited method. The digital version revealed problems in the navigation and the position of dialogs or buttons. The results of this research indicate which evaluation criteria are useful for different levels of abstraction of prototypes. [ABSTRACT FROM AUTHOR]