Search

Evaluation professionals need to be nimble and innovative in their approaches in order to be relevant and provide useful evidence to decision-makers, stakeholders, and society in the crowded public policy landscape. In this article, we offer serious games as a method that can be employed by evaluators to address three persisting challenges in current evaluation practice: inclusion of stakeholders, understanding of causal mechanisms, and utilization of evaluation findings. We provide a framework that distinguishes among games along two crucial aspects of evaluation inquiry - its function and the nature of the evaluand. We offer examples of successfully implemented games in each set of the four arenas we delineate: teaching knowables, testing retention, crash-testing mechanisms, and exploring systems. We explain how games can be employed to promote learning about and among stakeholders, and to collect valuable intelligence about the operations of programs and policies.

The potential of real‐world data to inform clinical trial design and supplement control arms has gained much interest in recent years. The most common approach relies on reproducing control arm outcomes by matching real‐world patient cohorts to clinical trial baseline populations. However, recent studies pointed out that there is a lack of replicability, generalisability, and consensus. In this article, we propose a novel approach that aims to explore and examine these discrepancies by concomitantly investigating the impact of selection criteria and operations on the measurements of outcomes from the patient data. We tested the approach on a dataset consisting of small‐cell lung cancer patients receiving platinum‐based chemotherapy regimens from a real‐world data cohort (n = 223) and six clinical trial control arms (n = 1224). The results showed that the discrepancy between real‐world and clinical trial data potentially depends on differences in both patient populations and operational conditions (e.g., frequency of assessments, and censoring), for which further investigation is required. Discovering and accounting for confounders, including hidden effects of differences in operations related to the treatment process and clinical trial study protocol, would potentially allow for improved translation between clinical trials and real‐world data. Continued development of the method presented here to systematically explore and account for these differences could pave the way for transferring learning across clinical studies and developing mutual translation between the real‐world and clinical trials to inform clinical study design. [ABSTRACT FROM AUTHOR]

BackgroundSerious games can be a powerful learning tool in higher education. However, the literature indicates that the learning outcome in a serious game depends on the facilitators’ competencies. Although professional facilitators in commercial game-based training have undergone specific instruction, facilitators in higher education cannot rely on such formal instruction, as game facilitation is only an occasional part of their teaching activities. ObjectiveThis study aimed to address the actual competencies of occasional game facilitators and their perceived competency deficits. MethodsHaving many years of experience as professional and occasional facilitators, we (n=7) defined requirements for the occasional game facilitator using individual reflection and focus discussion. Based on these results, guided interviews were conducted with additional occasional game facilitators (n=4) to check and extend the requirements. Finally, a group of occasional game facilitators (n=30) answered an online questionnaire based on the results of the requirement analysis and existing competency models. ResultsOur review produced the following questions: Which competencies are needed by facilitators and what are their training needs? What do current training courses for occasional game facilitators in higher education look like? How do the competencies of occasional game facilitators differ from other competencies required in higher education? The key findings of our analysis are that a mix of managerial and technical competencies is required for facilitating serious games in higher educational contexts. Further, there is a limited or no general competence model for game facilitators, and casual game facilitators rarely undergo any specific, formal training. ConclusionsThe results identified the competencies that game facilitators require and a demand for specific formal training. Thus, the study contributes to the further development of a competency model for game facilitators and enhances the efficiency of serious games.

BackgroundUsing serious games for learning in operations management is well established. However, especially for logistics skills in health care operations, there is little work on the design of game mechanics for learning engagement and the achievement of the desired learning goals. ObjectiveThis contribution presents a serious game design representing patient flow characteristics, systemic resource configurations, and the roles of the players based on a real Swedish emergency ward. The game was tested in a set of game-based learning practices in the modalities of a physical board game and an online multiplayer serious game that implemented the same game structure. MethodsFirst, survey scores were collected using the Game Experience Questionnaire Core and Social Presence Modules to evaluate the experience and acceptance of the proposed design to gamify real processes in emergency care. Second, lag sequential analysis was applied to analyze the impact of the game mechanics on learning behavior transitions. Lastly, regression analysis was used to understand whether learning engagement attributes could potentially serve as significant predicting variables for logistical performance in a simulated learning environment. ResultsA total of 36 students from courses in engineering and management at KTH Royal Institute of Technology participated in both game-based learning practices during the autumn and spring semesters of 2019 and 2020. For the Core Module, significant differences were found for the scores for negative affect and tension compared with the rest of the module. For the Social Presence Module, significant differences were found in the scores for the psychological involvement – negative feelings dimension compared with the rest of the module. During the process of content generation, the participant had access to circulating management resources and could edit profiles. The standard regression analysis output yielded a ΔR2 of 0.796 (F14,31=2725.49, P

BackgroundTelemedicine innovations are rarely adopted into routine health care, the reasons for which are not well understood. Teleguidance, a promising service for remote surgical guidance during endoscopic retrograde cholangiopancreatography (ERCP) was due to be scaled up, but there were concerns that user attitudes might influence adoption. ObjectiveOur objective was to gain a deeper understanding of ERCP practitioners’ attitudes toward teleguidance. These findings could inform the implementation process and future evaluations. MethodsWe conducted semistructured interviews with ERCP staff about challenges during work and beliefs about teleguidance. Theoretical constructs from the technology acceptance model (TAM) guided the thematic analysis. Our findings became input to a 16-item questionnaire, investigating surgeons’ beliefs about teleguidance’s contribution to performance and factors that might interact with implementation. ResultsResults from 20 interviews with ERCP staff from 5 hospitals were used to adapt a TAM questionnaire, exchanging the standard “Ease of Use” items for “Compatibility and Implementation Climate.” In total, 23 ERCP specialists from 15 ERCP clinics responded to the questionnaire: 9 novices (500 ERCP procedures). The average agreement ratings for usefulness items were 64% (~9/14) among experts and 75% (~7/9) among novices. The average agreement ratings for compatibility items were somewhat lower (experts 64% [~9/14], novices 69% [~6/9]). The averages have been calculated from the sum of several items and therefore, they only approximate the actual values. While 11 of the 14 experts (79%) and 8 of the 9 novices (89%) agreed that teleguidance could improve overall quality and patient safety during ERCP procedures, only 8 of the 14 experts (57%) and 6 of the 9 novices (67%) agreed that teleguidance would not create new patient safety risks. Only 5 of the 14 experts (36%) and 3 of the 9 novices (33%) were convinced that video and image transmission would function well. Similarly, only 6 of the 14 experts (43%) and 6 of the 9 novices (67%) agreed that administration would work smoothly. There were no statistically significant differences between the experts and novices on any of the 16 items (P