Search

Phenomenon: Because of its importance in residency selection, the United States Medical Licensing Examination Step 1 occupies a critical position in medical education, stimulating national debate about appropriate score use, equitable selection criteria, and the goals of undergraduate medical education. Yet, student perspectives on these issues and their implications for engagement with health systems science-related curricular content are relatively underexplored. Approach: We conducted an online survey of medical students at 19 American allopathic medical schools from March-July, 2019. Survey items were designed to elicit student opinions on the Step 1 examination and the impact of the examination on their engagement with new, non-test curricular content related to health systems science. Findings: A total of 2856 students participated in the survey, representing 23.5% of those invited. While 87% of students agreed that doing well on the Step 1 exam was their top priority, 56% disagreed that studying for Step 1 had a positive impact on engagement in the medical school curriculum. Eighty-two percent of students disagreed that Step 1 scores should be the top item residency programs use to offer interviews. When asked whether Step 1 results should be reported pass/fail with no numeric score, 55% of students agreed, while 33% disagreed. The majority of medical students agreed that health systems science topics were important but disagreed that studying for Step 1 helped learn this content. Students reported being more motivated to study a topic if it was on the exam, part of a course grade, prioritized by residency program directors, or if it would make them a better physician in the future. Insights: These results confirm the primacy of the United States Medical Licensing Examination Step 1 exam in preclinical medical education and demonstrate the need to balance the objectives of medical licensure and residency selection with the goals of the broader medical profession. The survey responses suggest several potential solutions to increase student engagement in health systems science curricula which may be especially important after Step 1 examination results are reported as pass/fail.

This article shares lessons learned while evaluating the implementation of a Clinical and Translational Research Centre (CTR). To meet its overarching goals, the CTR consists of numerous research support units (e.g., biostatistics, community engagement, professional development) that are intended to work together collaboratively. It is then argued that an evaluation approach grounded in system thinking was the best fit to evaluate this key CTR design feature. The rationale for selecting systems evaluation theory (SET) as the evaluation framework best suited to evaluate the CTR infrastructure is then presented. The application of SET and the lessons learned are then shared. This article concludes that there are many similarly structured programmes worldwide to which the lessons learned can be applied and upfront investments in using a system approach are rewarded by providing meaningful and useful evaluation recommendations for system change.

This article shares lessons learned in applying system evaluation theory (SET) to evaluate a Clinical and Translational Research Center (CTR) funded by the National Institutes of Health. After describing how CTR support cores are intended to work interdependently as a system, the case is made for SET as the best fit for evaluating this evaluand. The article then details how the evaluation was also challenged to facilitate a CTR culture shift, helping support cores to move from working autonomously to working together and understanding how the cores’ individual operating processes impact each other. This was achieved by incorporating the Homeland Security Exercise and Evaluation Program (HSEEP) building block approach to implement SET. Each of the seven HSEEP building blocks is examined for alignment with each of SET’s three steps and the ability to systematically support the goal of moving CTR cores toward working interdependently. The implications of using HSEEP to support SET implementation for future evaluations are discussed.

Promoting optimal health outcomes for diverse patients and populations requires the acknowledgement and strengthening of interdependent relationships between health professions education programs, health systems, and the communities they serve. Educational programs must recognize their role as integral components of a larger system. Educators must strive to break down silos and synergize efforts to foster a health care workforce positioned for collaborative, equitable, community-oriented practice. Sharing interprofessional and interinstitutional strategies can foster wide propagation of educational innovation while accommodating local contexts. This paper outlines how member schools of the American Medical Association Accelerating Change in Medical Education Consortium leveraged interdependence to accomplish transformative innovations catalyzed by systems thinking and a community of innovation.

Empirical evidence suggests that respondents systematically overreport socially desirable behaviors and systematically underreport socially undesirable behaviors. This "social desirability response bias (SDRB)" presents significant challenges for research that relies on self-report measures to assess behaviors that adhere to or violate social norms. The present study used a state-of-the-art polygraph system to examine SDRB in widely used aggression questionnaires, including the Buss-Perry Aggression Questionnaire-Short Form, the Indirect Aggression subscale of the Aggression Questionnaire by Western Psychological Services, and the Reactive-Proactive Questionnaire. Sixteen college students with no criminal record, no known prior history of aggressive behavior, and no reported drug abuse, responded to verbally administered forms of the instruments. Indirect aggression items produced the largest, statistically significant physiological response across the sampled channels. The magnitude of this response was negatively and significantly correlated with the self-report ratings of the frequency of such behaviors. The mean separation between physiological (skin conductance) and self-report responses for indirect aggression remained significant and consistent with correlational analyses when both types of responses were converted to the same scale and compared directly. Finally, the relative magnitude of skin conductance response for items assessing indirect aggression was significantly greater than the relative magnitude of skin conductance response for direct aggression. Overall, the study suggests that the use of automated state-of-the art polygraph systems may potentially identify sensitive items on self-report instruments where social responsibility response bias is possible. Implications for the use of this procedure with such instruments are discussed.

Background Science, technology, engineering, and math (STEM) jobs are expected to make up a significant portion of the U.S. workforce. Unfortunately, the trend in retaining students in STEM majors has been going down. If higher education institutions are going to retain more students in STEM majors, it will be important to understand who leaves STEM fields and why. More than 32% of women college students who declare a STEM major are likely to switch to non-STEM majors before they graduate, whereas only 25% of their male counterparts do so, and women may be as much as 1.5 times more likely than men to leave STEM fields. Thus, women represent a significant potential source for increasing STEM majors. Research suggests that values and expectations are powerful predictors of motivation and persistence in a wide variety of activities, tasks, and careers. This paper describes the development and validation of an instrument to measure student motivation, particularly that of women, leading to decisions to persist in or switch out of collegiate STEM programs. Results The Value-Expectancy STEM Assessment Scale (VESAS), adapted from the Values, Interest, and Expectations Scale, or VIES, was validated with 356 women students from a Midwestern research university as part of a larger study on the reasons that women persist or leave STEM majors. A confirmatory factor analysis suggested a two-factor model, which reflected the components of Eccles et al.’s expectancy-value model. Cronbach’s alphas suggested that the VESAS subscales had high internal consistency. Statistically significant differences were found between STEM switchers and persisters on all of the VESAS subscales, thus lending additional support for the validity of the instrument. Conclusions The VESAS appears to be a valid scale for measuring female college students’ value for and expectations regarding STEM majors. Suggestions are made for use of the VESAS in future studies to examine how motivations of women students enrolled in STEM programs change over time and to better understand when retention interventions might be needed and with whom.

Health professionals have limited opportunities to learn about population health in their curriculum. With a shortage of geriatricians nationwide, health care systems need different ways to provide evidence-based geriatric care. To address both these shortcomings, a serious game, called GeriPOP has been developed to allow trainees to explore the impact of assessment and management of principles of geriatric care (the 4Ms+) on quality of life, health, longevity, and health care costs by applying them to a virtual older adult population. Trainees assume the role of a system manager who is asked to explore ways to optimize health outcomes and lower costs. They develop their population health plan around a framework of Geriatric 4Ms+ and apply it in a virtual panel of older adult patients that move longitudinally into different age bands (65-74; 75-84; 85+). As the game progresses, a dashboard helps trainees track the impact of their treatment decisions across the population. Several levels of play allow trainees to explore various issues intersecting with aging such as gender, diversity, social determinants, and multiple chronic conditions. Periodic debriefings and explanatory pop ups during the game allow trainees to further explore evidence–based Geriatrics. The game engages health care trainees to strengthen their knowledge of Geriatrics through exploration of systems change. Future study is needed on whether Geri POP changes learner attitudes, future clinical practice or healthcare outcomes.

Perceived realism is a key concept in explaining the mental processing of media messages and the societal impact of media. Despite its importance, few studies have examined its conceptualization and dimensional structure from an empirical point of view, especially with regard to digital games. In this paper we present three studies aimed at 1) validating a dimensional structure of realism judgements on shooting games and 2) creating a valid measurement instrument. Exploratory and confirmatory factor analyses support five dimensions of perceived game realism that are commonly presented in previous studies. An investigation of the internal reliability and psychometric properties of the constructs reveals that the measurement instrument conforms to a large extent to the requirements of scale validation. (C) 2016 Elsevier Ltd. All rights reserved.

A controlled, prospective, 2‐year cohort observational study was conducted to test whether weekly geriatric questions delivered through Twitter Poll could improve geriatrics knowledge during an internal medicine clerkship for third‐year medical students. Pre‐ and post‐rotation test results used a modified University of California, Los Angeles geriatric knowledge test that included questions linked to 26 Association of American Medical Colleges geriatric competencies for medical students. Data were analyzed using a general linear model repeated‐measure design and Student t‐test. The primary outcome showed that Twitter Poll participants had more than twice the geriatrics knowledge (p = .002) than students who did not use Twitter Poll. Subset analysis showed different test performances according to sex (p = .03), training site (p = .002), and cohort (p = .003). This study is the first demonstration of Twitter Poll efficacy in medical education and raises questions about whether it could be even more effective if linked to spaced timing of didactic content or supported by annotated answers to geriatrics questions. J Am Geriatr Soc 66:2389–2393, 2018.

Air traffic control (ATC) operations are critical to the U.S. aviation infrastructure, making ATC training a critical area of study. Because ATC performance is heavily dependent on visual processing, it is important to understand how to screen for or promote relevant visual processing abilities. While conventional wisdom has maintained that such abilities are largely innate and stable (i.e., not impacted by training or experience), recent research has begun to question this assumption. For example, intelligence has been thought to be stable, yet intelligence scores on the Ravens Progressive matrices (RPM) change over time. Because the RPM, like ATC, relies on visual pattern recognition, one hypothesized reason for these changes is the increased exposure to visually intensive technologies like videogames. Other constructs such as field-dependence/independence (FD-I) which, like the RPM, rely on visual pattern detection and are thought to be stable, might therefore also be affected by visual technology exposure. Two studies sought to examine the role of videogame play on FD-I and ATC simulation performance by ATC students at a Midwestern university. No benefits of videogame play were found. However, more fourth-year ATC students were field-independent (FI) than were first-year students and the general population. Findings suggest that exposure to the ATC training program over 4 years may result in more FI students, and that FD-I may not be stable. Possible explanations and implications are discussed.

The need for science, technology, engineering, and mathematics majors for our future workforce is growing, yet fewer students are choosing to major in science, technology, engineering, and mathematics areas, and many are underprepared, in part because elementary school preservice teachers are also underprepared. This National Science Foundation-supported project developed and tested the first of several planned modules of a video game based on the Next Generation Science Standards. Results suggest that preservice teachers who play the video game demonstrate improved science content knowledge. The study also found that preservice teachers had positive attitudes toward video games as instructional tools. Implications for preservice teacher education relating to games and science education are discussed.

Like many rapidly growing industries, advances in video game technology are far outpacing research on its design and effectiveness. Relatively little is understood about how to apply what we know about teaching and learning to optimize game-based learning. For the most part, instructional designers know little about game development and video game developers may know little about training, education and instructional design. In this three part series of articles, four recognized and emerging experts in instructional game design discuss their perspectives on preparing instructional designers to optimize game-based learning. In Part I, we set the context for the series of articles and one of four faculty members who teach a graduate level course on game design discusses what he believes instructional designers should know about instructional game design based on his experiences. Part II will present alternative perspectives from two additional faculty members who teach courses in instructional game design, and Part III will present a fourth perspective along with conclusion that compares the four views.

Many suggest that digital games are a way to address problems with schools, yet research on their ability to promote problem solving, critical thinking, and twenty-first century skill sets appears to be mixed. In this chapter, I suggest that the problem lies not with digital games, but with our conceptualization of what it means to promote problem solving and critical thinking, and how transfer of such skills works in general and, specifically, with games. The power of digital games lies not in some magical power of the medium, but from embedded theories (e.g., situated learning and problem-centered instruction) and from good instructional design (the principles of learning and teaching to which all good instruction must adhere). This chapter describes situated, authentic problem solving (SAPS): a model to explain how digital games can promote transfer and improve attitudes toward mathematics. By examining research on the instructional practices (situated learning) and outcomes (transfer, problem solving, attitudes) that lie at the heart of SAPS, we can chart a path forward for best practices of digital games in mathematics education.

This study was designed to determine the effect of contextualized advisement and competition on transfer of mathematics skills in a computer-based simulation game in which participants helped their “aunt and uncle” fix up a house. Contextualized advisement referred to whether the participant had access to video-based advisement delivered by the aunt and uncle about how to solve the problem, and competition referred to whether or not the participant was playing against a computer character. A total of 123 seventh-and eighth-grade students were randomly assigned to one of five conditions formed by crossing the two independent variables and adding a control group. Results indicated an interaction between competition and contextualized advisement. Participants in the noncompetitive condition had higher transfer scores when they had access to contextualized advisement, while participants in the competitive condition had higher transfer scores when they had no access to contextualized advisement.

The need for STEM majors for our future workforce is growing, yet fewer students are choosing to major in STEM areas, and many are under prepared, in part because elementary school preservice teachers (PSTs) are also under prepared. This NSF-supported project developed and tested the first of several planned modules of a video game based on the Next Generation Science Standards. Results suggest that PSTs who play the video game demonstrate improved science content knowledge. The study also found that PSTs had positive attitudes toward video games as instructional tools. Implications for PST education relating to games and science education are discussed.

A curriculum for prevention of risk-behaviors related to sexually transmitted diseases (STDs), pregnancy, and human immunodeficiency virus (HIV)/AIDS was developed for incarcerated youth, ages thirteen to nineteen. Spanning five general goal areas of self-esteem, communication and condom negotiation, knowledge, attitude, and behavior, the curriculum was developed to encompass twenty-one to twenty-four hours of training. Participants were ninety-nine incarcerated youth at a detention center in the southeastern US. With a pretest–posttest format, assessment of goals consisted of one to twenty items per objective, using t -test comparisons. With eighteen objectives, a significant change ( p =0.05 or less) was found from pretest to posttest in fourteen areas. Areas with greatest change impact were self-esteem, peer mentor communication, knowledge (in five content areas), perceived susceptibility, and intentions to decrease risk behaviors. Curriculum and assessment items are included in the appendix, as well as demographics and plans for future studies.

Problem solving is often discussed as one of the benefits of games and game-based learning (e.g., Gee, 2007a, Van Eck 2006a), yet little empirical research exists to support this assertion. It will be critical to establish and validate models of problem solving in games (Van Eck, 2007), but this will be difficult if not impossible without a better understanding of problem solving than currently exists in the field of serious games. While games can be used to teach a variety of content across multiple domains (Van Eck, 2006b, 2008), the ability of games to promote problem solving may be more important to the field of serious games because problem solving skills cross all domains and are among the most difficult learning outcomes to achieve. This may be particularly important in science, technology, engineering, and math (STEM), which is why serious game researchers are building games to promote problem solving in science (e.g., Gaydos & Squire, this volume; Van Eck, Hung, Bowman, & Love, 2009). This is perhaps why serious game researchers are building games to promote problem solving in science Current research and design theory in serious games are insufficient to explain the relationship between problem solving and games, nor do they support the design of educational games intended to promote problem solving. Problem solving and problem-based learning (PBL) have been studied intensely in both Europe and the United States for more than 75 years, and while the focus of that study and conceptualization of problem solving have evolved during that time, there is a tremendous body of knowledge to draw from. Most recently, researchers (e.g., Jonassen, 1997, 2000, & 2002; Hung, 2006a; Jonassen & Hung, 2008) have made advances in both the delineation and definition of problem types and models for designing effective problems and PBL. Any models and research on the relation of games and problem solving must build on the existing research base in problem solving and PBL rather than unwittingly covering old ground in these areas. In this chapter, the authors present an overview of the dimensions upon which different problems vary, including domain knowledge, structuredness, and their associated learning outcomes. We then propose a classification of gameplay (as opposed to game genre) that accounts for the cognitive skills encountered during gameplay, relying in part on previous classifications systems (e.g., Apperley, 2006), Mark Wolf’s (2006) concept of grids of interactivity (which we call iGrids), and our own cognitive analysis of gameplay. We then use this classification system, the iGrids, and example games to describe eleven different types of problems, the ways in which they differ, and the gameplay types most likely to support them. We conclude with a description of the ability of problems and games themselves to address specific learning outcomes independent of problem solving, including domain-specific learning, higher-order thinking, psychomotor skills, and attitude change. Implications for future research are also described. We believe that this approach can guide the design of games intended to promote problem solving and points the way toward future research in problem solving and games.

Many of the educational outcomes we seek to promote in public education, such as problem solving and critical thinking, are difficult to achieve given the constraints of the real-world classroom. Commercial Off-the-Shelf (COTS) games make excellent tools for addressing both content-based and higher-order learning outcomes, and many educators are exploring their use in the classroom. But making effective use of commercial games in the classroom requires that we understand how games function in relation to the typical instructional strategies and practices of the classroom. The first part of this chapter will examine the theories that underlie the successful integration of commercial games in the classroom and look at an empirically based model, the NTeQ (iNtegrating Technology through inQuiry), for designing lessons that integrate COTS games. This will lay the groundwork for the second part of the chapter in which these theories and the model are discussed in the context of actually designing COTS game-based learning (GBL).

The idea of digital game-based learning (DGBL) is gaining acceptance among researchers, game designers, educators, parents, and students alike. Building new educational games that meet educational goals without sacrificing what makes games engaging remains largely unrealized, however. If we are to build the next generation of learning games, we must recognize that while digital games might be new, the theory and technologies we need to create DGBL has been evolving in multiple disciplines for the last 30 years. This chapter will describe an approach, based on theories and technologies in education, instructional design, artificial intelligence, and cognitive psychology, that will help us build intelligent learning games (ILGs).