Search

Background Graduate medical educators interested in designing and conducting education research may seek foundational general overview articles on education research methods. Objective We aimed to identify the most useful foundational education research methods articles for medical educators. Methods We identified candidate articles through a 2020 Ovid MEDLINE literature search augmented by the authors' personal files and by cross-checking references of included articles. Articles that were primarily about general education research principles were included; articles were excluded if they were not focused on medical education research methods, were published prior to the year 2000, were written in a non-English language, or had no available abstracts. We conducted a modified Delphi study with 10 participants representing a range of specialties and education research experience to find consensus about the most useful articles. We planned 3 rounds of the Delphi process, the first to eliminate any articles not deemed useful for this audience, and the second and third rounds to include articles that at least 80% of the panel deemed "most useful" to education researchers. Results Of 25 relevant articles identified in the literature search: one was excluded in round 1, 7 met the a priori threshold of 80% agreement for inclusion in round 2, and an additional 2 met inclusion in round 3. These 9 foundational education research methods articles relevant to graduate medical educators are described, along with a capsule summary and specific use for education researchers. Conclusions Our modified Delphi study of foundational education research methods articles identified 9 articles deemed useful for graduate medical educators who are seeking methods resources.

Purpose: Entrustable professional activities (EPAs) are a widely used framework for curriculum and assessment, yet the variability in emergency medicine (EM) training programs mandates the development of EPAs that meet the needs of the specialty as a whole. This requires eliciting and incorporating the perspectives of multiple stakeholders (i.e., faculty, residents, and patients) in the development of EPAs. Without a shared understanding of what a resident must be able to do upon graduation, we run the risk of advancing ill-prepared residents that may provide inconsistent care., Methods: In an effort to address these challenges, beginning in February 2020, the authors assembled an advisory board of 25 EM faculty to draft and reach consensus on a final list of EPAs that can be used across all training programs within the specialty of EM. Using modified Delphi methodology, the authors came to consensus on an initial list of 22 EPAs. The authors presented these EPAs to faculty supervisors, residents, and patients for refinement. The authors collated and analyzed feedback from focus groups of residents and patients using thematic analysis. The EPAs were subsequently refined based on this feedback., Results: Stakeholders in EM residency training endorsed a final revised list of 22 EPAs. Stakeholder focus groups highlighted two main thematic considerations that helped shape the finalized list of EM EPAs: attention to the meaningful nuances of EPA language and contextualizing the EPAs and viewing them developmentally., Conclusions: To foreground all key stakeholders within the EPA process for EM, the authors chose within the development process to draft; come to consensus; and refine EPAs for EM in collaboration with relevant faculty, patient, and resident stakeholders. Each stakeholder group contributed meaningfully to the content and intended implementation of the EPAs. This process may serve as a model for others in developing stakeholder-responsive EPAs., Competing Interests: The authors declare no conflicts of interest., (© 2024 Society for Academic Emergency Medicine.)

Background: It is essential that medical education (MedEd) fellows achieve desired outcomes prior to graduation. Despite the increase in postgraduate MedEd fellowships in emergency medicine (EM), there is no consistently applied competency framework. We sought to develop entrustable professional activities (EPAs) for EM MedEd fellows., Methods: From 2021 to 2022, we used a modified Delphi method to achieve consensus for EPAs. EM education experts generated an initial list of 173 EPAs after literature review. In each Delphi round, panelists were asked to make a binary choice of whether to include the EPA. We determined an inclusion threshold of 70% agreement a priori. After the first round, given the large number of EPAs meeting inclusion threshold, panelists were instructed to vote whether each EPA should be included in the "20 most important" EPAs for a MedEd fellowship. Modifications were made between rounds based on expert feedback. We calculated descriptive statistics., Results: Seventeen experts completed four Delphi rounds each with 100% response. After Round 1, 87 EPAs were eliminated and two were combined. Following Round 2, 46 EPAs were eliminated, seven were combined, and three were included in the final list. After the third round, one EPA was eliminated and 13 were included. After the fourth round, 11 EPAs were eliminated. The final list consisted of 16 EPAs in domains of career development, education theory and methods, research and scholarship, and educational program administration., Conclusions: We developed a list of 16 EPAs for EM MedEd fellowships, the first step in implementing competency-based MedEd., Competing Interests: The authors declare no conflicts of interest., (© 2024 The Authors. AEM Education and Training published by Wiley Periodicals LLC on behalf of Society for Academic Emergency Medicine.)

Introduction: Resident-as-teacher (RAT) curricula have improved teaching behavior, ability, and confidence among resident participants. However, there are limited data on the appropriate format, length, and content. With teaching being a core residency competency and residents delivering one-third of student teaching in the clinical setting, properly training residents in clinical teaching is essential. We created a formal, scalable, asynchronous RAT curriculum. We report the pilot implementation of this curriculum along with feasibility, user acceptability, and preliminary knowledge outcomes., Methods: In this pilot pre-post interventional study, senior emergency medicine residents completed a formalized online education curriculum during their "teach month." The curriculum consisted of three online modules completed weekly over a 4-week rotation. Topics included adult learning, assessment and feedback, and group teaching. Several surveys were sent to residents before and after curriculum implementation. The surveys rated satisfaction and asked several education-specific knowledge questions to assess learning. Ratings were analyzed using means and confidence intervals (95%). Knowledge questions were graded and then analyzed by ANOVA and Fisher's LSD test., Results: After the online modules were completed, the intervention group residents' mean score on knowledge questions was significantly higher than that prior to the curriculum and significantly higher than that the control group (previous graduated residents; 6.00 vs. 2.70, p  = 0.0001; and 6.00 vs. 3.00, p  = 0.0003, respectively). This score was maintained 3 months after completing the online modules. Intervention group residents were more satisfied with the online education resources than the control group ( p  = 0.048)., Conclusions: Residents participating in a formalized online curriculum during their teach month demonstrate a high comprehension of education concepts and increased satisfaction with the provided educational resources and report high satisfaction with the teach month. Our pilot study suggests that a short online education-focused curriculum is an effective method of providing RAT training and may be applicable to clinical teachers across specialties and experience levels., Competing Interests: The authors declare no conflicts of interest., (© 2024 Society for Academic Emergency Medicine.)

Background Curriculum development is an essential domain for medical educators, yet specific training in this area is inconsistent. With competing demands for educators' time, a succinct resource for best practice is needed. Objective To create a curated list of the most essential articles on curriculum development to guide education scholars in graduate medical education. Methods We used a modified Delphi method, a systematic consensus strategy to increase content validity, to achieve consensus on the most essential curriculum development articles. We convened a panel of 8 experts from the United States in curricular development, with diverse career stages, institutions, gender, and specialty. We conducted a literature search across PubMed and Google Scholar with keywords, such as "curriculum development" and "curricular design," to identify relevant articles focusing on a general overview or approach to curriculum development. Articles were reviewed across 3 iterative Delphi rounds to narrow down those that should be included in a list of the most essential articles on curriculum development. Results Our literature search yielded 1708 articles, 90 of which were selected for full-text review, and 26 of which were identified as appropriate for the modified Delphi process. We had a 100% response rate for each Delphi round. The panelists narrowed the articles to a final list of 5 articles, with 4 focusing on the development of new curriculum and 1 on curriculum renewal. Conclusions We developed a curated list of 5 essential articles on curriculum development that is broadly applicable to graduate medical educators.

Purpose: The process of screening and selecting trainees for postgraduate training has evolved significantly in recent years, yet remains a daunting task. Postgraduate training directors seek ways to feasibly and defensibly select candidates, which has resulted in an explosion of literature seeking to identify root causes for the problems observed in postgraduate selection and generate viable solutions. The authors therefore conducted a scoping review to analyze the problems and priorities presented within the postgraduate selection literature to explore practical implications and present a research agenda., Method: Between May 2021 and February 2022, the authors searched PubMed, EMBASE, Web of Science, ERIC, and Google Scholar for English language literature published after 2000. Articles that described postgraduate selection were eligible for inclusion. 2,273 articles were ultimately eligible for inclusion. Thematic analysis was performed on a subset of 100 articles examining priorities and problems within postgraduate selection. Articles were sampled to ensure broad thematic and geographical variation across the breadth of articles that were eligible for inclusion., Results: Five distinct perspectives or value statements were identified in the thematic analysis: (1) Using available metrics to predict performance in postgraduate training; (2) identifying the best applicants via competitive comparison; (3) seeking alignment between applicant and program in the selection process; (4) ensuring diversity, mitigation of bias, and equity in the selection process; and (5) optimizing the logistics or mechanics of the selection process., Conclusions: This review provides insight into the framing and value statements authors use to describe postgraduate selection within the literature. The identified value statements provide a window into the assumptions and subsequent implications of viewing postgraduate selection through each of these lenses. Future research must consider the outcomes and consequences of the value statement chosen and the impact on current and future approaches to postgraduate selection., (Copyright © 2023 by the Association of American Medical Colleges.)

Background: Faculty development (FD) encompasses structured programming that aims to enhance educator knowledge, skill, and behavior. No uniform framework for faculty development exists, and academic institutions vary in their faculty development programming, ability to overcome barriers, resource utilization, and achievement of consistent outcomes., Objective: The authors aimed to assess current FD needs among emergency medicine educators from six geographically and clinically distinct academic institutions to inform overall faculty development advancement in emergency medicine (EM)., Methods: This cross-sectional study assessed FD needs among EM educators. A survey was developed, piloted, and distributed to faculty via each academic institution's internal e-mail listserv. Respondents were asked to rate their comfort level with and interest in several domains of FD. Respondents were also asked to identify their previous experience, satisfaction with the FD they have received, and barriers to receiving FD., Results: Across six sites, 136/471 faculty completed the survey in late 2020 (response rate of 29%): 69.1% of respondents reported being satisfied overall with the FD they have participated in, and 50.7% reported being satisfied with education FD specifically. Faculty report higher comfort levels and interest in several domains when satisfied with the education-specific FD they have received compared with those who report not being satisfied., Conclusions: EM faculty report generally high satisfaction with the overall faculty development they have received, although only half express satisfaction with their education-related faculty development. Faculty developers in EM may incorporate these results to inform future faculty development programs and frameworks., (Copyright © 2023 Elsevier Inc. All rights reserved.)

Objectives: Faculty development is essential for academic emergency physicians to maintain clinical skills and succeed in administrative and leadership roles and for career advancement and satisfaction. Faculty developers in emergency medicine (EM) may struggle to find shared resources to guide faculty development efforts in a way that builds on existing knowledge. We aimed to review the EM-specific faculty development literature since 2000 and come to a consensus about the most useful for EM faculty developers., Methods: A database search was conducted on the topic of faculty development in EM from 2000 to 2020. After identifying relevant articles, we performed a modified Delphi process in three rounds, using a team of educators with a range of experiences with faculty development and education research, to identify articles that would be most useful to a broad audience of faculty developers., Results: We identified 287 potentially relevant articles on the topic of EM faculty development, 244 from the initial literature search, 42 from manual review of the references of the papers meeting inclusion criteria, and one by our study group's recommendation. Thirty-six papers met final inclusion criteria and underwent full-text review by our team. The Delphi process yielded six articles that were deemed most highly relevant over the three rounds. Each of these articles is described here, along with summaries and implications for faculty developers., Conclusions: We present the most useful EM papers from the past two decades for faculty developers seeking to develop, implement, or revise faculty development interventions., Competing Interests: The authors declare no potential conflict of interest., (© 2023 Society for Academic Emergency Medicine.)

Promotion and tenure (P&T) letters are a key component of the academic advancement portfolio. Despite their importance, many faculty are not trained to write these letters and there is limited literature describing the approach and key components. This paper reviews the role of P&T letters and provides general guidelines for writers. We present a step-by-step guide, which includes how to respond to requests, the role of institutional guidelines, providing context to the letter, evaluating candidates, and delivering an overall recommendation. Finally, we discuss current controversies in P&T letters. This paper is intended to help novice and more experienced writers to enhance their P&T letters, while also helping applicants for promotion understand what is being asked of their letter writers., (© 2022 by the Society for Academic Emergency Medicine.)

Background: The health professions education (HPE) landscape has shifted substantively with increasing professionalization of research and scholarship. Clinician educators seeking to become competitive in this domain often pursue fellowships and master's degrees in HPE, but there are few resources for the continuing professional development (CPD) of those who seek to continue developing their scholarly practice within HPE. Acknowledging the multiple players in this landscape, the authors sought to design a new "beyond beginners" HPE research program using a novel needs assessment planning process., Methods: The authors developed and conducted a new three-phase, five-step process that sets forth a programmatic approach to conducting a needs assessment for a CPD course in HPE research. The five steps of the CLAIM method are: Competitive analysis, Literature review with thematic analysis, Ask stakeholders, Internal review by experts, and Mapping of a curriculum. These steps are organized into three phases (Discovery, Convergence, and Synthesis)., Results: Over a 12-month period, the authors completed a comprehensive needs assessment. The CLAIM process revealed that longitudinal digital connection, diverse and in depth exposure to HPE research methods, skills around scholarly publishing, and leadership and management of research would be beneficial to our design., Conclusions: The CLAIM method provided scaffolding to help the authors create a robust curriculum that adopts a scholarly approach for developing a HPE research course. This needs assessment methodology may be useful in other CPD contexts., Competing Interests: Drs. Chan and Gottlieb report teaching honoraria from the ALiEM, LLC for their work on the Faculty Incubator program. Drs. Lawson and Santen reports that their institution has received special funding from the American Medical Association. Drs. Chan and Clarke report funding for their work from the University. Dr. Chan has received funding from the Canadian Association of Emergency Physicians, Royal College of Physicians and Surgeons of Canada, and the PSI Foundation., (© 2022 by the Society for Academic Emergency Medicine.)

Cognitive bias permeates almost every learner assessment in medical education. Assessment bias has the potential to affect a learner's education, future career and sense of self-worth. Decades of data show that there is little educators can do to overcome bias in learner assessments. Using in-group favouritism as an example, we offer an evidence-based, three-step solution to understand and move forward with cognitive bias in assessment: (1) Name: a simple admission about the presence of inherent bias in assessment, (2) Reframe: a rephrasing of assessment language to shed light on the assessor's subjectivity and (3) Check-in: a chance to ensure learner understanding and open lines of bidirectional communication. This process is theory-informed and based on decades of educational, sociological and psychological literature; we offer it as a logical first step towards a much-needed paradigm shift towards addressing bias in learner assessment., (© 2021 John Wiley & Sons Ltd and The Association for the Study of Medical Education.)

Racism in medicine affects patients, trainees, and practitioners and contributes to health care inequities. An effective strategy to actively oppose the structural racism ingrained in the fabric of medicine is to intentionally and systematically address diversity, equity, and inclusion (DEI) in medical education and research. As part of ARMED MedEd, a new longitudinal cohort course in advanced research methods in medical education, sponsored by the Society for Academic Emergency Medicine, the leadership team deliberately included a nested DEI curriculum. The goal of the DEI curriculum is to reduce bias in development, recruitment, and implementation of education research studies to promote equity and inclusion in medical education, research, and ultimately, patient care. A team of medical educators with expertise in DEI developed curricular elements focusing on DEI in education research. The two major components are a didactic curriculum (including implicit bias training) to teach researchers to consider equity as they design studies and a consultative service to refine research protocols to address lingering unintended bias. A dedicated focus on DEI can be incorporated into an advanced education research methodology course to raise awareness and provide tools to avoid bias in research design and implementation of interventions. Over time, the network of education researchers who are trained in DEI awareness will grow and provide equitable offerings to their learners to mitigate health inequities., Competing Interests: Dr. Coates serves on the Board of Directors of the Society for Academic Emergency Medicine, the organization that sponsors ARMED MedEd. She recused herself from the vote for approval. The other authors have no potential conflicts to disclose., (© 2021 Society for Academic Emergency Medicine.)

Background: In 2017, the Accreditation Council for Graduate Medical Education (ACGME) revised its Common Program Requirements to support trainees and faculty by mandating programs to provide dedicated wellness resources and education. Emergency medicine may benefit from this change due to high burnout rates within the specialty. However, the current state of wellness interventions in emergency medicine (EM) residency programs has not yet been well described. Understanding current practices is necessary to assess unmet needs and inform the development and evaluation of future interventions that aim to improve trainee wellness., Objectives: The goal of this study was to describe currently implemented wellness interventions in EM residency programs., Methods: This descriptive study surveyed 250 ACGME-accredited EM residency programs between March 1 and June 1, 2020, regarding wellness interventions. The survey included demographic questions; structured multiple-choice questions about cost, frequency, and champions; and free-text response options to briefly describe interventions. Respondents were also asked to classify the interventions according to the seven factors described in the National Academy of Medicine Model of Clinician Well-Being and Resilience., Results: Ninety of 250 (36% response rate) residency programs participated, describing 162 unique wellness interventions. Respondents classified the majority of interventions ( n  = 136) as targeting personal factors according to the National Academy of Medicine model. Qualitative analysis revealed five major themes describing the interventions: program culture, program factors, environmental and clinical factors, wellness activities and practices, and wellness resources., Conclusions: Results of this survey may help to inform a national needs assessment addressing the current state of wellness interventions in EM residency programs. Our results highlight the need for more interventions targeting external factors impacting resident wellness., Competing Interests: The authors have no potential conflicts to disclose., (© 2021 by the Society for Academic Emergency Medicine.)

Background: Research abstracts are submitted for presentation at scientific conferences; however, criteria for judging abstracts are variable. We sought to develop two rigorous abstract scoring rubrics for education research submissions reporting (1) quantitative data and (2) qualitative data and then to collect validity evidence to support score interpretation., Methods: We used a modified Delphi method to achieve expert consensus for scoring rubric items to optimize content validity. Eight education research experts participated in two separate modified Delphi processes, one to generate quantitative research items and one for qualitative. Modifications were made between rounds based on item scores and expert feedback. Homogeneity of ratings in the Delphi process was calculated using Cronbach's alpha, with increasing homogeneity considered an indication of consensus. Rubrics were piloted by scoring abstracts from 22 quantitative publications from AEM Education and Training "Critical Appraisal of Emergency Medicine Education Research" (11 highlighted for excellent methodology and 11 that were not) and 10 qualitative publications (five highlighted for excellent methodology and five that were not). Intraclass correlation coefficient (ICC) estimates of reliability were calculated., Results: Each rubric required three rounds of a modified Delphi process. The resulting quantitative rubric contained nine items: quality of objectives, appropriateness of methods, outcomes, data analysis, generalizability, importance to medical education, innovation, quality of writing, and strength of conclusions (Cronbach's α for the third round = 0.922, ICC for total scores during piloting = 0.893). The resulting qualitative rubric contained seven items: quality of study aims, general methods, data collection, sampling, data analysis, writing quality, and strength of conclusions (Cronbach's α for the third round = 0.913, ICC for the total scores during piloting = 0.788)., Conclusion: We developed scoring rubrics to assess quality in quantitative and qualitative medical education research abstracts to aid in selection for presentation at scientific meetings. Our tools demonstrated high reliability., Competing Interests: The authors have no potential conflicts to disclose., (© 2021 by the Society for Academic Emergency Medicine.)

Learning critical care medicine in the pediatric intensive care unit (PICU) can be stressful. Through semistructured interviews ( n  = 16), this study explored the emotions, perceptions, and motivations of pediatric medicine (PM) and emergency medicine (EM) residents, as they prepared for their first PICU rotation. Qualitative data were collected and analyzed using the grounded theory method. Three resultant themes emerged: (1) residents entered the PICU with a range of intense emotions and heightened expectations; (2) they experienced prior history of psychologically traumatic learning events (adverse learning experiences or ALEs); and (3) informed by ALEs, residents prepared for their rotation by focusing heavily on their most basic level of physiological needs and adopting a survival mindset prior to the start of the rotation. These three themes led to a substantive, or working, theory that ALE-associated events may affect how residents approach upcoming learning opportunities. Consequently, adapting a trauma-informed approach as a component of medical education may improve resident learning experiences in the PICU and beyond., Competing Interests: Conflict of Interest B.W.S. involvement was supported by a National Library of Medicine training grant 5T15LM007088. The rest authors declare no conflict of interest., (Thieme. All rights reserved.)

Introduction: The Academic Life in Emergency Medicine (ALiEM) Faculty Incubator program is a longitudinal, 1-year, virtual faculty development program for early- and mid-career faculty members that crosses specialties and institutions. This study sought to evaluate the outcomes among 3 years of participants., Methods: This cross-sectional survey study evaluated postcourse and 1-year outcomes from three graduated classes of the ALiEM Faculty Incubator program. The program evaluation survey was designed to collect outcomes across multiple Kirkpatrick levels using pre/post surveys and tracking of abstracts, publications, speaking opportunities, new leadership positions, and new curricula., Results: Over 3 years, 89 clinician educators participated in the program. Of those, 59 (66%) completed the initial survey and 33 (37%) completed the 1-year survey. Participants reported a significant increase in knowledge (4.1/9.0 vs. 7.0/9.0). The number of abstracts, publications, and invited presentations significantly increased after course completion and continued postcourse. A total of 37 of 59 (62.7%) developed a new curriculum during the course and 19 of 33 (57.6%) developed another new curriculum after the course. A total of 29 of 59 (49.2%) began a new leadership position upon course completion with 15 of 33 (45.5%) beginning another new leadership position 1 year later., Discussion: The ALiEM Faculty Incubator program demonstrated an increase in perceived knowledge and documented academic productivity among early- and mid-career medical educators., (© 2021 Society for Academic Emergency Medicine.)

Background: Didactic lectures remain common in medical education. Many faculty physicians do not receive formal training on public presentations or leading instructional sessions. Coaching has emerged in medical education with the potential to positively impact skills. We sought to evaluate a novel, national faculty peer-coaching program created to improve lecture presentation skills and foster career development., Methods: This was a mixed-methods study of participant and faculty perceptions after completing the Council of Residency Directors in Emergency Medicine Academy Coaching Program. Participants completed an online evaluative survey consisting of multiple choice and Likert-type items. Program coaches participated in semistructured interviews. Descriptive statistics were reported for survey data. Thematic qualitative analysis by two independent reviewers was performed on interview data., Results: During 2012 to 2017, a total of 30 participants and 11 coaches from 37 residency programs across the United States engaged in the program. Twenty-four (80%) participants completed the survey. Eight (73%) coaches participated in semistructured interviews. Data were collected between October and December 2018. The mean ± SD numbers of national presentations participants had given before and after the coaching program were 6.92 ± 7.68 and 16.42 ± 15.43, respectively. Since their coaching, most participants (87.5%) have been invited to give a lecture at another institution. Many participants felt that the program improved their lecture evaluations, public speaking, ability to engage an audience, and professional development. Almost all (92%) would recommend the program to a colleague. The coaches perceived multiple benefits including improved skills, self-reflection, networking, career advancement, and personal fulfillment. Suggestions for improvement included improved administrative processes, more clear expectations, increased marketing, and increased participant and coach engagement., Conclusion: Participants and coaches perceived multiple benefits from this novel, national faculty coaching program. With identification of the success, challenges, and suggestions for improvement, others may benefit as they develop coaching programs in medical education., Competing Interests: The authors have no potential conflicts to disclose., (© 2021 by the Society for Academic Emergency Medicine.)

Introduction: Recent research demonstrates burnout prevalence rates as high as 76% in emergency medicine (EM) residents. In 2017 the Accreditation Council for Graduate Medical Education (ACGME) required that all training programs provide dedicated wellness education for their trainees as a requirement for accreditation. We aimed to conduct a systematic review of published wellness interventions conducted in EM residency programs following the implementation of the 2017 ACGME Common Program Requirements change in order to characterized published intervention and evaluate their effectiveness., Methods: We applied a published approach to conducting systematic reviews of the medical education literature. We performed a search of the literature from January 1, 2017-February 1, 2020. Studies were included for final review if they described a specific intervention and reported outcomes with the primary goal of improving EM resident wellness. Outcomes were characterized using the Kirkpatrick training evaluation model., Results: Eight of 35 identified studies met inclusion criteria. Most described small convenience samples of EM residents from single training programs and used the satisfaction rates of participants as primary outcome data. Only quantitative assessment methods were used. Studies addressed only a limited number of factors affecting resident wellness. The majority of interventions focused on personal factors, although a few also included sociocultural factors and the learning and practice environment., Conclusion: There is a relative dearth of literature in the area of research focused on interventions designed to improve EM resident wellness. Furthermore, the studies we identified are narrow in scope, involve relatively few participants, and describe programmatic changes of limited variety. Future directions include an increase and emphasis on multi-institutional studies, randomized controlled trials, qualitative methodology, and opportunities for funded research.

Purpose: A previous study found that milestone ratings at the end of training were higher for male than for female residents in emergency medicine (EM). However, that study was restricted to a sample of 8 EM residency programs and used individual faculty ratings from milestone reporting forms that were designed for use by the program's Clinical Competency Committee (CCC). The objective of this study was to investigate whether similar results would be found when examining the entire national cohort of EM milestone ratings reported by programs after CCC consensus review., Method: This study examined longitudinal milestone ratings for all EM residents (n = 1,363; 125 programs) reported to the Accreditation Council for Graduate Medical Education every 6 months from 2014 to 2017. A multilevel linear regression model was used to estimate differences in slope for all subcompetencies, and predicted marginal means between genders were compared at time of graduation., Results: There were small but statistically significant differences between males' and females' increase in ratings from initial rating to graduation on 6 of the 22 subcompetencies. Marginal mean comparisons at time of graduation demonstrated gender effects for 4 patient care subcompetencies. For these subcompetencies, males were rated as performing better than females; differences ranged from 0.048 to 0.074 milestone ratings., Conclusions: In this national dataset of EM resident milestone assessments by CCCs, males and females were rated similarly at the end of their training for the majority of subcompetencies. Statistically significant but small absolute differences were noted in 4 patient care subcompetencies.

In 2012, the Society for Academic Emergency Medicine convened a consensus conference on the state of medical education research with goals of defining and advancing a future research agenda. Since that time, emergency medicine (EM) education research has grown significantly. A task force of EM education experts was assembled and sought to understand the current state of EM education research and future directions. Among the advances are increases in medical education fellowships, advanced degree and certification programs, faculty development programs, publication venues, and funding. These findings are discussed in light of the prior objectives from the 2012 consensus conference, and recommendations for future directions are provided., (© 2019 by the Society for Academic Emergency Medicine.)

Study Objective: Research in basic, translational, and clinical emergency medicine has made great strides since the formalization of emergency medicine as a specialty. Our objective is to identify and analyze strategies used by emergency medicine research pioneers to inform further advancement of research in emergency medicine, particularly for aspiring researchers and those in emerging areas, using emergency medicine medical education as one example., Methods: This was a prospective, grounded-theory, qualitative study, using a constructivist/interpretivist paradigm. Leading basic science, translational, and clinical emergency medicine researchers who completed residency before 1995 were eligible for structured interviews. Thematic coding followed an iterative process until saturation was reached. A theoretic model was developed and analyzed., Results: Research pioneers valued advanced methodological training and mentorship. Barriers to funding were lack of recognition of emergency medicine as a specialty, absence of a research history, and lack of training and funding resources. Deliberate interventions to improve emergency medicine research included educational sessions at national meetings, external (to emergency medicine) mentor pairings, targeted funding by emergency medicine organizations, and involvement with funding agencies. Pioneers facilitate research excellence by serving as mentors and allocating funds or protected time to develop researchers. To advance emerging subfields of research in emergency medicine, pioneers recommend advanced methodological training that is specific to the area, deliberate mentorship, and the formation of research consortia to conduct generalizable outcomes-based studies., Conclusion: Research pioneers in emergency medicine cite mentorship, advanced skills obtained through fellowship or graduate degrees, deliberate collaboration with experienced researchers, support from emergency medicine organizations, and forming networks as the cornerstones of success., (Copyright © 2018 American College of Emergency Physicians. Published by Elsevier Inc. All rights reserved.)

In this issue of Academic Medicine, Lundsgaard and colleagues present "Embracing Multiple Stakeholder Perspectives in Defining Trainee Competence," a qualitative exploration of how various stakeholders contribute to the understanding of trainee competence. Drawing on stakeholder theory from business management, the authors of that report explore how the perceptions of key stakeholders (leaders/administrators, nurses/nurse practitioners, trainees, and patients) either confirm, enhance, or complicate the picture of competence that emerges from the perspectives of senior physician supervisors.In this Invited Commentary, the author considers the potential effects of applying stakeholder theory to educational assessment and elaborates on Lundsgaard and colleagues' findings that additional stakeholder perceptions may be redundant in some instances and present conflicting understandings of competence in others. She also emphasizes the importance of a process of defining competence, and ultimately developing assessment programs, that includes measuring and illuminating conflicting perceptions of what constitutes competence. Finally, the author considers potential applications of Lundsgaard and colleagues' results for educators and assessors with comprehensive assessment programs already in place.There are limitations and risks to applying a conceptual framework that aims to increase value for a company to the training of physicians. However, with thoughtful application, incorporating multiple stakeholders into instrument development and workplace-based assessment may provide educators with a more nuanced picture of competence that incorporates the values of those ultimately impacted by trainee performance.

Introduction: The objective of this study was to review and critically appraise the medical education literature pertaining to a flipped-classroom (FC) education model, and to highlight influential papers that inform our current understanding of the role of the FC in medical education., Methods: A search of the English-language literature querying Education Resources Information Center (ERIC), PsychINFO, PubMed, and Scopus identified 296 papers related to the FC using either quantitative, qualitative, or review methods. Two reviewers independently screened each category of publications using previously established exclusion criteria. Eight reviewers then independently scored the remaining 54 publications using either a qualitative, quantitative, or review-paper scoring system. Each scoring system consisted of nine criteria and used parallel metrics that have been previously used in critical appraisals of education research., Results: A total of 54 papers (33 quantitative, four qualitative, and 17 review) on FC met a priori criteria for inclusion and were critically appraised and reviewed. The top 10 highest scoring articles (five quantitative studies, two qualitative studies, and three review papers) are summarized in this article., Conclusion: This installment of the Council of Emergency Medicine Residency Directors (CORD) Academy Critical Appraisal series highlights 10 papers that describe the current state of literature on the flipped classroom, including an analysis of the benefits and drawbacks of an FC approach, practical implications for emergency medicine educators, and next steps for future research., Competing Interests: Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.