Search

Abstract: Precision education (PE) uses personalized educational interventions to empower trainees and improve learning outcomes. While PE has the potential to represent a paradigm shift in medical education, a theoretical foundation to guide the effective implementation of PE strategies has not yet been described. Here, the authors introduce a theoretical foundation for the implementation of PE, integrating key learning theories with the digital tools that allow them to be operationalized. Specifically, the authors describe how the master adaptive learner (MAL) model, transformative learning theory, and self-determination theory can be harnessed in conjunction with nudge strategies and audit and feedback dashboards to drive learning and meaningful behavior change. The authors also provide practical examples of these theories and tools in action by describing precision interventions already in use at one academic medical center, concretizing PE's potential in the current clinical environment. These examples illustrate how a firm theoretical grounding allows educators to most effectively tailor PE interventions to fit individual learners' needs and goals, facilitating efficient learning and ultimately improving patient and health system outcomes., (Copyright © 2023 the Association of American Medical Colleges.)

Abstract: Previous eras of assessment in medical education have been defined by how assessment is done, from knowledge exams popularized in the 1960s to the emergence of work-based assessment in the 1990s to current efforts to integrate multiple types and sources of performance data through programmatic assessment. Each of these eras was a response to why assessment was performed (e.g., assessing medical knowledge with exams; assessing communication, professionalism, and systems competencies with work-based assessment). Despite the evolution of assessment eras, current evidence highlights the graduation of trainees with foundational gaps in the ability to provide high-quality care to patients presenting with common problems, and training program leaders report they graduate trainees they would not trust to care for themselves or their loved ones. In this article, the authors argue that the next era of assessment should be defined by why assessment is done: to ensure high-quality, equitable care. Assessment should place focus on demanding graduates possess the knowledge, skills, attitudes, and adaptive expertise to meet the needs of all patients and ensuring that graduates are able to do this in an equitable fashion. The authors explore 2 patient-focused assessment approaches that could help realize the promise of this envisioned era: entrustable professional activities (EPAs) and resident sensitive quality measures (RSQMs)/TRainee Attributable and Automatable Care Evaluations in Real-time (TRACERs). These examples illustrate how the envisioned next era of assessment can leverage existing and new data to provide precision education assessment that focuses on providing formative and summative feedback to trainees in a manner that seeks to ensure their learning outcomes prepare them to ensure high-quality, equitable patient outcomes., (Copyright © 2023 the Association of American Medical Colleges.)

Purpose: The era of precision education is increasingly leveraging electronic health record (EHR) data to assess residents' clinical performance. But precision in what the EHR-based resident performance metrics are truly assessing is not fully understood. For instance, there is limited understanding of how EHR-based measures account for the influence of the team on an individual's performance-or conversely how an individual contributes to team performances. This study aims to elaborate on how the theoretical understandings of supportive and collaborative interdependence are captured in residents' EHR-based metrics., Method: Using a mixed methods study design, the authors conducted a secondary analysis of 5 existing quantitative and qualitative datasets used in previous EHR studies to investigate how aspects of interdependence shape the ways that team-based care is provided to patients., Results: Quantitative analyses of 16 EHR-based metrics found variability in faculty and resident performance (both between and within resident). Qualitative analyses revealed that faculty lack awareness of their own EHR-based performance metrics, which limits their ability to act interdependently with residents in an evidence-informed fashion. The lens of interdependence elucidates how resident practice patterns develop across residency training, shifting from supportive to collaborative interdependence over time. Joint displays merging the quantitative and qualitative analyses showed that residents are aware of variability in faculty's practice patterns and that viewing resident EHR-based measures without accounting for the interdependence of residents with faculty is problematic, particularly within the framework of precision education., Conclusions: To prepare for this new paradigm of precision education, educators need to develop and evaluate theoretically robust models that measure interdependence in EHR-based metrics, affording more nuanced interpretation of such metrics when assessing residents throughout training., (Copyright © 2024 the Association of American Medical Colleges.)

Abstract: The goal of medical education is to produce a physician workforce capable of delivering high-quality equitable care to diverse patient populations and communities. To achieve this aim amidst explosive growth in medical knowledge and increasingly complex medical care, a system of personalized and continuous learning, assessment, and feedback for trainees and practicing physicians is urgently needed. In this perspective, the authors build on prior work to advance a conceptual framework for such a system: precision education (PE).PE is a system that uses data and technology to transform lifelong learning by improving personalization, efficiency, and agency at the individual, program, and organization levels. PE "cycles" start with data inputs proactively gathered from new and existing sources, including assessments, educational activities, electronic medical records, patient care outcomes, and clinical practice patterns. Through technology-enabled analytics , insights are generated to drive precision interventions . At the individual level, such interventions include personalized just-in-time educational programming. Coaching is essential to provide feedback and increase learner participation and personalization. Outcomes are measured using assessment and evaluation of interventions at the individual, program, and organizational levels, with ongoing adjustment for repeated cycles of improvement. PE is rooted in patient, health system, and population data; promotes value-based care and health equity; and generates an adaptive learning culture.The authors suggest fundamental principles for PE, including promoting equity in structures and processes, learner agency, and integration with workflow (harmonization). Finally, the authors explore the immediate need to develop consensus-driven standards: rules of engagement between people, products, and entities that interact in these systems to ensure interoperability, data sharing, replicability, and scale of PE innovations., (Copyright © 2024 the Association of American Medical Colleges.)

Abstract: Precision education (PE) systematically leverages data and advanced analytics to inform educational interventions that, in turn, promote meaningful learner outcomes. PE does this by incorporating analytic results back into the education continuum through continuous feedback cycles. These data-informed sequences of planning, learning, assessing, and adjusting foster competence and adaptive expertise. PE cycles occur at individual (micro), program (meso), or system (macro) levels. This article focuses on program- and system-level PE.Data for PE come from a multitude of sources, including learner assessment and program evaluation. The authors describe the link between these data and the vital role evaluation plays in providing evidence of educational effectiveness. By including prior program evaluation research supporting this claim, the authors illustrate the link between training programs and patient outcomes. They also describe existing national reports providing feedback to programs and institutions, as well as 2 emerging, multiorganization program- and system-level PE efforts. The challenges encountered by those implementing PE and the continuing need to advance this work illuminate the necessity for increased cross-disciplinary collaborations and a national cross-organizational data-sharing effort.Finally, the authors propose practical approaches for funding a national initiative in PE as well as potential models for advancing the field of PE. Lessons learned from successes by others illustrate the promise of these recommendations., (Copyright © 2023 the Association of American Medical Colleges.)

Problem: Reviewing residency application narrative components is time intensive and has contributed to nearly half of applications not receiving holistic review. The authors developed a natural language processing (NLP)-based tool to automate review of applicants' narrative experience entries and predict interview invitation., Approach: Experience entries (n = 188,500) were extracted from 6,403 residency applications across 3 application cycles (2017-2019) at 1 internal medicine program, combined at the applicant level, and paired with the interview invitation decision (n = 1,224 invitations). NLP identified important words (or word pairs) with term frequency-inverse document frequency, which were used to predict interview invitation using logistic regression with L1 regularization. Terms remaining in the model were analyzed thematically. Logistic regression models were also built using structured application data and a combination of NLP and structured data. Model performance was evaluated on never-before-seen data using area under the receiver operating characteristic and precision-recall curves (AUROC, AUPRC)., Outcomes: The NLP model had an AUROC of 0.80 (vs chance decision of 0.50) and AUPRC of 0.49 (vs chance decision of 0.19), showing moderate predictive strength. Phrases indicating active leadership, research, or work in social justice and health disparities were associated with interview invitation. The model's detection of these key selection factors demonstrated face validity. Adding structured data to the model significantly improved prediction (AUROC 0.92, AUPRC 0.73), as expected given reliance on such metrics for interview invitation., Next Steps: This model represents a first step in using NLP-based artificial intelligence tools to promote holistic residency application review. The authors are assessing the practical utility of using this model to identify applicants screened out using traditional metrics. Generalizability must be determined through model retraining and evaluation at other programs. Work is ongoing to thwart model "gaming," improve prediction, and remove unwanted biases introduced during model training., (Copyright © 2023 by the Association of American Medical Colleges.)

Medical schools and residency programs are increasingly incorporating personalization of content, pathways, and assessments to align with a competency-based model. Yet, such efforts face challenges involving large amounts of data, sometimes struggling to deliver insights in a timely fashion for trainees, coaches, and programs. In this article, the authors argue that the emerging paradigm of precision medical education (PME) may ameliorate some of these challenges. However, PME lacks a widely accepted definition and a shared model of guiding principles and capacities, limiting widespread adoption. The authors propose defining PME as a systematic approach that integrates longitudinal data and analytics to drive precise educational interventions that address each individual learner's needs and goals in a continuous, timely, and cyclical fashion, ultimately improving meaningful educational, clinical, or system outcomes. Borrowing from precision medicine, they offer an adapted shared framework. In the P4 medical education framework, PME should (1) take a proactive approach to acquiring and using trainee data; (2) generate timely personalized insights through precision analytics (including artificial intelligence and decision-support tools); (3) design precision educational interventions (learning, assessment, coaching, pathways) in a participatory fashion, with trainees at the center as co-producers; and (4) ensure interventions are predictive of meaningful educational, professional, or clinical outcomes. Implementing PME will require new foundational capacities: flexible educational pathways and programs responsive to PME-guided dynamic and competency-based progression; comprehensive longitudinal data on trainees linked to educational and clinical outcomes; shared development of requisite technologies and analytics to effect educational decision-making; and a culture that embraces a precision approach, with research to gather validity evidence for this approach and development efforts targeting new skills needed by learners, coaches, and educational leaders. Anticipating pitfalls in the use of this approach will be important, as will ensuring it deepens, rather than replaces, the interaction of trainees and their coaches., (Copyright © 2023 by the Association of American Medical Colleges.)

Competency-based medical education (CBME) is an outcomes-based approach to education and assessment that focuses on what competencies trainees need to learn in order to provide effective patient care. Despite this goal of providing quality patient care, trainees rarely receive measures of their clinical performance. This is problematic because defining a trainee's learning progression requires measuring their clinical performance. Traditional clinical performance measures (CPMs) are often met with skepticism from trainees given their poor individual-level attribution. Resident-sensitive quality measures (RSQMs) are attributable to individuals, but lack the expeditiousness needed to deliver timely feedback and can be difficult to automate at scale across programs. In this eye opener, the authors present a conceptual framework for a new type of measure - TRainee Attributable & Automatable Care Evaluations in Real-time (TRACERs) - attuned to both automation and trainee attribution as the next evolutionary step in linking education to patient care. TRACERs have five defining characteristics: meaningful (for patient care and trainees), attributable (sufficiently to the trainee of interest), automatable (minimal human input once fully implemented), scalable (across electronic health records [EHRs] and training environments), and real-time (amenable to formative educational feedback loops). Ideally, TRACERs optimize all five characteristics to the greatest degree possible. TRACERs are uniquely focused on measures of clinical performance that are captured in the EHR, whether routinely collected or generated using sophisticated analytics, and are intended to complement (not replace) other sources of assessment data. TRACERs have the potential to contribute to a national system of high-density, trainee-attributable, patient-centered outcome measures., Competing Interests: The authors have no competing interests to declare., (Copyright: © 2023 The Author(s).)

Problem: Residency program directors use clerkship grades for high-stakes selection decisions despite substantial variability in grading systems and distributions. The authors apply clustering techniques from data science to identify groups of schools for which grading distributions were statistically similar in the internal medicine clerkship., Approach: Grading systems (e.g., honors/pass/fail) and distributions (i.e., percent of students in each grade tier) were tabulated for the internal medicine clerkship at U.S. MD-granting medical schools by manually reviewing Medical Student Performance Evaluations (MSPEs) in the 2019 and 2020 residency application cycles. Grading distributions were analyzed using k-means cluster analysis, with the optimal number of clusters selected using model fit indices., Outcomes: Among the 145 medical schools with available MSPE data, 64 distinct grading systems were reported. Among the 135 schools reporting a grading distribution, the median percent of students receiving the highest and lowest tier grade was 32% (range: 2%-66%) and 2% (range: 0%-91%), respectively. Four clusters was the most optimal solution (η 2 = 0.8): cluster 1 (45% [highest grade tier]-45% [middle tier]-10% [lowest tier], n = 64 [47%] schools), cluster 2 (25%-30%-45%, n = 40 [30%] schools), cluster 3 (20%-75%-5%, n = 25 [19%] schools), and cluster 4 (15%-25%-25%-25%-10%, n = 6 [4%] schools). The findings suggest internal medicine clerkship grading systems may be more comparable across institutions than previously thought., Next Steps: The authors will prospectively review reported clerkship grading approaches across additional specialties and are conducting a mixed-methods analysis, incorporating a sequential explanatory model, to interview stakeholder groups on the use of the patterns identified., (Copyright © 2022 by the Association of American Medical Colleges.)

The transition from medical student to resident is a pivotal step in the medical education continuum. For applicants, successfully obtaining a residency position is the actualization of a dream after years of training and has life-changing professional and financial implications. These high stakes contribute to a residency application and Match process in the United States that is increasingly complex and dysfunctional, and that does not effectively serve applicants, residency programs, or the public good. In July 2020, the Coalition for Physician Accountability (Coalition) formed the Undergraduate Medical Education-Graduate Medical Education Review Committee (UGRC) to critically assess the overall transition to residency and offer recommendations to solve the growing challenges in the system. In this Invited Commentary, the authors reflect on their experience as the trainee representatives on the UGRC. They emphasize the importance of trainee advocacy in medical education change efforts; reflect on opportunities, concerns, and tensions with the final UGRC recommendations (released in August 2021); discuss factors that may constrain implementation; and call for the medical education community-and the Coalition member organizations in particular-to accelerate fully implementing the UGRC recommendations. By seizing the momentum created by the UGRC, the medical education community can create a reimagined transition to residency that reshapes its approach to training a more diverse, competent, and growth-oriented physician workforce., (Copyright © 2022 by the Association of American Medical Colleges.)

The transition from undergraduate medical education (UME) to graduate medical education (GME) constitutes a complex system with important implications for learner progression and patient safety. The transition is currently dysfunctional, requiring students and residency programs to spend significant time, money, and energy on the process. Applications and interviews continue to increase despite stable match rates. Although many in the medical community acknowledge the problems with the UME-GME transition and learners have called for prompt action to address these concerns, the underlying causes are complex and have defied easy fixes. This article describes the work of the Coalition for Physician Accountability's Undergraduate Medical Education to Graduate Medical Education Review Committee (UGRC) to apply a quality improvement approach and systems thinking to explore the underlying causes of dysfunction in the UME-GME transition. The UGRC performed a root cause analysis using the 5 whys and an Ishikawa (or fishbone) diagram to deeply explore problems in the UME-GME transition. The root causes of problems identified include culture, costs and limited resources, bias, systems, lack of standards, and lack of alignment. Using the principles of systems thinking (components, connections, and purpose), the UGRC considered interactions among the root causes and developed recommendations to improve the UME-GME transition. Several of the UGRC's recommendations stemming from this work are explained. Sustained monitoring will be necessary to ensure interventions move the process forward to better serve applicants, programs, and the public good., (Copyright © 2022 by the Association of American Medical Colleges.)

Background: Residents receive infrequent feedback on their clinical reasoning (CR) documentation. While machine learning (ML) and natural language processing (NLP) have been used to assess CR documentation in standardized cases, no studies have described similar use in the clinical environment., Objective: The authors developed and validated using Kane's framework a ML model for automated assessment of CR documentation quality in residents' admission notes., Design, Participants, Main Measures: Internal medicine residents' and subspecialty fellows' admission notes at one medical center from July 2014 to March 2020 were extracted from the electronic health record. Using a validated CR documentation rubric, the authors rated 414 notes for the ML development dataset. Notes were truncated to isolate the relevant portion; an NLP software (cTAKES) extracted disease/disorder named entities and human review generated CR terms. The final model had three input variables and classified notes as demonstrating low- or high-quality CR documentation. The ML model was applied to a retrospective dataset (9591 notes) for human validation and data analysis. Reliability between human and ML ratings was assessed on 205 of these notes with Cohen's kappa. CR documentation quality by post-graduate year (PGY) was evaluated by the Mantel-Haenszel test of trend., Key Results: The top-performing logistic regression model had an area under the receiver operating characteristic curve of 0.88, a positive predictive value of 0.68, and an accuracy of 0.79. Cohen's kappa was 0.67. Of the 9591 notes, 31.1% demonstrated high-quality CR documentation; quality increased from 27.0% (PGY1) to 31.0% (PGY2) to 39.0% (PGY3) (p < .001 for trend). Validity evidence was collected in each domain of Kane's framework (scoring, generalization, extrapolation, and implications)., Conclusions: The authors developed and validated a high-performing ML model that classifies CR documentation quality in resident admission notes in the clinical environment-a novel application of ML and NLP with many potential use cases., (© 2022. The Author(s), under exclusive licence to Society of General Internal Medicine.)

Background: Residents and fellows receive little feedback on their clinical reasoning documentation. Barriers include lack of a shared mental model and variability in the reliability and validity of existing assessment tools. Of the existing tools, the IDEA assessment tool includes a robust assessment of clinical reasoning documentation focusing on four elements (interpretive summary, differential diagnosis, explanation of reasoning for lead and alternative diagnoses) but lacks descriptive anchors threatening its reliability., Objective: Our goal was to develop a valid and reliable assessment tool for clinical reasoning documentation building off the IDEA assessment tool., Design, Participants, and Main Measures: The Revised-IDEA assessment tool was developed by four clinician educators through iterative review of admission notes written by medicine residents and fellows and subsequently piloted with additional faculty to ensure response process validity. A random sample of 252 notes from July 2014 to June 2017 written by 30 trainees across several chief complaints was rated. Three raters rated 20% of the notes to demonstrate internal structure validity. A quality cut-off score was determined using Hofstee standard setting., Key Results: The Revised-IDEA assessment tool includes the same four domains as the IDEA assessment tool with more detailed descriptive prompts, new Likert scale anchors, and a score range of 0-10. Intraclass correlation was high for the notes rated by three raters, 0.84 (95% CI 0.74-0.90). Scores ≥6 were determined to demonstrate high-quality clinical reasoning documentation. Only 53% of notes (134/252) were high-quality., Conclusions: The Revised-IDEA assessment tool is reliable and easy to use for feedback on clinical reasoning documentation in resident and fellow admission notes with descriptive anchors that facilitate a shared mental model for feedback., (© 2021. Society of General Internal Medicine.)

Purpose: Residency programs face overwhelming numbers of residency applications, limiting holistic review. Artificial intelligence techniques have been proposed to address this challenge but have not been created. Here, a multidisciplinary team sought to develop and validate a machine learning (ML)-based decision support tool (DST) for residency applicant screening and review., Method: Categorical applicant data from the 2018, 2019, and 2020 residency application cycles (n = 8,243 applicants) at one large internal medicine residency program were downloaded from the Electronic Residency Application Service and linked to the outcome measure: interview invitation by human reviewers (n = 1,235 invites). An ML model using gradient boosting was designed using training data (80% of applicants) with over 60 applicant features (e.g., demographics, experiences, academic metrics). Model performance was validated on held-out data (20% of applicants). Sensitivity analysis was conducted without United States Medical Licensing Examination (USMLE) scores. An interactive DST incorporating the ML model was designed and deployed that provided applicant- and cohort-level visualizations., Results: The ML model areas under the receiver operating characteristic and precision recall curves were 0.95 and 0.76, respectively; these changed to 0.94 and 0.72, respectively, with removal of USMLE scores. Applicants' medical school information was an important driver of predictions-which had face validity based on the local selection process-but numerous predictors contributed. Program directors used the DST in the 2021 application cycle to select 20 applicants for interview that had been initially screened out during human review., Conclusions: The authors developed and validated an ML algorithm for predicting residency interview offers from numerous application elements with high performance-even when USMLE scores were removed. Model deployment in a DST highlighted its potential for screening candidates and helped quantify and mitigate biases existing in the selection process. Further work will incorporate unstructured textual data through natural language processing methods., (Copyright © 2021 by the Association of American Medical Colleges.)

Background: Calls to reform the US resident selection process are growing, given increasing competition and inefficiencies of the current system. Though numerous reforms have been proposed, they have not been comprehensively cataloged., Objective: This scoping review was conducted to characterize and categorize literature proposing systems-level reforms to the resident selection process., Methods: Following Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines, searches of Embase, MEDLINE, Scopus, and Web of Science databases were performed for references published from January 2005 to February 2020. Articles were included if they proposed reforms that were applicable or generalizable to all applicants, medical schools, or residency programs. An inductive approach to qualitative content analysis was used to generate codes and higher-order categories., Results: Of 10 407 unique references screened, 116 met our inclusion criteria. Qualitative analysis generated 34 codes that were grouped into 14 categories according to the broad stages of resident selection: application submission, application review, interviews, and the Match. The most commonly proposed reforms were implementation of an application cap (n = 28), creation of a standardized program database (n = 21), utilization of standardized letters of evaluation (n = 20), and pre-interview screening (n = 13)., Conclusions: This scoping review collated and categorized proposed reforms to the resident selection process, developing a common language and framework to facilitate national conversations and change., Competing Interests: Conflict of interest: The authors declare they have no competing interests.

Holistic review of residency applications is touted as the gold standard for selection, yet vast application numbers leave programs reliant on screening using filters such as United States Medical Licensing Examination scores that do not reliably predict resident performance and may threaten diversity. Applicants struggle to identify which programs to apply to, and devote attention to these processes throughout most of the fourth year, distracting from their clinical education. In this perspective, educators across the undergraduate and graduate medical education continuum propose new models for student-program compatibility based on design thinking sessions with stakeholders in obstetrics and gynecology education from a broad range of training environments. First, we describe a framework for applicant-program compatibility based on applicant priorities and program offerings, including clinical training, academic training, practice setting, residency culture, personal life, and professional goals. Second, a conceptual model for applicant screening based on metrics, experiences, attributes, and alignment with program priorities is presented that might facilitate holistic review. We call for design and validation of novel metrics, such as situational judgment tests for professionalism. Together, these steps could improve the transparency, efficiency and fidelity of the residency application process. The models presented can be adapted to the priorities and values of other specialties., Competing Interests: Financial Disclosure John L. Dalrymple disclosed that he is on the Association of Professors of Gynecology and Obstetrics Board of Directors. David Marzano disclosed that he has served on the ACOG Simulation Working Group, APGO Board of Directors, and the APGO Testing and Assessment Committee, and received reimbursement for travel for all three activities. The other authors did not report any potential conflicts of interest., (Copyright © 2020 by the American College of Obstetricians and Gynecologists. Published by Wolters Kluwer Health, Inc. All rights reserved.)

The 2019 novel coronavirus (COVID-19) pandemic has led to dramatic changes in the 2020 residency application cycle, including halting away rotations and delaying the application timeline. These stressors are laid on top of a resident selection process already under duress with exploding application and interview numbers-the latter likely to be exacerbated with the widespread shift to virtual interviewing. Leveraging their trainee perspective, the authors propose enforcing a cap on the number of interviews that applicants may attend through a novel interview ticket system (ITS). Specialties electing to participate in the ITS would select an evidence-based, specialty-specific interview cap. Applicants would then receive unique electronic tickets-equal in number to the cap-that would be given to participating programs at the time of an interview, when the tickets would be marked as used. The system would be self-enforcing and would ensure each interview represents genuine interest between applicant and program, while potentially increasing the number of interviews-and thus match rate-for less competitive applicants. Limitations of the ITS and alternative approaches for interview capping, including an honor code system, are also discussed. Finally, in the context of capped interview numbers, the authors emphasize the need for transparent preinterview data from programs to inform applicants and their advisors on which interviews to attend, learning from prior experiences and studies on virtual interviewing, adherence to best practices for interviewing, and careful consideration of how virtual interviews may shift inequities in the resident selection process., (Copyright © 2020 by the Association of American Medical Colleges.)

The COVID-19 pandemic has challenged front-line clinical decision-making, leading to numerous published prognostic tools. However, few models have been prospectively validated and none report implementation in practice. Here, we use 3345 retrospective and 474 prospective hospitalizations to develop and validate a parsimonious model to identify patients with favorable outcomes within 96 h of a prediction, based on real-time lab values, vital signs, and oxygen support variables. In retrospective and prospective validation, the model achieves high average precision (88.6% 95% CI: [88.4-88.7] and 90.8% [90.8-90.8]) and discrimination (95.1% [95.1-95.2] and 86.8% [86.8-86.9]) respectively. We implemented and integrated the model into the EHR, achieving a positive predictive value of 93.3% with 41% sensitivity. Preliminary results suggest clinicians are adopting these scores into their clinical workflows., Competing Interests: Competing interestsThe authors declare no competing interests., (© The Author(s) 2020.)

Background: There is emerging evidence that learners may be suboptimally prepared for the expectations of residency. In order to address these concerns, many medical schools are implementing residency preparation courses (RPCs)., Objective: We aimed to determine trainees' perceptions of their transition to residency and whether they felt that they benefited from participation in an RPC., Methods: All residents and fellows at the University of Michigan (n = 1292) received an electronic survey in July 2018 that queried respondents on demographics, whether medical school had prepared them for intern year, and whether they had participated in an RPC., Results: The response rate was 44% (563 of 1292) with even distribution across gender and postgraduate years (PGYs). Most (78%, 439 of 563) felt that medical school prepared them well for intern year. There were no differences in reported preparedness for intern year across PGY, age, gender, or specialty. Overall, 28% (156 of 563) of respondents participated in an RPC and endorsed feeling prepared for intern year, which was more than RPC non-participants (85% [133 of 156] vs 70% [306 of 439], P = .029). Participation in longer RPCs was also associated with higher perceived preparedness for residency., Conclusions: This study found that residents from multiple specialties reported greater preparedness for residency if they participated in a medical school fourth-year RPC, with greater perceptions of preparedness for longer duration RPCs, which may help to bridge the medical school to residency gap., Competing Interests: Conflict of interest: The authors report no competing interests.

Introduction: The role of medical students in catalyzing and leading curricular change in US medical schools is not well described. Here, American Medical Association student and physician leaders in the Accelerating Change in Medical Education initiative use qualitative methods to better define student leadership in curricular change. Methods: The authors developed case studies describing student leadership in curricular change efforts. Case studies were presented at a national medical education workshop; participants provided worksheet reflections and were surveyed, and responses were transcribed. Kotter's change management framework was used to categorize reported student roles in curricular change. Thematic analysis was used to identify barriers to student engagement and activators to overcome these barriers. Results: Student roles spanned all eight steps of Kotter's change management framework. Barriers to student engagement were related to faculty (e.g. view student roles narrowly), students (e.g. fear change or expect faculty-led curricula), or both (e.g. lack leadership training). Activators were: (1) recruiting collaborative faculty, staff, and students; (2) broadening student leadership roles; (3) empowering student leaders; and (4) recognizing student successes. Conclusions: By applying these activators, medical schools can build robust student-faculty partnerships that maximize collaboration, moving students beyond passive educational consumption to change agency and curricular co-creation.

Introduction: The United States Medical Licensing Examination (USMLE) Step 1 and Step 2 Clinical Knowledge (CK) are important for trainee medical knowledge assessment and licensure, medical school program assessment, and residency program applicant screening. Little is known about how USMLE performance varies between institutions. This observational study attempts to identify institutions with above-predicted USMLE performance, which may indicate educational programs successful at promoting students' medical knowledge., Methods: Self-reported institution-level data was tabulated from publicly available US News and World Report and Association of American Medical Colleges publications for 131 US allopathic medical schools from 2012-2014. Bivariate and multiple linear regression were performed. The primary outcome was institutional mean USMLE Step 1 and Step 2 CK scores outside a 95% prediction interval (≥2 standard deviations above or below predicted) based on multiple regression accounting for students' prior academic performance., Results: Eighty-nine US medical schools (54 public, 35 private) reported complete USMLE scores over the three-year study period, representing over 39,000 examinees. Institutional mean grade point average (GPA) and Medical College Admission Test score (MCAT) achieved an adjusted R2 of 72% for Step 1 (standardized βMCAT 0.7, βGPA 0.2) and 41% for Step 2 CK (standardized βMCAT 0.5, βGPA 0.3) in multiple regression. Using this regression model, 5 institutions were identified with above-predicted institutional USMLE performance, while 3 institutions had below-predicted performance., Conclusions: This exploratory study identified several US allopathic medical schools with significant above- or below-predicted USMLE performance. Although limited by self-reported data, the findings raise questions about inter-institutional USMLE performance parity, and thus, educational parity. Additional work is needed to determine the etiology and robustness of the observed performance differences., Competing Interests: We have read the journal's policy and the authors of this manuscript have the following competing interests: Dr. Burk-Rafel reports working as a research consultant for ScholarRx, a digital learning platform that includes USMLE preparation services, during the late stages of writing this manuscript. ScholarRx was not involved in this study in any way. All other authors declare no competing interests. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

This article was migrated. The article was marked as recommended. Students have traditionally held a singular role in medical education - the learner. This narrow view neglects students unique perspective and ability to shape the future of medical education. In recognizing the need for deliberate leadership skill development and networking opportunities for medical student leaders, the American Medical Association (AMA) supported the first AMA Accelerating Change in Medical Education Student-Led Conference on Leadership in Medical Education. A planning committee of 19 students from seven medical schools collaborated to develop this conference, which took place on August 4-5, 2017 at the University of Michigan, Ann Arbor. The primary goal of the conference was for students to learn about leadership skills, connect with other student leaders, feel empowered to lead change, and continue to lead from their roles as students. Attendees participated in a variety of workshops and presentations focused on developing practical leadership skills. In addition, students formed multi-institutional teams to participate on in the MedEd Impact Challenge, attempting to address issues in medical education such as leadership curriculum development, wellness, and culture change. Post-conference surveys showed an overwhelming majority of students connected with other student leaders, shared ideas, developed collaborations, and felt empowered to enact change. Looking forward, we believe that similar student-led conferences focused on broadening the medical student role would provide avenues for positive change in medical education., (Copyright: © 2019 Thomas C et al.)

Purpose: The extent of medical trainees' engagement in scholarly medical education publication is not well described. This study sought to quantify the prevalence of medical student- and resident-authored medical education publications over 15 years, a benchmark essential for understanding current and future trends in trainee scholarship., Method: Of 91 identified journals, 16 met inclusion criteria as indexed general medical education journals. Only Academic Medicine provided complete author role information, allowing identification of medical student and resident authors. The authors retrospectively compiled and analyzed citation records from Academic Medicine from 2002 to 2016, tracking trainee authorship, author position, and publication type., Results: A total of 6,280 publications were identified, of which 4,635 publications, by 16,068 authors, met inclusion criteria. Trainees were 6.0% (966/16,068) of all authors and authored 14.5% (673/4,635) of all publications. Trainee authorship rates varied by publication type: Trainees authored 33.3% (160/480) of medical humanities publications versus 6.9% (27/392) of commentaries. From 2002-2004 to 2014-2016, the proportion of authors who were trainees increased from 3.9% (73/1,853) to 7.1% (330/4,632) (P < .001 for trend). Over the same period, the percentage of trainee-authored publications increased: 9.4% (58/620) to 18.8% (225/1,199) (P < .001 for trend), driven primarily by increased trainee first authorship., Conclusions: Trainees constitute a small but growing proportion of authors and authored publications in Academic Medicine. Further work is needed to understand what trainee-, institutional-, and journal-level factors contribute to this trend, and whether similar increases in trainee authorship are occurring in other journals and fields.

New instructional technologies have been increasingly incorporated into the medical school learning environment, including lecture video recordings as a substitute for live lecture attendance. The literature presents varying conclusions regarding how this alternative experience impacts students' academic success. Previously, a multi-year study of the first-year medical histology component at the University of Michigan found that live lecture attendance was positively correlated with learning success, while lecture video use was negatively correlated. Here, three cohorts of first-year medical students (N = 439 respondents, 86.6% response rate) were surveyed in greater detail regarding lecture attendance and video usage, focusing on study behaviors that may influence histology learning outcomes. Students who reported always attending lectures or viewing lecture videos had higher average histology scores than students who employed an inconsistent strategy (i.e., mixing live attendance and video lectures). Several behaviors were negatively associated with histology performance. Students who engaged in "non-lecture activities" (e.g., social media use), students who reported being interrupted while watching the lecture video, or feeling sleepy/losing focus had lower scores than their counterparts not engaging in these behaviors. This study suggests that interruptions and distractions during medical learning activities-whether live or recorded-can have an important impact on learning outcomes. Anat Sci Educ 11: 366-376. © 2017 American Association of Anatomists., (© 2017 American Association of Anatomists.)

Purpose: To determine medical students' study behaviors when preparing for the United States Medical Licensing Examination (USMLE) Step 1, and how these behaviors are associated with Step 1 scores when controlling for likely covariates., Method: The authors distributed a study-behaviors survey in 2014 and 2015 at their institution to two cohorts of medical students who had recently taken Step 1. Demographic and academic data were linked to responses. Descriptive statistics, bivariate correlations, and multiple linear regression analyses were performed., Results: Of 332 medical students, 274 (82.5%) participated. Most students (n = 211; 77.0%) began studying for Step 1 during their preclinical curriculum, increasing their intensity during a protected study period during which they averaged 11.0 hours studying per day (standard deviation [SD] 2.1) over a period of 35.3 days (SD 6.2). Students used numerous third-party resources, including reading an exam-specific 700-page review book on average 2.1 times (SD 0.8) and completing an average of 3,597 practice multiple-choice questions (SD 1,611). Initiating study prior to the designated study period, increased review book usage, and attempting more practice questions were all associated with higher Step 1 scores, even when controlling for Medical College Admission Test scores, preclinical exam performance, and self-identified score goal (adjusted R = 0.56, P < .001)., Conclusions: Medical students at one public institution engaged in a self-directed, "parallel" Step 1 curriculum using third-party study resources. Several study behaviors were associated with improved USMLE Step 1 performance, informing both institutional- and student-directed preparation for this high-stakes exam.

Purpose: Scholarly concentration programs-also known as scholarly projects, pathways, tracks, or pursuits-are increasingly common in U.S. medical schools. However, systematic, data-driven program development methods have not been described., Method: The authors examined scholarly concentration programs at U.S. medical schools that U.S. News & World Report ranked as top 25 for research or primary care (n = 43 institutions), coding concentrations and mission statements. Subsequently, the authors conducted a targeted needs assessment via a student-led, institution-wide survey, eliciting learners' preferences for 10 "Pathways" (i.e., concentrations) and 30 "Topics" (i.e., potential content) augmenting core curricula at their institution. Exploratory factor analysis (EFA) and a capacity optimization algorithm characterized best institutional options for learner-focused Pathway development., Results: The authors identified scholarly concentration programs at 32 of 43 medical schools (74%), comprising 199 distinct concentrations (mean concentrations per program: 6.2, mode: 5, range: 1-16). Thematic analysis identified 10 content domains; most common were "Global/Public Health" (30 institutions; 94%) and "Clinical/Translational Research" (26 institutions; 81%). The institutional needs assessment (n = 468 medical students; response rate 60% overall, 97% among first-year students) demonstrated myriad student preferences for Pathways and Topics. EFA of Topic preferences identified eight factors, systematically related to Pathway preferences, informing content development. Capacity modeling indicated that offering six Pathways could guarantee 95% of first-year students (162/171) their first- or second-choice Pathway., Conclusions: This study demonstrates a generalizable, data-driven approach to scholarly concentration program development that reflects student preferences and institutional strengths, while optimizing program diversity within capacity constraints.

Importance: Our study indicates a prototype blood-based variant Creutzfeldt-Jakob disease (vCJD) assay has sufficient sensitivity and specificity to justify a large study comparing vCJD prevalence in the United Kingdom with a bovine spongiform encephalopathy-unexposed population. In a clinical diagnostic capacity, the assay's likelihood ratios dramatically change an individual's pretest disease odds to posttest probabilities and can confirm vCJD infection., Objectives: To determine the diagnostic accuracy of a prototype blood test for vCJD and hence its suitability for clinical use and for screening prion-exposed populations., Design, Setting, and Participants: Retrospective, cross-sectional diagnostic study of blood samples from national blood collection and prion disease centers in the United States and United Kingdom. Anonymized samples were representative of the US blood donor population (n = 5000), healthy UK donors (n = 200), patients with nonprion neurodegenerative diseases (n = 352), patients in whom a prion disease diagnosis was likely (n = 105), and patients with confirmed vCJD (n = 10)., Main Outcome and Measure: Presence of vCJD infection determined by a prototype test (now in clinical diagnostic use) that captures, enriches, and detects disease-associated prion protein from whole blood using stainless steel powder., Results: The assay's specificity among the presumed negative American donor samples was 100% (95% CI, 99.93%-100%) and was confirmed in a healthy UK cohort (100% specificity; 95% CI, 98.2%-100%). Of potentially cross-reactive blood samples from patients with nonprion neurodegenerative diseases, no samples tested positive (100% specificity; 95% CI, 98.9%-100%). Among National Prion Clinic referrals in whom a prion disease diagnosis was likely, 2 patients with sporadic CJD tested positive (98.1% specificity; 95% CI, 93.3%-99.8%). Finally, we reconfirmed but could not refine our previous sensitivity estimate in a small blind panel of samples from unaffected individuals and patients with vCJD (70% sensitivity; 95% CI, 34.8%-93.3%)., Conclusions and Relevance: In conjunction with the assay's established high sensitivity (71.4%; 95% CI, 47.8%-88.7%), the extremely high specificity supports using the assay to screen for vCJD infection in prion-exposed populations. Additionally, the lack of cross-reactivity and false positives in a range of nonprion neurodegenerative diseases supports the use of the assay in patient diagnosis.

Self-assembled monolayers (SAMs) bearing pendant carbohydrate functionality are frequently employed to tailor glycan-specific bioactivity onto gold substrates. The resulting glycoSAMs are valuable for interrogating glycan-mediated biological interactions via surface analytical techniques, microarrays, and label-free biosensors. GlycoSAM composition can be readily modified during assembly by using mixed solutions containing thiolated species, including carbohydrates, oligo(ethylene glycol) (OEG), and other inert moieties. This intrinsic tunability of the self-assembled system is frequently used to optimize bioavailability and antibiofouling properties of the resulting SAM. However, until now, our nanoscale understanding of the behavior of these mixed glycoSAMs has lacked detail. In this study, we examined the time-dependent clustering of mixed sugar + OEG glycoSAMs on ultraflat gold substrates. Composition and surface morphologic changes in the monolayers were analyzed by X-ray photoelectron spectroscopy (XPS) and atomic force microscopy (AFM), respectively. We provide evidence that the observed clustering is consistent with a phase separation process in which surface-bound glycans self-associate to form dense glycoclusters within the monolayer. These observations have significant implications for the construction of mixed glycoSAMs for use in biosensing and glycomics applications.

Covalent modification of surfaces with carbohydrates (glycans) is a prerequisite for a variety of glycomics-based biomedical applications, including functional biomaterials, glycoarrays, and glycan-based biosensors. The chemistry of glycan immobilization plays an essential role in the bioavailability and function of the surface bound carbohydrate moiety. However, the scarcity of analytical methods to characterize carbohydrate-modified surfaces complicates efforts to optimize glycan surface chemistries for specific applications. Time-of-Flight Secondary Ion Mass Spectrometry (ToF-SIMS) is a surface sensitive technique suited for probing molecular composition at the biomaterial interface. Expanding ToF-SIMS analysis to interrogate carbohydrate-modified materials would increase our understanding of glycan surface chemistries and advance novel tools in the nascent field of glycomics. In this study, a printed glycan microarray surface was fabricated and subsequently characterized by ToF-SIMS imaging analysis. A multivariate technique based on principal component analysis (PCA) was used to analyze the ToF-SIMS dataset and reconstruct ToF-SIMS images of functionalized surfaces. These images reveal chemical species related to the immobilized glycan, underlying glycan-reactive chemistries, gold substrates, and outside contaminants. Printed glycoarray elements (spots) were also interrogated to resolve the spatial distribution and spot homogeneity of immobilized glycan. The bioavailability of the surface-bound glycan was validated using a specific carbohydrate-binding protein (lectin) as characterized by Surface Plasmon Resonance Imaging (SPRi). Our results demonstrate that ToF-SIMS is capable of characterizing chemical features of carbohydrate-modified surfaces and, when complemented with SPRi, can play an enabling role in optimizing glycan microarray fabrication and performance.