Search

Introduction: Self-regulated learning is a cyclical process of forethought, performance, and self-reflection that has been used as an assessment tool in medical education. No prior studies have evaluated SRL processes for answering multiple-choice questions (MCQs) and most evaluated one or two iterations of a non-MCQ task. SRL assessment during MCQs may elucidate reasons why learners are successful or not on these questions that are encountered repeatedly during medical education., Methods: Internal medicine clerkship students at three institutions participated in a SRL microanalytic protocol that targeted strategic planning, metacognitive monitoring, causal attributions, and adaptive inferences across seven MCQs. Responses were transcribed and coded according to previously published methods for microanalytic protocols., Results: Forty-four students participated. In the forethought phase, students commonly endorsed prioritizing relevant features as their diagnostic strategy (n = 20, 45%) but few mentioned higher-order diagnostic reasoning processes such as integrating clinical information (n = 5, 11%) or comparing/contrasting diagnoses (n = 0, 0%). However, in the performance phase, students' metacognitive processes included high frequencies of integration (n = 38, 86%) and comparing/contrasting (n = 24, 55%). In the self-reflection phase, 93% (n = 41) of students faulted their management reasoning and 84% (n = 37) made negative references to their abilities. Less than 10% (n = 4) of students indicated that they would adapt their diagnostic reasoning process for these questions., Discussion: This study describes in detail student self-regulatory processes during MCQs. We found that students engaged in higher-order diagnostic reasoning processes but were not explicit about it and seldom reflected critically on these processes after selecting an incorrect answer. Self-reflections focused almost exclusively on management reasoning and negative references to abilities which may decrease self-efficacy. Encouraging students to identify and evaluate diagnostic reasoning processes and make attributions to controllable factors may improve performance., Competing Interests: The authors have no competing interests to declare., (Copyright: © 2023 The Author(s).)

To be safe and effective practitioners and learners, medical professionals must be able to accurately assess their own performance to know when they need additional help. This study explored the metacognitive judgments of 157 first-year medical students; in particular, the study examined students' self-assessments or calibration as they engaged in a virtual-patient simulation targeting clinical reasoning practices. Examining two key subtasks of a patient encounter, history (Hx) and physical exam (PE), the authors assessed the level of variation in students' behavioral performance (i.e., effectiveness and efficiency) and judgments of performance (i.e., calibration bias and accuracy) across the two subtasks. Paired t tests revealed that the Hx subtask was deemed to be more challenging than the PE subtask when viewed in terms of both actual and perceived performance. In addition to students performing worse on the Hx subtask than PE, they also perceived that they performed less well for Hx. Interestingly, across both subtasks, the majority of participants overestimated their performance (98% of participants for Hx and 95% for PE). Correlation analyses revealed that the participants' overall level of accuracy in metacognitive judgments was moderately stable across the Hx and PE subtasks. Taken together, findings underscore the importance of assessing medical students' metacognitive judgments at different points during a clinical encounter.

Problem: Medical schools face the challenge of developing efficacious resources to promote well-being and foster resilience in students. The authors developed, implemented, and evaluated a shame resilience seminar for second-year clerkship medical students., Approach: In February 2018, the authors conducted a 2.5-hour seminar (part of a longitudinal series) about shame, a common and potentially damaging emotion. The seminar consisted of a large-group session to introduce the psychology of shame, during which speakers shared their personal experiences with the emotion. Next, a small-group session allowed students to discuss their reactions to the large-group content in a safe and familiar environment. Before the seminar, faculty development was provided to small-group leaders (upper-level medical students and faculty) to increase their comfort leading discussions about shame. Students completed a pre/post retrospective survey immediately following the seminar., Outcomes: The authors found statistically significant increases in students' confidence in identifying shame and differentiating it from guilt; in their attitudes regarding the importance of identifying shame reactions in themselves and others; and in their willingness to reach out to others during a shame reaction. Surveys of group leaders revealed no reports of significant student distress during or after the seminar., Next Steps: This seminar represents a reproducible means of promoting shame resilience in medical students. The speakers' personal shame experiences and the safety of the small groups for discussions about shame were central to the seminar's apparent success. Next steps include developing an empirically derived, longitudinal shame resilience curriculum spanning the medical school years.

Background: To support evidence-informed education, health professions education (HPE) stakeholders encourage the creation and use of knowledge syntheses or reviews. However, it is unclear if these knowledge syntheses are ready for translation into educational practice. Without understanding the readiness, defined by three criteria-quality, accessibility and relevance-we risk translating weak evidence into practice and/or providing information that is not useful to educators., Methods: A librarian searched Web of Science for knowledge syntheses, specifically Best Evidence in Medical Education (BEME) Guides. This meta-synthesis focuses on BEME Guides because of their explicit goal to inform educational practice and policy. Two authors extracted data from all Guides, guided by the 25-item STructured apprOach to the Reporting In healthcare education of Evidence Synthesis (STORIES)., Results: Forty-two Guides published in Medical Teacher between 1999 and 2017 were analyzed. No Guide met all STORIES criteria, but all included structured summaries and most described their literature search (n = 39) and study inclusion/exclusion (n = 40) procedures. Eleven Guides reported the presence of theory and/or educational principles, and eight consulted with external subject matter experts. Accessibility to each Guide's full-text and supplemental materials was variable., Discussion: For a subset of HPE knowledge syntheses, BEME Guides, this meta-synthesis identifies factors that support readiness and indicates potential areas of improvement, such as consistent access to Guides and inclusion of external subject matter experts on the review team. This analysis is useful for understanding the current readiness of HPE knowledge syntheses and informing future reviews to evolve so they can catalyze translation of evidence into educational practice.

Helping medical educators obtain and use assessment data to assist medical students, residents, and physicians in reducing diagnostic errors and other forms of ineffective clinical practice is of critical importance. Self-Regulated Learning-Microanalytic Assessment and Training is an assessment-to-intervention framework designed to address this need by generating data about trainees' strategic processes (e.g., focusing on clinical task procedures), regulatory processes (e.g., planning how to do a task), and motivational processes (e.g., increasing confidence for performing a task) as they perform clinical activities. In this article, the authors review several studies that have used an innovative assessment approach, called self-regulated learning (SRL) microanalysis, to generate data about how trainees regulate their thinking and actions during clinical reasoning tasks. Across the studies, initial findings revealed that medical students often do not exhibit strategic thinking and action during clinical reasoning practice tasks even though some regulatory processes (e.g., planning) are predictive of important medical education outcomes. Further, trainees' motivation beliefs, strategic thinking, and self-evaluative judgments tend to shift rapidly during clinical skills practice and may also vary across different parts of a patient encounter. Collectively, these findings underscore the value of dynamically assessing trainees' SRL as they complete clinical tasks. The findings also set the stage for exploring how medical educators can best use SRL microanalytic assessment data to guide remedial practices and the provision of feedback to trainees. Implications and future research directions for connecting assessments to intervention in medical education are discussed.

Purpose: To investigate the association between poor performance on National Board of Medical Examiners clinical subject examinations across six core clerkships and performance on the United States Medical Licensing Examination Step 3 examination., Method: In 2012, the authors studied matriculants from the Uniformed Services University of the Health Sciences with available Step 3 scores and subject exam scores on all six clerkships (Classes of 2007-2011, N = 654). Poor performance on subject exams was defined as scoring one standard deviation (SD) or more below the mean using the national norms of the corresponding test year. The association between poor performance on the subject exams and the probability of passing or failing Step 3 was tested using contingency table analyses and logistic regression modeling., Results: Students performing poorly on one subject exam were significantly more likely to fail Step 3 (OR 14.23 [95% CI 1.7-119.3]) compared with students with no subject exam scores that were 1 SD below the mean. Poor performance on more than one subject exam further increased the chances of failing (OR 33.41 [95% CI 4.4-254.2]). This latter group represented 27% of the entire cohort, yet contained 70% of the students who failed Step 3., Conclusions: These findings suggest that individual schools could benefit from a review of subject exam performance to develop and validate their own criteria for identifying students at risk for failing Step 3.

Objectives: The primary objectives of this study were to examine the regulatory processes of medical students as they completed a diagnostic reasoning task and to examine whether the strategic quality of these regulatory processes were related to short-term and longer-term medical education outcomes., Methods: A self-regulated learning (SRL) microanalytic assessment was administered to 71 second-year medical students while they read a clinical case and worked to formulate the most probable diagnosis. Verbal responses to open-ended questions targeting forethought and performance phase processes of a cyclical model of SRL were recorded verbatim and subsequently coded using a framework from prior research. Descriptive statistics and hierarchical linear regression models were used to examine the relationships between the SRL processes and several outcomes., Results: Most participants (90%) reported focusing on specific diagnostic reasoning strategies during the task (metacognitive monitoring), but only about one-third of students referenced these strategies (e.g. identifying symptoms, integration) in relation to their task goals and plans for completing the task. After accounting for prior undergraduate achievement and verbal reasoning ability, strategic planning explained significant additional variance in course grade (ΔR(2 ) = 0.15, p < 0.01), second-year grade point average (ΔR(2) = 0.14, p < 0.01), United States Medical Licensing Examination Step 1 score (ΔR(2) = 0.08, p < 0.05) and National Board of Medical Examiner subject examination score in internal medicine (ΔR(2) = 0.10, p < 0.05)., Conclusions: These findings suggest that most students in the formative stages of learning diagnostic reasoning skills are aware of and think about at least one key diagnostic reasoning process or strategy while solving a clinical case, but a substantially smaller percentage set goals or develop plans that incorporate such strategies. Given that students who developed more strategic plans achieved better outcomes, the potential importance of forethought regulatory processes is underscored., (Published 2014. This article is a U.S. Government work and is in the public domain in the USA. Medical Education published by John Wiley & Sons Ltd.)

Background: Recently, there has been a surge in the use of objective structured clinical examinations (OSCEs) at medical schools around the world, and with this growth has come the concomitant need to validate such assessments., Purposes: The current study examined the associations between student performance on several school-level clinical skills and knowledge assessments, including two OSCEs, the National Board of Medical Examiners® (NBME) Subject Examinations, and the United States Medical Licensing Examination® (USMLE) Step 2 Clinical Skills (CS) and Step 3 assessments., Methods: The sample consisted of 806 medical students from the Uniformed Services University of the Health Sciences. We conducted Pearson correlation analysis as well as stepwise multiple linear regression modeling to examine the strength of associations between students' performance on 2nd- and 3rd-year OSCEs and their two Step 2 CS component scores and Step 3 scores., Results: Positive associations were found between the OSCE variables and the USMLE scores; in particular, student performance on both the 2nd- and 3rd-year OSCEs was more strongly associated with the two Step 2 CS component scores than with Step 3 scores., Conclusions: These findings, although preliminary, provide some predictive validity evidence for the use of OSCEs in determining readiness of medical students for clinical practice and licensure.

Purpose: Determine whether the National Board of Medical Examiners (NBME) Subject Examination performance from six clerkships correlated with United States Medical Licensing Examination (USMLE) Steps 1 and 2 Clinical Knowledge (CK) Examination scores. Also, examine correlations between medical students' preclinical and clinical year mean cumulative grade point average (GPA), subject exam, and USMLE performance., Method: The sample consisted of 507 students from the 2008-2010 graduating classes from the authors' medical school. Pearson correlations followed by stepwise linear regressions were used to investigate variance in USMLE Steps 1 and 2 CK scores explained by subject exam scores and GPA., Results: Data from 484 (95.5%) students were included. USMLE Steps 1 and 2 CK scores had moderate-to-large positive correlations with all subject exam scores and with both GPA variables. Correlations between composite subject exam scores and USMLE Steps 1 and 2 CK exams were 0.69 and 0.77, respectively. Regression analysis demonstrated that subject exams and GPA accounted for substantial variance in Steps 1 and 2 CK exam scores (62% and 61%); when entered into the regression model first, primary care clerkship subject examination scores accounted for most of this variance., Conclusions: The moderate-to-large correlations between subject exam performance and USMLE scores provide reassurance that subject exam scores are associated with USMLE performance. Furthermore, the considerable variance in USMLE scores accounted for by primary care NBME scores may be due to primary care topics being reinforced through all clerkships and comprising a significant portion of the USMLE examinations, particularly Step 2 CK.

Purpose: Practicing physicians have a societal obligation to maintain their competence. Unfortunately, the self-regulated learning skills likely required for lifelong learning are not explicitly addressed in most medical schools. The authors examined how medical students' perceptions of the learning environment relate to their self-regulated learning behaviors. They also explored how students' perceptions and behaviors correlate with performance and change across medical school., Method: The authors collected survey data from 304 students at different phases of medical school training. The survey items assessed students' perceptions of the learning environment, as well as their metacognition, procrastination, and avoidance-of-help-seeking behaviors. The authors operationalized achievement as cumulative medical school grade point average (GPA) and, for third- and fourth-year students, collected clerkship outcomes., Results: Students' perceptions of the learning environment were associated with their metacognition, procrastination, and help-avoidance behaviors. These behaviors were also related to academic outcomes. Specifically, avoidance of help seeking was negatively correlated with cumulative medical school GPA (r=-0.23, P<.01) as well as exam (r=-0.22, P<.05) and clinical performance (r=-0.34, P<.01) in the internal medical clerkship; these help-avoidance behaviors were also positively correlated with students' presentation at a grade adjudication committee (r=0.20, P<.05). Additionally, students' perceptions of the learning environment varied as a function of their phase of training., Conclusions: Medical students' perceptions of the learning environment are related, in predictable ways, to their use of self-regulated learning behaviors; these perceptions seem to change across medical school.

Objective: The purpose of the study was to investigate the relationship between authenticity (how close to actual practice) of preclerkship instructional formats and preclerkship and clerkship outcome measures. A secondary purpose was to investigate the effect of student's small-group assignment during preclerkship period on instructional formats and preclerkship and clerkship outcome measurements., Methods: A prospective, randomized, crossover study was carried out with preclerkship students taking a clinical reasoning course. Students were randomized to small groups and exposed to three formats of differing instructional authenticity across three subject areas. Three student cohorts were taught using one instructional format per subject area with each cohort receiving a different instructional format for each of the three areas. Outcome measures at the end of the preclerkship year and the clerkship year were selected to determine the effect of each instructional format. Hierarchical linear modeling was performed to assess impact of format on outcomes and to assess potential group effect on outcomes., Results: Increasingly authentic instructional formats did not significantly improve clinical reasoning performance. Small-group assignment did not significantly contribute to the outcomes providing evidence that teacher training was successful., Conclusions: Increasing authenticity of instructional formats does not appear to significantly improve clinical reasoning performance. Faculty can be successfully trained to teach consistently in a clinical reasoning course. Medical educators should balance increasing authenticity with factors such as cognitive load and learner experience as well as engaging in frame-of-reference training to minimize group effects with designing new instructional formats.