Search

Clinical skills instruction is a standard part of medical school curricula, but how institutions address learners who struggle in this area is less clear. Although recommendations for the remediation of clinical skills at an institutional level have been published, how these recommendations are being implemented on a national scale is unknown. In this descriptive study, we characterize current clinical skills remediation practices at US medical schools and US-accredited Caribbean medical schools.We conducted a cross-sectional survey of medical educators who work with struggling students. From March 24, 2020 to April 9, 2020, the Directors of Clinical Skills Remediation Working Group conducted an e-mail survey incorporating four aspects of remediation program design and function: identification, assessment, active remediation, and ongoing evaluation.In total, 92 individuals representing 45 institutions provided descriptive information about their respective remediation programs. The majority of respondents have a formal process of identifying (75%) and assessing (86%) students who are identified as struggling with clinical skills, but lack a standardized method of categorizing deficits. Fewer institutions have a standardized approach to active remediation and ongoing evaluation of struggling learners. Fifty-two percent of institutions provide training to faculty involved in the remediation process.Although most institutions are able to identify struggling students, they lack a standardized approach to intervene. Remediation effectiveness is limited by a lack of student buy-in and institutional time, expertise, and resources. These findings highlight the need for more formalized structure and standardization in remediation program design and implementation.

Curricular development and modification involve first identifying a problem and then performing a needs assessment, which can guide the design of curricular components. Pedagogical changes, coupled with reductions in curricular time for gross anatomy, pose challenges and impose restrictions within medical school curricula. In order to make anatomy education effective and efficient, it is important to determine the anatomy considered essential for medical education through a targeted needs assessment. In this study, 50 adult primary care resident physicians in Family Medicine (FM) and Internal Medicine (IM) were surveyed to assess the importance of 907 anatomical structures, or groups of structures, across all anatomical regions from a curated list based on the boldface terms in four primary anatomy texts. There were no statistically significant differences in the ratings of structures between the two groups for any anatomical region. In total, 17.0% of structures, or groups of structures, were classified as essential, 58.0% as more important, 24.4% as less important, and 0.7% as not important. FM residents rated tissues classified as skeleton, nerves, fasciae, anatomical spaces, blood vessels, lymphatics and surface anatomy (P 0.0001) significantly higher than IM residents, but there were no differences in the rating of muscles or organs (P 0.0056). It was notable that 100.0% of cranial nerves were classified as essential, and 94.5% of surface anatomy structures were classified as essential or more important. It is proposed that results of this study can serve to inform curricular development and revision.

BACKGROUND: Many medical schools offer a culminating internship readiness experience. Curricula focus on particular knowledge and skills critical to internship, such as answering urgent nursing pages. Studies have shown student performance improvement with mock paging education programs, but the role of feedback versus self-regulated practice has not been studied. DESIGN AND METHODS: The interprofessional mock paging program included 156 medical students enrolled in a 4(th)-year internship readiness course and 44 master’s level direct entry nursing students. Medical students were randomized to receive verbal feedback immediately after each of the three phone calls (intervention group) or delayed written feedback (control group) after the third phone call only. Specialty-specific case scenarios were developed and a single checklist for all scenarios was developed using the communication tool ISBAR. Medical students and nursing students had separate training sessions before the pages commenced. The nursing students administered the phone calls and evaluated the medical students by ISBAR checklist. An interrater reliability measure was obtained with physician observation of a selection of phone calls. RESULTS: After adjusting for the case effects (different case scenarios for different specialties), students showed no statistically significant differences on checklist scores for case 1 (first case, F = 1.491, df = 1, p = .224), but did show statistically significant differences on checklist scores for case 3 (final case, F = 12.238, df = 1, p = .001). Strong interrater reliability was found between the faculty physician and observed nursing students (ICC = .89). CONCLUSIONS: Immediate feedback significantly improves student checklist scores with a mock paging program. This finding suggests that coaching with feedback may have advantages above self-regulated learning.

BACKGROUND: Use of EPA-based entrustment-supervision ratings to determine a learner’s readiness to assume patient care responsibilities is expanding. OBJECTIVE: In this study, we investigate the correlation between narrative comments and supervision ratings assigned during ad hoc assessments of medical students’ performance of EPA tasks. DESIGN: Data from assessments completed for students enrolled in the clerkship phase over 2 academic years were used to extract a stratified random sample of 100 narrative comments for review by an expert panel. PARTICIPANTS: A review panel, comprised of faculty with specific expertise related to their roles within the EPA program, provided a “gold standard” supervision rating using the comments provided by the original assessor. MAIN MEASURES: Interrater reliability (IRR) between members of review panel and correlation coefficients (CC) between expert ratings and supervision ratings from original assessors. KEY RESULTS: IRR among members of the expert panel ranged from .536 for comments associated with focused history taking to .833 for complete physical exam. CC (Kendall’s correlation coefficient W) between panel members’ assignment of supervision ratings and the ratings provided by the original assessors for history taking, physical examination, and oral presentation comments were .668, .697, and .735 respectively. The supervision ratings of the expert panel had the highest degree of correlation with ratings provided during assessments done by master assessors, faculty trained to assess students across clinical contexts. Correlation between supervision ratings provided with the narrative comments at the time of observation and supervision ratings assigned by the expert panel differed by clinical discipline, perhaps reflecting the value placed on, and perhaps the comfort level with, assessment of the task in a given specialty. CONCLUSIONS: To realize the full educational and catalytic effect of EPA assessments, assessors must apply established performance expectations and provide high-quality narrative comments aligned with the criteria.

Problem: Development of a novel, competency-based program of assessment requires creation of a plan to measure the processes that enable successful implementation. The principles of implementation science outline the importance of considering key drivers that support and sustain transformative change within an educational program. The introduction of Entrustable Professional Activities (EPAs) as a framework for assessment has underscored the need to create a structured plan to prepare assessors to engage in a new paradigm of assessment. Although approaches to rater training for workplace-based assessments have been described, specific strategies to prepare assessors to apply standards related to the level of supervision a student needs have not been documented. Intervention: We describe our systematic approach to prepare assessors, faculty and postgraduate trainees, to complete EPA assessments for medical students during the clerkship phase of our curriculum. This institution-wide program is designed to build assessors' skills in direct observation of learners during authentic patient encounters. Assessors apply new knowledge and practice skills in using established performance expectations to determine the level of supervision a learner needs to perform clinical tasks. Assessors also learn to provide feedback and narrative comments to coach students and promote their ongoing clinical development. Data visualizations for assessors facilitate reinforcement of the tenets learned during training. Collaborative learning and peer feedback during faculty development sessions promote the formation of a community of practice among assessors. Context: Faculty development for assessors was implemented in advance of implementation of the EPA program. Assessors in the program include residents/fellows who work closely with students, faculty with discipline-specific expertise and a group of experienced clinicians who were selected to serve as experts in competency-based EPA assessments, the Master Assessors. Training focused on creating a shared understanding about the application of criteria used to evaluate student performance. EPA assessments based on the AAMC's Core Entrustable Professional Activities for Entering Residency, were completed in nine core clerkships. EPA assessments included a supervision rating based on a modified scale for use in undergraduate medical education. Impact: Data from EPA assessments completed during the first year of the program were analyzed to evaluate the effectiveness of the faculty development activities implemented to prepare assessors to consistently apply standards for assessment. A systematic approach to training and attention to critical drivers that enabled institution-wide implementation, led to consistency in the supervision rating for students' first EPA assessment completed by any type of assessor, ratings by assessors done within a specific clinical context, and ratings assigned by a group of specific assessors across clinical settings. Lessons learned: A systematic approach to faculty development with a willingness to be flexible and reach potential participants using existing infrastructure, can facilitate assessors' engagement in a new culture of assessment. Interaction among participants during training sessions not only promotes learning but also contributes to community building. A leadership group responsible to oversee faculty development can ensure that the needs of stakeholders are addressed and that a change in assessment culture is sustained.

Clinical reasoning encompasses the process of data collection, synthesis, and interpretation to generate a working diagnosis and make management decisions. Situated cognition theory suggests that knowledge is relative to contextual factors, and clinical reasoning in urgent situations is framed by pressure of consequential, time-sensitive decision-making for diagnosis and management. These unique aspects of urgent clinical care may limit the effectiveness of traditional tools to assess, teach, and remediate clinical reasoning.Using two validated frameworks, a multidisciplinary group of clinicians trained to remediate clinical reasoning and with experience in urgent clinical care encounters designed the novel Rapid Evaluation Assessment of Clinical Reasoning Tool (REACT). REACT is a behaviorally anchored assessment tool scoring five domains used to provide formative feedback to learners evaluating patients during urgent clinical situations. A pilot study was performed to assess fourth-year medical students during simulated urgent clinical scenarios. Learners were scored using REACT by a separate, multidisciplinary group of clinician educators with no additional training in the clinical reasoning process. REACT scores were analyzed for internal consistency across raters and observations.Overall internal consistency for the 41 patient simulations as measured by Cronbach's alpha was 0.86. A weighted kappa statistic was used to assess the overall score inter-rater reliability. Moderate reliability was observed at 0.56.To our knowledge, REACT is the first tool designed specifically for formative assessment of a learner's clinical reasoning performance during simulated urgent clinical situations. With evidence of reliability and content validity, this tool guides feedback to learners during high-risk urgent clinical scenarios, with the goal of reducing diagnostic and management errors to limit patient harm.

Entrustable professional activities (EPAs) have been introduced as a framework for teaching and assessment in competency-based educational programs. With growing use, has come a call to examine the validity of EPA assessments. We sought to explore the correlation of EPA assessments with other clinical performance measures to support use of supervision ratings in decisions about medical students' curricular progression.Spearman rank coefficients were used to determine correlation of supervision ratings from EPA assessments with scores on clerkship evaluations and performance on an end-of-clerkship-year Objective Structured Clinical Examination (CPX).Both overall clinical evaluation items score (rho 0.40;Correlation of EPA supervision ratings with commonly used clinical performance measures offers support for their use in undergraduate medical education. Data supporting the validity of EPA assessments promotes stakeholders' acceptance of their use in summative decisions about students' readiness for increased patient care responsibility.

Introduction New medical interns face a steep learning curve as they must manage complex medical scenarios, many of which they have only seen before in a classroom setting. To ameliorate these challenges, medical schools are increasingly including courses designed to address the transition from student to doctor. One of the biggest challenges for new interns is learning to triage and manage nursing pages, so we designed a mock paging program incorporated within our fourth-year transitions course. Methods We developed a database of clinical scenarios to occur via telephone between a nurse and a medicine intern. Throughout the 2-week course, these cases were administered to 40 fourth-year medical students by Master's level nursing students and nurse evaluators. The nurses used checklists to evaluate medical student management and communication, and at the end of the phone encounter students received immediate feedback. We used an observational prospective design, using a within subjects method with repeated measures. Results Data from a total of 216 phone calls were analyzed for 36 students. No statistically significant improvement of checklist scores was observed. Substantial interrater reliability was observed for the four observed cases with a Fleiss-Kappa of .76. Student comments indicated the activity was helpful for preparing them to answer pages. Discussion Our paging program offered students the chance to simulate being on call, as well as the opportunity to receive immediate feedback. It did not show improvement in checklists across time. Limitations included a small sample size and few common variables across the cases.

CONTEXT: Situativity theory posits that learning and the development of clinical reasoning skills are grounded in context. In case-based teaching, this context comes from recreating the clinical environment, through emulation, as with manikins, or description. In this study, we sought to understand the difference in student clinical reasoning abilities after facilitated patient case scenarios with or without a manikin. METHODS: Fourth-year medical students in an internship readiness course were randomized into patient case scenarios without manikin (control group) and with manikin (intervention group) for a chest pain session. The control and intervention groups had identical student-led case progression and faculty debriefing objectives. Clinical reasoning skills were assessed after the session using a 64-question script concordance test (SCT). The test was developed and piloted prior to administration. Hospitalist and emergency medicine faculty responses on the test items served as the expert standard for scoring. RESULTS: Ninety-six students were randomized to case-based sessions with (n = 48) or without (n = 48) manikin. Ninety students completed the SCT (with manikin n = 45, without manikin n = 45). A statistically significant mean difference on test performance between the two groups was found (t = 3.059, df = 88, p = .003), with the manikin group achieving higher SCT scores. CONCLUSION: Use of a manikin in simulated patient case discussion significantly improves students’ clinical reasoning skills, as measured by SCT. These results suggest that using a manikin to simulate a patient scenario situates learning, thereby enhancing skill development. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1007/s40670-019-00904-0) contains supplementary material, which is available to authorized users.

All accrediting organizations for medical education in the US require teaching hospitals to ensure the learning environment fosters professionalism behaviors of trainees and faculty. This study analyzes the learning environment of professionalism from the perspective of residents. An on-line anonymous survey that explored the learning climate of professionalism was sent to all residents at the University of Virginia in 2013-14. Residents rated their personal commitment, their residency program's, and the institution's commitment to demonstrating professionalism behaviors, described professionalism education, reasons for not participating in curricular offerings, the quality of role modeling, and barriers to professionalism. Nearly half the residents completed the survey (47%, N = 365/771). Residents rated their personal commitment and commitment of their residency program significantly greater than the institution's commitment to demonstrating professionalism.(p 0.001) They noted only 25% of faculty modeled these behaviors all the time; and more than half stated poor role modeling impacted their attitudes about the importance of professionalism. Other areas in need of improvement include communicating with patients with cultural differences, and inter-professional teamwork. Despite accreditation requirements for learning environments, residency curricula, and faculty development programs to promote professionalism, residents perceive their commitment to professionalism greater than the institution where they work.

Success in residency matching is largely contingent upon standardized exam scores. Identifying predictors of standardized exam performance could promote primary intervention and lead to design insights for preclinical courses. We hypothesized that clinically relevant courses with an emphasis on higher-order cognitive understanding are most strongly associated with performance on United States Medical Licensing Examination Step exams and National Board of Medical Examiners clinical subject exams. Academic data from students between 2007 and 2012 were collected. Preclinical course scores and standardized exam scores were used for statistical modeling with multiple linear regression. Preclinical courses were categorized as having either a basic science or a clinical knowledge focus. Medical College Admissions Test scores were included as an additional predictive variable. The study sample comprised 795 graduating medical students. Median score on Step 1 was 234 (interquartile range 219-245.5), and 10.2 % (81/795) scored lower than one standard deviation below the national average (205). Pathology course score was the strongest predictor of performance on all clinical subject exams and Step exams, outperforming the Medical College Admissions Test in strength of association. Using Pathology score

The flipped classroom educational model proposes that students will achieve higher order and deeper levels of learning if they work collaboratively during class to apply what they know. However, medical students decline to participate in flipped classroom activities if they perceive that studying alone will be equally efficient in terms of learning. Unfortunately, institutional policies and practices can challenge the ability of medical educators to hold students accountable for attending and engaging in class. As a result, flipped classroom activities must be designed with care, and both the classroom activities and subsequent assessments must challenge students’ higher-order thinking skills.

Objectives The University of Virginia School of Medicine recently transformed its pre-clerkship medical education programme to emphasise student engagement and active learning in the classroom. As in other medical schools, many students are opting out of attending class and others are inattentive while in class. We sought to understand why, especially with a new student-centred curriculum, so many students were still opting to learn on their own outside of class or to disengage from educational activities while in class. Methods Focus groups were conducted with students from two classes who had participated in the new curriculum, which is designed to foster small-group and collaborative learning. The sessions were audio-recorded and then transcribed. The authors read through all of the transcripts and then reviewed them for themes. Quotes were analysed and organised by theme. Results Interview transcripts revealed candid responses to questions about learning and the learning environment. The semi-structured nature of the interviews enabled the interviewers to probe unanticipated issues (e.g. reasons for choosing to sit with friends although that diminishes learning and attention). A content analysis of these transcripts ultimately identified three major themes embracing multiple sub-themes: (i) learning studio physical space; (ii) interaction patterns among learners, and (iii) the quality of and engagement in learning in the space. Conclusions Students' reluctance to engage in class activities is not surprising if classroom exercises are passive and not consistently well designed or executed as active learning exercises that students perceive as enhancing their learning through collaboration. Students' comments also suggest that their reluctance to participate regularly in class may be because they have not yet achieved the developmental level compatible with adult and active learning, on which the curriculum is based. Challenges include helping students better understand the nature of deep learning and their own developmental progress as learners, and providing robust faculty development to ensure the consistent deployment of higher-order learning activities linked with higher-order assessments.

Navigating the biomedical, emotional, and logistical complexity of end-of-life (EOL) care requires seamless interprofessional teamwork. Unfortunately, EOL care, interprofessional collaboration, and the role of support services such as hospice are not adequately emphasized in nursing and medical curricula. This article describes a student-run program, entitled the "HeArt of Medicine", which was designed to foster a reflective and collaborative approach to EOL care. The program consists of three workshops with a novel blend of art, science, and practical information, highlighting the need for interprofessional teamwork. Participants were surveyed before and after the workshops on their attitudes toward EOL care. Composite participant scores after workshops demonstrated increased comfort with and knowledge of EOL care topics (p = 0.001). The results show that this program has had a positive impact on participants' knowledge, comfort, and collaboration in EOL settings.

To measure the impact of a change in grading system in the first two years of medical school, from graded (A, B, C, D, F) to pass/fail, on medical students' academic performance, attendance, residency match, satisfaction, and psychological well-being.For both the graded and pass/fail classes, objective data were collected on academic performance in the first- and second-year courses, the clerkships, United States Medical Licensing Examination (USMLE) Steps 1 and 2 Clinical Knowledge (CK), and residency placement. Self-report data were collected using a Web survey (which included the Dupuy General Well-Being Schedule) administered each of the first four semesters of medical school. The study was conducted from 2002 to 2007 at the University of Virginia School of Medicine.The pass/fail class exhibited a significant increase in well-being during each of the first three semesters of medical school relative to the graded class, greater satisfaction with the quality of their medical education during the first four semesters of medical school, and greater satisfaction with their personal lives during the first three semesters of medical school. The graded and pass/fail classes showed no significant differences in performance in first- and second-year courses, grades in clerkships, scores on USMLE Step 1 and Step 2CK, success in residency placement, and attendance at academic activities.A change in grading from letter grades to pass/fail in the first two years of medical school conferred distinct advantages to medical students, in terms of improved psychological well-being and satisfaction, without any reduction in performance in courses or clerkships, USMLE test scores, success in residency placement, or level of attendance.

Introduction During critical incidents, teamwork failures can compromise patient safety. This study provides evidence that virtual humans can be used in simulated critical incidents to assess the learning needs of health professionals, and provide important information that can inform the development of continuing education programs in patient safety. We explored the effectiveness of information transfer during a devolving medical situation between postanesthesia care unit (PACU) nurses and a virtual attending physician. Methods We designed a three-stage scenario: tutorial, patient transfer, and critical incident. We developed 2 checklists to assess information transfer: Critical Patient Information and Interprofessional Communication Skills. All participants were videotaped; 2 raters reviewed all videos and assessed performance using the checklists. Results Participants (n = 43) who completed all 3 stages scored 62.3% correct on critical patient information transfer and 61.6% correct on interprofessional communication skills. Almost 87% missed a fatal drug error. The checklists measured each item on a 1/0 (done/not) calculation. Additionally, no relationship was found between years of nursing experience and performance on either checklist. Discussion The PACU nurses in this study did not consistently share critical information with an attending (virtual) physician during a critical incident, and most missed a fatal dosage error. These findings strongly suggest a crucial need for additional structured team training among practicing health care teams, and they demonstrate the utility of using virtual humans to simulate team members.

Incarcerated adolescents are typically evaluated for their substance abuse treatment needs, often based on self-report information if their offense is not related to abuse. This study tests consistency of self-reports of alcohol and drug use by comparing responses youth give to corrections staff and to university research interviewers only weeks apart. Significantly more incarcerated adolescents (n= 894) report lifetime drug and alcohol use to researchers than to correctional staff. Sex and ethnic differences were found. Overall, a correlation of .57 is found between prevalence of alcohol and drug use as measured by the correctional staff and that measured by the researchers.

There is little research on training medical students in smoking cessation counseling (SCC). This study aimed to determine if a personal digital assistant (PDA)-based SCC tool can improve medical student SCC.We conducted a randomized, controlled trial with third-year medical students. SCC behaviors, comfort, and knowledge were assessed using a validated survey before students attended a workshop on SCC. Student groups were then randomized to receive a paper-based reminder tool or the reminder plus a PDA-based SCC tool. The validated survey was repeated upon clerkship completion, and a videotaped standardized patient interview was assessed by trained reviewers using a 24-item SCC checklist. Focus groups assessed satisfaction with the PDA tool, usability, and barriers to use.SCC behaviors, knowledge, and comfort increased among all participants, with no statistical differences between groups. The PDA tool group performed 62% of key SCC activities during the videotaped interview, while the control group performed 69%. Students reported discomfort using the PDA with patients, lack of time, and lack of training as barriers to use of the tool.We demonstrated improvement of SCC skills by third-year medical students using a workshop combined with a supplemental reference tool. However, a PDA-based tool did not increase key SCC behaviors compared with a paper-based reminder. For a PDA intervention to be effective in this setting, the tool must be simplified and additional training provided.

Objective To develop an observational coding instrument for evaluating the fidelity and quality of brief behavioral change interventions based on the behavioral theories of the 5 A's, Stages of Change and Motivational Interviewing. Methods Content and face validity were assessed prior to an intervention where psychometric properties were evaluated with a prospective cohort of 116 medical students. Properties assessed included the inter-rater reliability of the instrument, internal consistency of the full scale and sub-scales and descriptive statistics of the instrument. Construct validity was assessed based on student's scores. Results Inter-rater reliability for the instrument was 0.82 (intraclass correlation). Internal consistency for the full scale was 0.70 (KR20). Internal consistencies for the sub-scales were as follows: MI intervention component (KR20 = .7); stage-appropriate MI-based intervention (KR20 = .55); MI spirit (KR20 = .5); appropriate assessment (KR20 = .45) and appropriate assisting (KR20 = .56). Conclusions The instrument demonstrated good inter-rater reliability and moderate overall internal consistency when used to assess performing brief behavioral change interventions by medical students. Practice implications This practical instrument can be used with minimal training and demonstrates promising psychometric properties when evaluated with medical students counseling standardized patients. Further testing is required to evaluate its usefulness in clinical settings.

Objective: The objective of this study was to evaluate a handheld computer smoking cessation intervention tool designed to assist physicians in their smoking cessation counseling with patients. Methods: This study used a pre/post survey design, with a 4-month trial period for the software. Study participants included 22 faculty and resident physicians from the University of Virginia. Paired samples t tests were used to assess mean differences in the 4 main subscales (physician behavior, attitudes, comfort related to counseling patients about smoking cessation, and knowledge). Results: No statistically significant mean differences were found for physician behavior (mean increase = 0.44, P = .55) or physician attitude (mean increase = 0.44, P = .16). A statistically significant mean increase of 2.29 was observed for the physician comfort subscale (t = 3.87, df = 16, P = .001). Physicians indicated improved comfort in counseling patients about smoking cessation (P = .007) and improved comfort in using the Public Health Service Clinical Practice Guidelines (P = .012). Conclusion: Physician comfort level in counseling patients about smoking cessation can be improved through handheld computer software. When used in conjunction with other practice modifications, this tool has the potential to improve physician smoking cessation intervention practices.

Our objective was to determine the ability of the internal medicine In-Training Examination (ITE) to predict pass or fail outcomes on the American Board of Internal Medicine (ABIM) certifying examination and to develop an externally validated predictive model and a simple equation that can be used by residency directors to provide probability feedback for their residency programs. We collected a study sample of 155 internal medicine residents from the three Virginia internal medicine programs and a validation sample of 64 internal medicine residents from a residency program outside Virginia. Scores from both samples were collected across three class cohorts. The Kolmogorov-Smirnov z test indicated no statistically significant difference between the distribution of scores for the two samples (z = 1.284, p = .074). Results of the logistic model yielded a statistically significant prediction of ABIM pass or fail performance from ITE scores (Wald = 35.49, SE = 0.036, df = 1, p < .005) and overall correct classifications for the study sample and validation sample at 79% and 75%, respectively. The ITE is a useful tool in assessing the likelihood of a resident's passing or failing the ABIM certifying examination but is less predictive for residents who received ITE scores between 49 and 66.