Search

Describes a workshop in curriculum development for international medical fellows designed and implemented at the Department of Medical Education of the University of Illinois at Chicago. Definition of curriculum; Basic principles involved in the course; Educational strategies.

Importance: The sociopolitical and cultural context of graduate surgical education has changed considerably over the past 2 decades. Although new structures of graduate surgical training programs have been developed in response and the comparative value of formats are continually debated, it remains unclear how different time-based structural paradigms are preparing trainees for independent practice after program completion., Objective: To investigate the factors associated with trainees' and program directors' perception of trainee preparedness for independent surgical practice., Design, Setting, and Participants: This qualitative study used an instrumental case study approach and obtained information through semistructured interviews, which were analyzed using open-and-focused coding. Participants were recent graduates and program directors of vascular surgery training programs in the United States. The 2 training paradigms analyzed were the integrated vascular surgery residency program (0 + 5, with 0 indicating that the general surgery training experiences are fully integrated into the 5 years of overall training and 5 indicating the total number of years of training) and the traditional vascular surgery fellowship program (5 + 2, with 5 indicating the number of years of general surgery training and 2 indicating the number of years of vascular surgery training). All graduates completed their training in 2018. All interviews were conducted between July 1, 2018, and September 30, 2018., Main Outcomes and Measures: A conceptual framework to inform current and ongoing efforts to optimize graduate surgical training programs across specialties., Results: A total of 22 semistructured interviews were completed, involving 7 graduates of 5 + 2 programs, 9 graduates of 0 + 5 programs, and 6 vascular surgery program directors. Of the 22 participants, 15 were men (68%). Participants described 4 interconnected domains that were associated with trainees' perceived preparedness for practice: structural, individual, relational, and organizational. Structural factors included the overall and vascular surgery-specific time spent in training, whereas individual factors included innate technical skills, confidence, maturity, and motivation. Faculty-trainee relationships (or relational factors) were deemed important for building trust and granting of autonomy. Organizational factors included features of the local organization, including patient population, case volume, and case mix., Conclusions and Relevance: Findings suggest that restructuring training paradigms alone is insufficient to address the issue of trainees' perceived preparedness for practice. A framework was created from the results for evaluating and improving residency and fellowship programs as well as for developing graduate surgical training paradigms that incorporate all 4 domains associated with preparedness.

Purpose: Medical educators use key features examinations (KFEs) to assess clinical decision making in many countries, but not in U.S. medical schools. The authors developed an online KFE to assess third-year medical students' decision-making abilities during internal medicine (IM) clerkships in the United States. They used Messick's unified validity framework to gather validity evidence regarding response process, internal structure, and relationship to other variables., Method: From February 2012 through January 2013, 759 students (at eight U.S. medical schools) had 75 minutes to complete one of four KFE forms during their IM clerkship. They also completed a survey regarding their experiences. The authors performed item analyses and generalizability studies, comparing KFE scores with prior clinical experience and National Board of Medical Examiners Subject Examination (NBME-SE) scores., Results: Five hundred fifteen (67.9%) students consented to participate. Across KFE forms, mean scores ranged from 54.6% to 60.3% (standard deviation 8.4-9.6%), and Phi-coefficients ranged from 0.36 to 0.52. Adding five cases to the most reliable form would increase the Phi-coefficient to 0.59. Removing the least discriminating case from the two most reliable forms would increase the alpha coefficient to, respectively, 0.58 and 0.57. The main source of variance came from the interaction of students (nested in schools) and cases. Correlation between KFE and NBME-SE scores ranged from 0.24 to 0.47 (P < .01)., Conclusions: These results provide strong evidence for response-process and relationship-to-other-variables validity and moderate internal structure validity for using a KFE to complement other assessments in U.S. IM clerkships.

Background: To determine the psychometric characteristics of diagnostic justification scores based on the patient note format of the United States Medical Licensing Examination Step 2 Clinical Skills exam, which requires students to document history and physical findings, differential diagnoses, diagnostic justification, and plan for immediate workup., Method: End-of-third-year medical students at one institution wrote notes for five standardized patient cases in May 2013 (n = 180) and 2014 (n = 177). Each case was scored using a four-point rubric to rate each of the four note components. Descriptive statistics and item analyses were computed and a generalizability study done., Results: Across cases, 10% to 48% provided no diagnostic justification or had several missing or incorrect links between history and physical findings and diagnoses. The average intercase correlation for justification scores ranged from 0.06 to 0.16; internal consistency reliability of justification scores (coefficient alpha across cases) was 0.38. Overall, justification scores had the highest mean item discrimination across cases. The generalizability study showed that person-case interaction (12%) and task-case interaction (13%) had the largest variance components, indicating substantial case specificity., Conclusions: The diagnostic justification task provides unique information about student achievement and curricular gaps. Students struggled to correctly justify their diagnoses; performance was highly case specific. Diagnostic justification was the most discriminating element of the patient note and had the greatest variability in student performance across cases. The curriculum should provide a wide range of clinical cases and emphasize recognition and interpretation of clinically discriminating findings to promote the development of clinical reasoning skills.

Purpose: This study examines validity evidence for the Patient Note Scoring Rubric, which was developed for a local graduation competency exam (GCE) to assess patient notes written in the new United States Medical Licensing Examination (USMLE) Step 2 Clinical Skills format. The rubric was designed to measure three dimensions: Documentation, justified differential diagnosis (DDX), and Workup., Method: Analyses used GCE data from 170 fourth-year medical students who completed five standardized patient (SP) cases in May 2012. Five physician raters each scored all responses for one case. Internal structure was examined using correlations between dimensions and between cases; a generalizability study was also conducted. Relationship to other variables was examined by correlating patient note scores with SP encounter scores. Consequence was assessed by comparing pass-fail rates between the rubric and the previous global rating. Response process was examined using rater feedback., Results: Correlations between scores from different dimensions ranged between 0.33 and 0.44. Reliability of scores based on the phi coefficient was 0.43; 15 cases were required to reach a phi coefficient of 0.70. Evidence of case specificity was found. Documentation scores were moderately correlated with SP scores for data gathering (r = 0.47, P < .001). There was no meaningful change in pass-fail rates. Raters' feedback indicated that they required more training for scoring the DDX and Workup dimensions., Conclusions: There is initial validity evidence for use of this rubric to score local clinical exams that are based on the new USMLE patient note format.

Purpose: To identify the qualities and skills of exemplary and ideal pediatric hospitalist educators., Method: The authors conducted a prospective, multi-institutional qualitative study from November 2008 through January 2009 in which they interviewed pediatric hospitalists who were identified as exemplary educators at three academic pediatric residency programs. They then conducted focus groups with residents and medical students who had recently worked with these hospitalists. Qualitative analysis was used to identify themes., Results: All six hospitalists identified as exemplary participated. Among invited learners, 14/18 residents (78%) and 16/18 medical students (89%) participated. Together, the participants contributed 266 comments, which the authors categorized into 36 themes within the four domains of teaching skills, personal qualities, patient care skills, and role modeling. New qualities and skills--including self-reflection/insight, encouraging autonomy, time management, knowledge acquisition, and systems knowledge--and differences in perceptions among hospitalists, residents, and students were identified. Differences between the qualities and skills of actual exemplary hospitalist educators and perceptions of those of an ideal hospitalist educator were also identified., Conclusions: Pediatric hospitalists in academic residency programs have unique opportunities to significantly affect the education of medical students and residents. This study validates and expands on prior studies of the qualities and skills needed to be a successful hospitalist educator. Researchers and educators designing faculty development programs to train more successful hospitalist educators may wish to target these qualities and skills as well as the differences in medical student and resident needs.

Objective: The Accreditation Council for Graduate Medical Education (ACGME) has challenged program directors to assess their residents' core competencies, including communication and interpersonal skills (CIS). We report our institution's experience using a series of standardized patient encounters in an objective structured clinical examination (OSCE) to evaluate CIS in surgical residents., Methods: Standardized patients rated the residents' ability to maintain a patient-centered approach across 6 challenging communication tasks. Residents received verbal feedback from the patients after each encounter and completed a survey indicating their experience and comfort with each task. Individual and group reports documented resident competency and provided aggregate information for curriculum review. Formal grades were not assigned., Results: Twenty-two residents in 2 surgical residency programs piloted the assessment. The Generalizability of the assessment was 0.81. Scores of second- and third-year residents were not significantly different. Residents found the program to be helpful and able to assess their skills., Conclusions: The standardized patient-based OSCE is an effective method to assess communication and interpersonal skills and provides useful information for curriculum review.

Second-year medical students performed a head-to-toe screening physical examination on trained patient instructors (PIs) as a high-stakes examination before beginning clinical clerkships. PIs completed a 138-item checklist and instructed the students in the proper performance of any incorrectly executed or omitted items. To assess the accuracy of the PI ratings, 'monitor' PIs watching the encounter by video completed checklists for 11 retests of failed students in 2001 and for 28 randomly selected encounters in 2002. Checklist-level discrepancies measured by mean absolute checklist difference were 8% in 2001 and 9% in 2002. Case-level mean difference between raters in 2002 was -2.9%. The inter-rater reliability for a single rater, estimated by the intra-class correlation coefficient for total encounter scores, was 0.95. The assessment and feedback were well received by the students. The Head-to-Toe examination provides a reliable assessment of a student's ability to carry out a complete screening physical examination. Patient instructors can be trained to acceptable accuracy and reliability despite the length and complexity of the exercise, and can provide effective feedback and teaching targeted to individual student deficiencies.

Purpose: To determine whether psychiatry program directors and residents agree on the characteristics most important in determining the quality of a residency program., Method: The authors carried out factor analyses of the results of two national surveys that asked participants to rate the importance of 41 items in determining the quality of residency programs: a 1997 survey completed by 180 psychiatry residents and a 1998 survey completed by 234 psychiatry program directors and rotation heads., Results: Residents' factors determining program quality were the interpersonal culture in the program, the curriculum, academic resources and opportunities, clinical resources and opportunities, and outcomes. Program directors' factors were program administration, curriculum and clinical resources, the quality of the institution, the supportiveness of the program, and individual preferences., Conclusions: Program directors and residents focus on different indicators of the quality of residency programs, differences that can be conceptualized as those between the "producer" and the "consumer" of the program. Four domains appear to underlie the results of the resident and faculty factor analyses: the context, content, culture, and consequences of the program. Multidimensional evaluations by multiple stakeholders may be the most appropriate way to evaluate the quality of residency programs. These results also strongly suggest that the interpersonal culture of a program should be assessed as part of its evaluation process.

The authors propose a general definition of quality in residency train ing and describe a process for developing the definition. The definition and process go beyond the usual concerns with educational structures and processes and are customer-centered, outcomes-oriented, and consistent with views of quality widely applied in health care, manufacturing, and other businesses. The process begins by listing some of the more important customers of a training program--trainees, patients, funding agencies, health care employers, medical schools and teaching hospitals, and professional organizations--and then considers the needs of those customers. The result is a definition of quality in residency training that is multidimensional and includes aspects of residency training related to the clinical proficiency of trainees, socioeconomic dimensions of training, and ethical concerns. While some measures exist to put this multidimensional definition of quality into operation, much work will be needed to make it useful for the training of residents.