Search

Importance: Despite its importance to medical education and competency assessment for internal medicine trainees, evidence about the relationship between physicians' milestone residency ratings or the American Board of Internal Medicine's initial certification examination and their hospitalized patients' outcomes is sparse., Objective: To examine the association between physicians' milestone ratings and certification examination scores and hospital outcomes for their patients., Design, Setting, and Participants: Retrospective cohort analyses of 6898 hospitalists completing training in 2016 to 2018 and caring for Medicare fee-for-service beneficiaries during hospitalizations in 2017 to 2019 at US hospitals., Main Outcomes and Measures: Primary outcome measures included 7-day mortality and readmission rates. Thirty-day mortality and readmission rates, length of stay, and subspecialist consultation frequency were also assessed. Analyses accounted for hospital fixed effects and adjusted for patient characteristics, physician years of experience, and year., Exposures: Certification examination score quartile and milestone ratings, including an overall core competency rating measure equaling the mean of the end of residency milestone subcompetency ratings categorized as low, medium, or high, and a knowledge core competency measure categorized similarly., Results: Among 455 120 hospitalizations, median patient age was 79 years (IQR, 73-86 years), 56.5% of patients were female, 1.9% were Asian, 9.8% were Black, 4.6% were Hispanic, and 81.9% were White. The 7-day mortality and readmission rates were 3.5% (95% CI, 3.4%-3.6%) and 5.6% (95% CI, 5.5%-5.6%), respectively, and were 8.8% (95% CI, 8.7%-8.9%) and 16.6% (95% CI, 16.5%-16.7%) for mortality and readmission at 30 days. Mean length of stay and number of specialty consultations were 3.6 days (95% CI, 3.6-3.6 days) and 1.01 (95% CI, 1.00-1.03), respectively. A high vs low overall or knowledge milestone core competency rating was associated with none of the outcome measures assessed. For example, a high vs low overall core competency rating was associated with a nonsignificant 2.7% increase in 7-day mortality rates (95% CI, -5.2% to 10.6%; P = .51). In contrast, top vs bottom examination score quartile was associated with a significant 8.0% reduction in 7-day mortality rates (95% CI, -13.0% to -3.1%; P = .002) and a 9.3% reduction in 7-day readmission rates (95% CI, -13.0% to -5.7%; P < .001). For 30-day mortality, this association was -3.5% (95% CI, -6.7% to -0.4%; P = .03). Top vs bottom examination score quartile was associated with 2.4% more consultations (95% CI, 0.8%-3.9%; P < .003) but was not associated with length of stay or 30-day readmission rates., Conclusions and Relevance: Among newly trained hospitalists, certification examination score, but not residency milestone ratings, was associated with improved outcomes among hospitalized Medicare beneficiaries.

Introduction: In 2020, roughly 25% of applicants who matched into internal medicine (IM) residencies were international medical graduates (IMGs). We examine 12-year trends in distribution of IMGs among IM training programs and explore differences in program perceptions towards IMG recruitment., Methods: Since 2007, Association of Program Directors in Internal Medicine Annual Surveys have collected data about trainees by medical school graduate type. Sixteen additional questions regarding perceptions of IMGs were included in the 2017 spring survey., Results: The 2017 survey response rate was 63.3% (236/373) and ranged from 61.9 to 70.2% for the 2007-2019 Annual Surveys. During that 12-year period, 55-70% of community programs' and 22-30% of university programs' PGY1 positions were filled by IMGs. In 2017, 45% of community programs' and 15% of university programs' interview and ranking positions were allocated to IMGs. Departmental pressure (university 45.6% [95% CI 43.7-47.5]; community 28.2% [95% CI 26.6-29.7]; p = 0.007), institutional priority (university 64.0% [95% CI 62.1-66.0]; community 41% [95% CI 36.9-44.6]; p = 0.001), and reputational concerns (university 52.8% [95% CI 50.0-55.6]; community 38.5% [95% CI 36.0-40.9]; p = 0.045) were cited as factors influencing recruitment of IMGs., Conclusion: Our study was limited to exploring program factors in residency recruitment and did not assess applicant preferences. There is a large, longstanding difference in the recruitment of IMGs to US community-based and university residencies, beginning during the interview and ranking process. Further research in disparities in IMG recruitment is needed, including exploring pressures, preferences, and potential biases associated with the recruitment of IMGs., (© 2021. Society of General Internal Medicine.)

Purpose: To assess the correlations between United States Medical Licensing Examination (USMLE) performance, American College of Physicians Internal Medicine In-Training Examination (IM-ITE) performance, American Board of Internal Medicine Internal Medicine Certification Exam (IM-CE) performance, and other medical knowledge and demographic variables., Method: The study included 9,676 postgraduate year (PGY)-1, 11,424 PGY-2, and 10,239 PGY-3 internal medicine (IM) residents from any Accreditation Council for Graduate Medical Education-accredited IM residency program who took the IM-ITE (2014 or 2015) and the IM-CE (2015-2018). USMLE scores, IM-ITE percent correct scores, and IM-CE scores were analyzed using multiple linear regression, and IM-CE pass/fail status was analyzed using multiple logistic regression, controlling for USMLE Step 1, Step 2 Clinical Knowledge, and Step 3 scores; averaged medical knowledge milestones; age at IM-ITE; gender; and medical school location (United States or Canada vs international)., Results: All variables were significant predictors of passing the IM-CE with IM-ITE scores having the strongest association and USMLE Step scores being the next strongest predictors. Prediction curves for the probability of passing the IM-CE based solely on IM-ITE score for each PGY show that residents must score higher on the IM-ITE with each subsequent administration to maintain the same estimated probability of passing the IM-CE., Conclusions: The findings from this study should support residents and program directors in their efforts to more precisely identify and evaluate knowledge gaps for both personal learning and program improvement. While no individual USMLE Step score was as strongly predictive of IM-CE score as IM-ITE score, the combined relative contribution of all 3 USMLE Step scores was of a magnitude similar to that of IM-ITE score.

Resident physicians routinely perform bedside procedures that pose substantial risk to patients. However, no standard programmatic approach to supervision and procedural competency assessment among residents currently exists. The authors performed a national survey of internal medicine (IM) program directors to examine procedural assessment and supervision practices of IM residency programs. Procedures chosen were those commonly performed by medicine residents at the bedside. Of the 368 IM programs, 226 (61%) completed the survey. Programs reported the predominant method of training as 171 (74%) apprenticeship and 106 (46%) as module based. The majority of programs used direct observation to attest to competence, with 55% to 62% relying on credentialed residents. Most programs also relied on a minimum number of procedures to determine competence (64%-88%), 72% of which reported 5 procedures (a lapsed historical standard). This national survey demonstrates that procedural assessment practices for IM residents are insufficiently robust and may put patients at undue risk.

The American College of Cardiology In-Training Exam (ACC-ITE) is incorporated into most U.S. training programs, but its relationship to performance on the American Board of Internal Medicine Cardiovascular Disease (ABIM CVD) Certification Examination is unknown. ACC-ITE scores from third-year fellows from 2011 to 2014 (n = 1,918) were examined. Covariates for regression analyses included sex, age, medical school country, U.S. Medical Licensing Examination Step, and ABIM Internal Medicine Certification Examination scores. A secondary analysis examined fellows (n = 511) who took the ACC-ITE in the first and third years. ACC-ITE scores were the strongest predictor of ABIM CVD scores (p < 0.0001), and the most significant predictor of passing (p < 0.0001). The change in ACC-ITE scores from first to third year was a strong predictor of the ABIM CVD score (p < 0.001). The ACC-ITE is strongly associated with performance on the ABIM CVD Certification Examination., (Copyright © 2017 American College of Cardiology Foundation. Published by Elsevier Inc. All rights reserved.)

Background: Pregnancy and its impact on graduate medical training are not well understood., Objective: To examine the effect of gender and pregnancy for Internal Medicine (IM) residents on evaluations by peers and faculty., Design: This was a retrospective cohort study., Subjects: All IM residents in training from July 1, 2004-June 30, 2014, were included. Female residents who experienced pregnancy and male residents whose partners experienced pregnancy during training were identified using an existing administrative database., Main Measures: Mean evaluation scores by faculty and peers were compared relative to pregnancy (before, during, and after), accounting for the gender of both the evaluator and resident in addition to other available demographic covariates. Potential associations were assessed using mixed linear models., Key Results: Of 566 residents, 117 (20.7%) experienced pregnancy during IM residency training. Pregnancy was more common in partners of male residents (24.7%) than female residents (13.2%) (p = 0.002). In the post-partum period, female residents had lower peer evaluation scores on average than their male counterparts (p = 0.0099)., Conclusions: A large number of residents experience pregnancy during residency. Mean peer evaluation scores were lower after pregnancy for female residents. Further study is needed to fully understand the mechanisms behind these findings, develop ways to optimize training throughout pregnancy, and explore any unconscious biases that may exist.

Importance: US internal medicine residency programs are now required to rate residents using milestones. Evidence of validity of milestone ratings is needed., Objective: To compare ratings of internal medicine residents using the pre-2015 resident annual evaluation summary (RAES), a nondevelopmental rating scale, with developmental milestone ratings., Design, Setting, and Participants: Cross-sectional study of US internal medicine residency programs in the 2013-2014 academic year, including 21 284 internal medicine residents (7048 postgraduate-year 1 [PGY-1], 7233 PGY-2, and 7003 PGY-3)., Exposures: Program director ratings on the RAES and milestone ratings., Main Outcomes and Measures: Correlations of RAES and milestone ratings by training year; correlations of medical knowledge ratings with American Board of Internal Medicine (ABIM) certification examination scores; rating of unprofessional behavior using the 2 systems., Results: Corresponding RAES ratings and milestone ratings showed progressively higher correlations across training years, ranging among competencies from 0.31 (95% CI, 0.29 to 0.33) to 0.35 (95% CI, 0.33 to 0.37) for PGY-1 residents to 0.43 (95% CI, 0.41 to 0.45) to 0.52 (95% CI, 0.50 to 0.54) for PGY-3 residents (all P values <.05). Linear regression showed ratings differed more between PGY-1 and PGY-3 years using milestone ratings than the RAES (all P values <.001). Of the 6260 residents who attempted the certification examination, the 618 who failed had lower ratings using both systems for medical knowledge than did those who passed (RAES difference, -0.9; 95% CI, -1.0 to -0.8; P < .001; milestone medical knowledge 1 difference, -0.3; 95% CI, -0.3 to -0.3; P < .001; and medical knowledge 2 difference, -0.2; 95% CI, -0.3 to -0.2; P < .001). Of the 26 PGY-3 residents with milestone ratings indicating deficiencies on either of the 2 medical knowledge subcompetencies, 12 failed the certification examination. Correlation of RAES ratings for professionalism with residents' lowest professionalism milestone ratings was 0.44 (95% CI, 0.43 to 0.45; P < .001)., Conclusions and Relevance: Among US internal medicine residents in the 2013-2014 academic year, milestone-based ratings correlated with RAES ratings but with a greater difference across training years. Both rating systems for medical knowledge correlated with ABIM certification examination scores. Milestone ratings may better detect problems with professionalism. These preliminary findings may inform establishment of the validity of milestone-based assessment.

Background: High-quality assessment of resident performance is needed to guide individual residents' development and ensure their preparedness to provide patient care. To facilitate this aim, reporting milestones are now required across all internal medicine (IM) residency programs., Objective: To describe initial milestone ratings for the population of IM residents by IM residency programs., Design: Cross-sectional study., Setting: IM residency programs., Participants: All IM residents whose residency program directors submitted milestone data at the end of the 2013-2014 academic year., Measurements: Ratings addressed 6 competencies and 22 subcompetencies. A rating of "not assessable" indicated insufficient information to evaluate the given subcompetency. Descriptive statistics were calculated to describe ratings across competencies and training years., Results: Data were available for all 21 774 U.S. IM residents from all 383 programs. Overall, 2889 residents (1621 in postgraduate year 1 [PGY-1], 902 in PGY-2, and 366 in PGY-3) had at least 1 subcompetency rated as not assessable. Summaries of average ratings by competency and training year showed higher ratings for PGY-3 residents in all competencies. Overall ratings for each of the 6 individual competencies showed that fewer than 1% of third-year residents were rated as "unsatisfactory" or "conditional on improvement." However, when subcompetency milestone ratings were used, 861 residents (12.8%) who successfully completed training had at least 1 competency with all corresponding subcompetencies graded below the threshold of "readiness for unsupervised practice.", Limitation: Data were derived from a point in time in the first reporting period in which milestones were used., Conclusion: The initial milestone-based evaluations of IM residents nationally suggest that documenting developmental progression of competency is possible over training years. Subcompetencies may identify areas in which residents might benefit from additional feedback and experience. Future work is needed to explore how milestones are used to support residents' development and enhance residency curricula., Primary Funding Source: None.

Objectives: Whether salary disparities exist between men and women in medical education leadership roles is not known. The study objective was to determine whether salary disparities exist between male and female Internal Medicine residency program directors, and if so, to identify factors associated with the disparities and explore historical trends., Methods: The annual Association of Program Directors in Internal Medicine (APDIM) survey in August 2012 included items to assess the salary and demographic characteristics of program directors, which were merged with publically available program data. To assess historical trends, we used similarly obtained survey data from 2008 to 2011. The study included program directors of 370 APDIM member programs, representing 95.6% of the 387 accredited Internal Medicine training programs in the United States and Puerto Rico. Of the 370 APDIM member programs, 241 (65.1%) completed the survey, of whom 169 (70.1%) were men and 72 (29.9%) were women. Program directors' total annual salary, measured in $25,000 increments, ranged from $75,000 or less to more than $400,000. Historical trends of mode salary by gender from 2008 to 2012 were assessed., Results: The mode salary was $200,000 to 225,000 for men and $175,000 to $200,000 for women (P = .0005). After controlling for academic rank, career in general internal medicine, and program director age, the distribution of salary remained different by gender (P = .004). Historical trends show that the difference in mode salary has persisted since 2008., Conclusions: Leaders in academic medical centers, residency and fellowship directors, and all faculty in medical education need to be aware that salary disparities cited decades ago persist in this sample of medical educators. Closing the gender gap will require continued advocacy for measuring and reporting salary gaps, and changing the culture of academic medical centers., (Copyright © 2015 Alliance for Academic Internal Medicine. Published by Elsevier Inc. All rights reserved.)

Background: Point-of-care (POC) ultrasound has been shown to improve procedural outcomes and physical examination accuracy in multiple settings. There are limited data regarding the optimal way to train nonradiologists in POC ultrasound. This is a primary barrier to more widespread use of ultrasound in the physical examination., Objective: We created a workshop to instruct postgraduate year (PGY)-2 and PGY-3 internal medicine residents in POC ultrasound imaging of the abdominal aorta and kidneys., Methods: A half-day simulation center workshop was created to review ultrasound operations and teach residents to independently obtain ultrasound images of the abdominal aorta and kidneys on standardized patients with normal anatomy. The workshop incorporated didactic instruction and hands-on ultrasound practice in small groups. Each resident's ability to independently obtain ultrasound images was assessed using a preworkshop and postworkshop skills examination with a standardized patient. Resident knowledge and attitudes toward POC ultrasound were also assessed using a preworkshop and postworkshop test and survey., Results: A total of 58 residents completed the workshop, and 84% were able to independently obtain high-quality images of the abdominal aorta and kidney after workshop completion, compared with 16% on the preworkshop test. Residents demonstrated a statistically significant increase in their self-reported confidence with ultrasound operation and image acquisition., Conclusions: Training using standardized patients can prepare residents to independently obtain POC ultrasound images of the aorta and kidneys. Training resulted in increased resident confidence with POC ultrasound and self-reported likelihood of future use.

Problem: The transition from medical student to intern may cause stress and burnout in new interns and the delivery of suboptimal patient care. Despite a formal set of subinternship curriculum guidelines, program directors have expressed concern regarding the skill set of new interns and the lack of standardization in that skill set among interns from different medical schools. To address these issues, the Accreditation Council for Graduate Medical Education's Next Accreditation System focuses on the development of a competency-based education continuum spanning undergraduate, graduate, and continuing medical education., Approach: In 2010, the Clerkship Directors in Internal Medicine subinternship task force, in collaboration with the Association of Program Directors in Internal Medicine survey committee, surveyed internal medicine residency program directors to determine which competencies or skills they expected from new medical school graduates. The authors summarized the results using categories of interest., Outcomes: In both an item rank list and free-text responses, program directors were nearly uniform in ranking the skills they deemed most important for new interns-organization and time management and prioritization skills; effective communication skills; basic clinical skills; and knowing when to ask for assistance., Next Steps: Stakeholders should use the results of this survey as they develop a milestone-based curriculum for the fourth year of medical school and for the internal medicine subinternship. By doing so, they should develop a standardized set of skills that meet program directors' expectations, reduce the stress of transitions across the educational continuum, and improve the quality of patient care.

Background: There have been recent calls for improved internal medicine outpatient training, yet assessment of clinical and educational variables within existing models is lacking., Objective: To assess the impact of clinic redesign from a traditional weekly clinic model to a 50/50 outpatient-inpatient model on clinical and educational outcomes., Design: Pre-intervention and post-intervention study intervals, comparing the 2009-2010 and 2010-2011 academic years., Participants: Ninety-six residents in a Primary Care Internal Medicine site of a large academic internal medicine residency program who provide care for > 13,000 patients., Intervention: Continuity clinic redesign from a traditional weekly clinic model to a 50/50 model characterized by 50 % outpatient and 50 % inpatient experiences scheduled in alternating 1 month blocks, with twice weekly continuity clinic during outpatient months and no clinic during inpatient months., Main Measures: 1) Clinical outcomes (panel size, patient visits, adherence with chronic disease and preventive service guidelines, continuity of care, patient satisfaction, and perceived safety/teamwork in clinic); 2) Educational outcomes (attendance at teaching conference, resident and faculty satisfaction, faculty assessment of resident clinic performance, and residents' perceived preparedness for outpatient management)., Results: Redesign was associated with increased mean panel size (120 vs. 137.6; p ≤ 0.001), decreased continuity of care (63 % vs. 48 % from provider perspective; 61 % vs. 51 % from patient perspective; p ≤  0.001 for both; team continuity was preserved), decreased missed appointments (12.5 % vs. 10.9 %; p ≤  0.01), improved perceived safety and teamwork (3.6 vs. 4.1 on 5-point scale; p ≤  0.001), improved mean teaching conference attendance (57.1 vs. 64.4; p ≤  0.001), improved resident clinic performance (3.6 vs. 3.9 on 5-point scale; p ≤  0.001), and little change in other outcomes., Conclusion: Although this model requires further study in other settings, these results suggest that a 50/50 model may allow residents to manage more patients while enhancing the climate of teamwork and safety in the continuity clinic, compared to traditional models. Future work should explore ways to preserve continuity of care within this model.

Background: Patient care and medical knowledge are Accreditation Council for Graduate Medical Education (ACGME) core competencies. The correlation between amount of patient contact and knowledge acquisition is not known., Objective: To determine if a correlation exists between the number of patient encounters and in-training exam (ITE) scores in internal medicine (IM) and pediatric residents at a large academic medical center., Design: Retrospective cohort study, Participants: Resident physicians at Mayo Clinic from July 2006 to June 2010 in IM (318 resident-years) and pediatrics (66 resident-years)., Methods: We tabulated patient encounters through review of clinical notes in an electronic medical record during post graduate year (PGY)-1 and PGY-2. Using linear regression models, we investigated associations between ITE score and number of notes during the previous PGY, adjusted for previous ITE score, gender, medical school origin, and conference attendance., Key Results: For IM, PGY-2 admission and consult encounters in the hospital and specialty clinics had a positive linear association with ITE-3 % score (β = 0.02; p = 0.004). For IM, PGY-1 conference attendance is positively associated with PGY-2 ITE performance. We did not detect a correlation between PGY-1 patient encounters and subsequent ITE scores for IM or pediatric residents. No association was found between IM PGY-2 ITE score and inpatient, outpatient, or total encounters in the first year of training. Resident continuity clinic and total encounters were not associated with change in PGY-3 ITE score., Conclusions: We identified a positive association between hospital and subspecialty encounters during the second year of IM training and subsequent ITE score, such that each additional 50 encounters were associated with an increase of 1 % correct in PGY-3 ITE score after controlling for previous ITE performance and continuity clinic encounters. We did not find a correlation for volume of encounters and medical knowledge for IM PGY-1 residents or the pediatric cohort.

Purpose: To assess internal medicine (IM) and surgery program directors' views of the likely effects of the 2011 Accreditation Council for Graduate Medical Education duty hours regulations., Method: In fall 2010, investigators surveyed IM and surgery program directors, assessing their views of the likely impact of the 2011 duty hours standards on learning environment, workload, education opportunities, program administration, and patient outcomes., Results: Of 381 IM program directors, 287 (75.3%) responded; of 225 surgery program directors, 118 (52.4%) responded. Significantly more surgeons than internists indicated that the new regulations would likely negatively impact learning climate, including faculty morale and residents' relationships (P < .001). Most leaders in both specialties (80.8% IM, 80.2% surgery) felt that the regulations would likely increase faculty workload (P = .73). Both IM (82.2%) and surgery (96.6%) leaders most often rated, of all education opportunities, first-year resident clinical experience to be adversely affected (P < .001). Respondents from both specialties indicated that they will hire more nonphysician/midlevel providers (59.5% IM, 89.0% surgery, P < .001) and use more nonteaching services (66.8% IM, 70.1% surgery, P = .81). Respondents expect patient safety (45.1% IM, 76.9% surgery, P < .001) and continuity of care (83.6% IM across all training levels, 97.5% surgery regarding first-year residents) to decrease., Conclusions: IM and surgery program directors agree that the 2011 duty hours regulations will likely negatively affect the quality of the learning environment, workload, education opportunities, program administration, and patient outcomes. Careful evaluation of actual impact is important.

Objective: To examine the effect of census caps and unit-based admissions on resident workload, conference attendance, duty hour compliance, and patient safety., Participants and Methods: We implemented a census cap of 14 patients on 6 Mayo Clinic internal medicine resident hospital services and a unit-based admissions process in which patients and care teams were consolidated within hospital units. All 280 residents and 15,926 patient admissions to resident and nonresident services 1 year before the intervention (September 1, 2006, through August 31, 2007) and 1 year after the intervention (May 1, 2008, through April 30, 2009) were included. Residents' workload, conference attendance, and duty hours were tracked electronically. Patient safety variables including Rapid Response Team and cardiopulmonary resuscitation events, intensive care unit transfers, Patient Safety Indicators, and 30-day readmissions were compared preintervention and postintervention., Results: After the intervention, residents' mean (SE) ratings of workload appropriateness improved (3.10 [0.08] vs 3.87 [0.08] on a 5-point scale; P<.001), as did conference attendance (1523 [56. 8%] vs 1700 [63.5%] conferences attended; P<.001). Duty hour violations for working more than 30 consecutive hours and not having 10 hours off between duty periods decreased from 77 of 9490 possible violations (0.81%) to 27 (0.28%) and from 70 (0.74%) to 14 (0.15%) violations, respectively (both, P<.001). Thirty-day readmissions to resident services decreased (1010 [18.14%] vs 682 [15. 37%]; P<.001). All other patient safety measures remained unchanged. After adjustment for illness severity, there were no significant differences in patient outcomes between resident and nonresident services., Conclusion: Census caps and unit-based admissions were associated with improvements in resident workload, conference attendance, duty hour compliance, and readmission rates while patient outcomes were maintained., (Copyright © 2012 Mayo Foundation for Medical Education and Research. Published by Elsevier Inc. All rights reserved.)

Background: Few internal medicine residency programs provide formal ultrasound training. This study sought to assess the feasibility of simulation based ultrasound training among first year internal medicine residents and measure their comfort at effectively using ultrasound to perform invasive procedures before and after this innovative model of ultrasound training., Methods: A simulation based ultrasound training module was implemented during intern orientation that incorporated didactic and practical experiences in a simulation and cadaver laboratory. Participants completed anonymous pre and post surveys in which they reported their level of confidence in the use of ultrasound technology and their comfort in identifying anatomic structures including: lung, pleural effusion, bowel, peritoneal cavity, ascites, thyroid, and internal jugular vein. Survey items were structured on a 5-point Likert scales (1 = extremely unconfident, 5 = extremely confident)., Results: Seventy-five out of seventy-six interns completed the pre-intervention survey and 55 completed the post-survey. The mean confidence score (SD) increased to 4.00 (0.47) (p < 0.0001). The mean (SD) comfort ranged from 3.61 (0.84) for peritoneal cavity to 4.48 (0.62) for internal jugular vein. Confidence in identifying all anatomic structures showed an increase over the pre-intervention means (p < 0.002)., Conclusion: A simulation based ultrasound learning module can improve the self-reported confidence with which residents identify structures important in performing invasive ultrasound guided procedures. Incorporating an ultrasound module into residents' education may address perceived need for ultrasound training, improve procedural skills, and enhance patient safety.

Purpose: Residents' reflections on quality improvement (QI) opportunities are poorly understood. The authors used the Mayo Evaluation of Reflection on Improvement Tool (MERIT) to measure residents' reflection scores across three years and to determine associations between reflection scores and resident and adverse patient event characteristics., Method: From 2006 to 2009, 48 Mayo Clinic internal medicine residents completed biannual reflections on adverse events and classified event severity and preventability. Faculty assessed residents' reflections using MERIT, which contains 18 Likert-scaled items and measures three factors-personal reflection, systems reflection, and event merit. ANOVA was used to identify changes in MERIT scores across three years of training and among factors, paired t tests were used to identify differences between MERIT factor scores, and generalized estimating equations were used to examine associations between MERIT scores and resident and adverse event characteristics., Results: The residents completed 240 reflections. MERIT reflection scores were stable over time. Individual factor scores differed significantly (P < .0001), with event merit being the highest and systems reflection the lowest. Event preventability was significantly associated with MERIT factor scores and overall scores (beta = 0.415; CI = 0.186-0.643; P = .0004). No significant associations between MERIT scores and resident characteristics or event severity were identified., Conclusions: Residents' reflections on adverse events remained constant over time, were lowest for systems factors, and were associated with adverse event preventability. Future research should explore learners' emphasis on systems aspects of QI and the relationship between QI and event preventability.

Objective: To determine whether standardized admissions data in residents' Electronic Residency Application Service (ERAS) submissions were associated with multisource assessments of professionalism during internship., Participants and Methods: ERAS applications for all internal medicine interns (N=191) at Mayo Clinic entering training between July 1, 2005, and July 1, 2008, were reviewed by 6 raters. Extracted data included United States Medical Licensing Examination scores, medicine clerkship grades, class rank, Alpha Omega Alpha membership, advanced degrees, awards, volunteer activities, research experiences, first author publications, career choice, and red flags in performance evaluations. Medical school reputation was quantified using U.S. News & World Report rankings. Strength of comparative statements in recommendation letters (0 = no comparative statement, 1 = equal to peers, 2 = top 20%, 3 = top 10% or "best") were also recorded. Validated multisource professionalism scores (5-point scales) were obtained for each intern. Associations between application variables and professionalism scores were examined using linear regression., Results: The mean ± SD (minimum-maximum) professionalism score was 4.09 ± 0.31 (2.13-4.56). In multivariate analysis, professionalism scores were positively associated with mean strength of comparative statements in recommendation letters (β = 0.13; P = .002). No other associations between ERAS application variables and professionalism scores were found., Conclusion: Comparative statements in recommendation letters for internal medicine residency applicants were associated with professionalism scores during internship. Other variables traditionally examined when selecting residents were not associated with professionalism. These findings suggest that faculty physicians' direct observations, as reflected in letters of recommendation, are useful indicators of what constitutes a best student. Residency selection committees should scrutinize applicants' letters for strongly favorable comparative statements.

Objective: To measure the effect of duty periods no longer than 16 hours on patient care and resident education., Patients and Methods: As part of our Educational Innovations Project, we piloted a novel resident schedule for an inpatient service that eliminated shifts longer than 16 hours without increased staffing or decreased patient admissions on 2 gastroenterology services from August 29 to November 27, 2009. Patient care variables were obtained through medical record review. Resident well-being and educational variables were collected by weekly surveys, end of rotation evaluations, and an electronic card-swipe system., Results: Patient care metrics, including 30-day mortality, 30-day readmission rate, and length of stay, were unchanged for the 196 patient care episodes in the 5-week intervention month compared with the 274 episodes in the 8 weeks of control months. However, residents felt less prepared to manage cross-cover of patients (P = .006). There was a nonsignificant trend toward decreased perception of quality of education and balance of personal and professional life during the intervention month. Residents reported working fewer weekly hours overall during the intervention (64.3 vs 68.9 hours; P = .40), but they had significantly more episodes with fewer than 10 hours off between shifts (24 vs 2 episodes; P = .004)., Conclusion: Inpatient hospital services can be staffed with residents working shifts less than 16 hours without additional residents. However, cross-cover of care, quality of education, and time off between shifts may be adversely affected.

Objectives: transformative learning theory supports the idea that reflection on quality improvement (QI) opportunities and the ability to develop successful QI projects may be fundamentally linked. We used validated methods to explore associations between resident doctors' reflections on QI opportunities and the quality of their QI project proposals., Methods: eighty-six residents completed written reflections on practice improvement opportunities and developed QI proposals. Two faculty members assessed residents' reflections using the 18-item Mayo Evaluation of Reflection on Improvement Tool (MERIT), and assessed residents' QI proposals using the seven-item Quality Improvement Project Assessment Tool (QIPAT-7). Both instruments have been validated in previous work. Associations between MERIT and QIPAT-7 scores were determined. Internal consistency reliabilities of QIPAT-7 and MERIT scores were calculated., Results: there were no significant associations between MERIT overall and domain scores, and QIPAT-7 overall and item scores. The internal consistency of MERIT and QIPAT-7 item groups were acceptable (Cronbach's α 0.76-0.94)., Conclusions: the lack of association between MERIT and QIPAT-7 scores indicates a distinction between resident doctors' skills at reflection on QI opportunities and their abilities to develop QI projects. These findings suggest that practice-based reflection and QI project development are separate constructs, and that skilful reflection may not predict the ability to design meaningful QI initiatives. Future QI curricula should consider teaching and assessing QI reflection and project development as distinct components.

Objective: To examine the educational benefits of international elective rotations during graduate medical education., Participants and Methods: We studied Mayo International Health Program (MIHP) participants from April 1, 2001, through July 31, 2008. Data from the 162 resident postrotation reports were reviewed and used to quantitatively and qualitatively analyze MIHP elective experiences. Qualitative analysis of the narrative data was performed using NVivo7 (QRS International, Melbourne, Australia), a qualitative research program, and passages were coded and analyzed for trends and themes., Results: During the study period, 162 residents representing 20 different specialties were awarded scholarships through the MIHP. Residents rotated in 43 countries, serving over 40,000 patients worldwide. Their reports indicated multiple educational and personal benefits, including gaining experience with a wide variety of pathology, learning to work with limited resources, developing clinical and surgical skills, participating in resident education, and experiencing new peoples and cultures., Conclusion: The MIHP provides the structure and funding to enable residents from a variety of specialties to participate in international electives and obtain an identifiable set of unique, valuable educational experiences likely to shape them into better physicians. Such international health electives should be encouraged in graduate medical education.

Objective: To measure actual and perceived resident physician knowledge of underserved patient populations., Participants and Methods: Using the existing literature on vulnerable patient populations and interviews with experts in the field, we designed a cross-sectional, multi-institutional survey to assess actual and perceived resident knowledge of topics related to underserved populations. The survey of actual knowledge consisted of 30 multiple-choice questions, and the survey of perceived knowledge consisted of 15 items based on 3-point Likert scales of confidence., Results: A total of 498 surveys were completed at 18 residency programs representing 7 different specialties at 10 US institutions. Assessment of perceived knowledge demonstrated that residents were very confident only 14.0% of the time, somewhat confident 66.4% of the time, and not at all confident 19.6% of the time. Assessment of actual knowledge revealed that the average percent correct across all 30 questions was 38%. Women scored better than men (average score, 40.6% vs 36.0%; P=.01), and African Americans scored higher than members of other racial or ethnic groups (average score, 43.5% vs 38.0%; P=.04). Associations between residents' perceived and actual knowledge were generally high., Conclusion: For the US residents surveyed, the actual and perceived knowledge about most topics relating to underserved populations was low, whereas associations between actual and perceived knowledge were high. These findings suggest the need to improve resident education regarding underserved patient populations.

Objectives: Resident reflection on the clinical learning environment is prerequisite to identifying quality improvement (QI) opportunities and demonstrating competence in practice-based learning. However, residents' abilities to reflect on QI opportunities are unknown. Therefore, we developed and determined the validity of the Mayo Evaluation of Reflection on Improvement Tool (MERIT) for assessing resident reflection on QI opportunities., Methods: The content of MERIT, which consists of 18 items structured on 4-point scales, was based on existing literature and input from national experts. Using MERIT, six faculty members rated 50 resident reflections. Factor analysis was used to examine the dimensionality of MERIT instrument scores. Inter-rater and internal consistency reliabilities were calculated., Results: Factor analysis revealed three factors (eigenvalue; number of items): Reflection on Personal Characteristics of QI (8.5; 7); Reflection on System Characteristics of QI (1.9; 6), and Problem of Merit (1.5; 5). Inter-rater reliability was very good (intraclass correlation coefficient range: 0.73-0.89). Internal consistency reliability was excellent (Cronbach's alpha 0.93 overall and 0.83-0.91 for factors). Item mean scores were highest for Problem of Merit (3.29) and lowest for Reflection on System Characteristics of QI (1.99)., Conclusions: Validity evidence supports MERIT as a meaningful measure of resident reflection on QI opportunities. Our findings suggest that dimensions of resident reflection on QI opportunities may include personal, system and Problem of Merit factors. Additionally, residents may be more effective at reflecting on 'problems of merit' than personal and systems factors.

Background: Interactions with the pharmaceutical industry are known to affect the attitudes and behaviors of medical residents; however, to our knowledge, a nationally representative description of current practices has not been reported., Methods: The Association of Program Directors in Internal Medicine surveyed 381 US internal medicine residency program directors in 2006-2007 regarding pharmaceutical industry support to their training programs. The primary outcome measure was program director report of pharmaceutical financial support to their residency. Demographic and performance variables were analyzed with regard to these responses., Results: In all, 236 program directors (61.9%) responded to the survey. Of these, 132 (55.9%) reported accepting support from the pharmaceutical industry. One hundred seventy of the 236 program directors (72.0%) expressed the opinion that pharmaceutical support is not desirable. Residency programs were less likely to receive pharmaceutical support when the program director held the opinion that industry support was not acceptable (odds ratio [OR], 0.07; 95% confidence interval [CI], 0.02-0.22). Programs located in the southern United States were more likely to accept pharmaceutical support (OR, 8.45; 95% CI, 1.95-36.57). The American Board of Internal Medicine pass rate was inversely associated with acceptance of industry support: each 1% decrease in the pass rate was associated with a 21% increase in the odds of accepting industry support (OR, 1.21; 95% CI, 1.07-1.36)., Conclusions: Although most of the program directors did not find pharmaceutical support desirable, more than half reported acceptance of industry support. Acceptance of pharmaceutical industry support was less prevalent among residency programs with a program director who considered support unacceptable and those with higher American Board of Internal Medicine pass rates.

Background: Resident physicians are expected to demonstrate medical knowledge. However, little is known about the residents' reading habits and learning preferences., Aim: To assess residents' reading habits and preferred educational resources., Methods: Residents at five internal medicine training programs were surveyed regarding their reading and learning habits and preferences., Results: The majority (77.7%) of residents reported reading less than 7 h a week. Most residents (81.4%) read in response to patient care encounters. The preferred educational format was electronic; 94.6% of residents cited UpToDate as the most effective resource for knowledge acquisition, and 88.9% of residents reported that UpToDate was their first choice for answering clinical questions., Conclusions: Residents spent little time reading and sought knowledge primarily from electronic resources. Most residents read in the context of patient care. Future research should focus on strategies for helping resident physicians learn in the electronic age.

Context: The ways hospitalists interact with and contribute to internal medicine residencies in the United States have been described locally, but have not been documented on a national level., Objectives: To describe the penetration of hospitalists into medicine residency faculty nationally, and document their contributions to teaching activities., Design, Setting, and Participants: Survey of all 386 internal medicine residency directors in the United States in 2005 (272 respondents) and 2007 (236 respondents)., Measurements: Number of teaching hospitals utilizing hospitalists, number of programs utilizing hospitalists to teach, hospitalist teaching duties, and number with hospitalist tracks., Results: In 2005, program directors recalled 54% of teaching hospitals employed hospitalists before and 73% after implementation of work-hour limitations. Of those employing hospitalists, 92% of programs in the Northeast and West used them to teach. Two years later, the Midwest (78%) and South (76%) continued to lag behind in the proportion of teaching hospitalists. Specific teaching activities of hospitalists included: attending on teaching service (92%), conducting rounds (81%), observation of clinical skills (67%), lectures (68%), and morning report (52%). Seven percent of program directors reported other duties of hospitalists, including: supervising procedures, reviewing night float patients, serving as associate program directors, and writing curricula. Eleven percent of training programs had hospitalist tracks., Conclusions: As hospitalists have become prevalent and have become efficient clinicians in community and university hospitals, the majority of internal medicine residencies have enlisted them to provide rounds, lectures, and bedside teaching. A small number of residencies are beginning to develop tracks to facilitate this new career option for graduates., (Copyright 2009 Society of Hospital Medicine)

Background: Physicians often enter the workplace with substantial debt. The relationship between debt and reported career plans among internal medicine residents is unknown., Objective: To determine distributions of educational debt among internal medicine residents and associations of debt with reported career plans., Design: Cross-sectional survey using data from the annual Internal Medicine In-Training Examination Residents Questionnaire completed by U.S. categorical internal medicine residents., Setting: Categorical internal medicine residencies in the United States., Participants: 22,563 residents in their third (final) year of residency, representing 74.1% of all eligible U.S. categorical internal medicine residents from 2003 through 2007., Measurements: Distributions of educational debt were tabulated. Proportions of residents choosing career plans were calculated for various levels of debt., Results: International medical graduates represented 48.7% of the cross section and had considerably less debt than U.S. medical graduates: 53.8% of U.S. medical graduates had debt of 100,000 dollars or greater and 60.2% of international medical graduates had none. U.S. medical graduates with debt of 100,000 dollars to 150,000 dollars were less likely than those with no debt to choose a subspecialty career (57.5% vs. 63.5%). U.S. medical graduates with debt of 50,000 dollars to 99,999 dollars were more likely than those with no debt to choose a hospitalist career (8.5% vs. 6.2%), and this preference increased with increasing debt level (10.0% for those with >150,000 dollars debt). These associations are more pronounced for U.S. medical graduates than for international medical graduates., Limitation: The study addressed total educational debt, but not when it was incurred, and did not allow inferences related to causality., Conclusion: Educational debt is associated with differences in reported career plans among internal medicine residents.

Background: Little is known about the associations of previous standardized examination scores with scores on subsequent standardized examinations used to assess medical knowledge in internal medicine residencies., Objective: To examine associations of previous standardized test scores on subsequent standardized test scores., Design: Retrospective cohort study., Participants: One hundred ninety-five internal medicine residents., Methods: Bivariate associations of United States Medical Licensing Examination (USMLE) Steps and Internal Medicine In-Training Examination (IM-ITE) scores were determined. Random effects analysis adjusting for repeated administrations of the IM-ITE and other variables known or hypothesized to affect IM-ITE score allowed for discrimination of associations of individual USMLE Step scores on IM-ITE scores., Results: In bivariate associations, USMLE scores explained 17% to 27% of the variance in IME-ITE scores, and previous IM-ITE scores explained 66% of the variance in subsequent IM-ITE scores. Regression coefficients (95% CI) for adjusted associations of each USMLE Step with IM-ITE scores were USMLE-1 0.19 (0.12, 0.27), USMLE-2 0.23 (0.17, 0.30), and USMLE-3 0.19 (0.09, 0.29)., Conclusions: No single USMLE Step is more strongly associated with IM-ITE scores than the others. Because previous IM-ITE scores are strongly associated with subsequent IM-ITE scores, appropriate modeling, such as random effects methods, should be used to account for previous IM-ITE administrations in studies for which IM-ITE score is an outcome.

Objective: To examine the association of medical knowledge acquisition with attendance at specific conference types during internal medicine residency., Participants and Methods: Attendance at residency core curriculum, morbidity and mortality, and medical grand rounds conferences was tracked for 195 residents who took the Internal Medicine In-Training Examination (IM-ITE) 421 times while attending the Internal Medicine Residency Program at Mayo Clinic's site in Rochester, MN, between October 1, 1999, and October 1, 2003. Random effects modeling was used to adjust for variables known or hypothesized to be associated with IM-ITE scores and allowed discrimination of effects of individual conferences on IM-ITE scores., Results: Total conference attendance was significantly associated with increased IM-ITE scores, with cohort mean increase of 2.30% (95% confidence interval [CI], 1.24% to 3.36%; P<.001). Core curriculum conferences accounted for much more of this association than either medical grand rounds or morbidity and mortality, with cohort mean increases of 2.05% (95% CI, 0.13% to 3.90%; P=.04), 0.24% (95% CI, -0.56% to 1.03%; P=.61), and 0.03% (95% CI, -1.69% to 1.69%; P=.97), respectively., Conclusion: Conference attendance is associated with improved knowledge acquisition as measured by improved performance on IM-ITE. This finding does not appear to apply to all conferences at our institution but rather is specifically attributable to attendance at the core curriculum series.

Background: Disease registries, audit and feedback, and clinical reminders have been reported to improve care processes., Objective: To assess the effects of a registry-generated audit, feedback, and patient reminder intervention on diabetes care., Design: Randomized controlled trial conducted in a resident continuity clinic during the 2003-2004 academic year., Participants: Seventy-eight categorical Internal Medicine residents caring for 483 diabetic patients participated. Residents randomized to the intervention (n = 39) received instruction on diabetes registry use; quarterly performance audit, feedback, and written reports identifying patients needing care; and had letters sent quarterly to patients needing hemoglobin A1c or cholesterol testing. Residents randomized to the control group (n = 39) received usual clinic education., Measurements: Hemoglobin A1c and lipid monitoring, and the achievement of intermediate clinical outcomes (hemoglobin A1c <7.0%, LDL cholesterol <100 mg/dL, and blood pressure <130/85 mmHg) were assessed., Results: Patients cared for by residents in the intervention group had higher adherence to guideline recommendations for hemoglobin A1c testing (61.5% vs 48.1%, p = .01) and LDL testing (75.8% vs 64.1%, p = .02). Intermediate clinical outcomes were not different between groups., Conclusions: Use of a registry-generated audit, feedback, and patient reminder intervention in a resident continuity clinic modestly improved diabetes care processes, but did not influence intermediate clinical outcomes.

Background: Residency programs involve trainees in quality improvement (QI) projects to evaluate competency in systems-based practice and practice-based learning and improvement. Valid approaches to assess QI proposals are lacking., Objective: We developed an instrument for assessing resident QI proposals--the Quality Improvement Proposal Assessment Tool (QIPAT-7)-and determined its validity and reliability., Design: QIPAT-7 content was initially obtained from a national panel of QI experts. Through an iterative process, the instrument was refined, pilot-tested, and revised., Participants: Seven raters used the instrument to assess 45 resident QI proposals., Measurements: Principal factor analysis was used to explore the dimensionality of instrument scores. Cronbach's alpha and intraclass correlations were calculated to determine internal consistency and interrater reliability, respectively., Results: QIPAT-7 items comprised a single factor (eigenvalue = 3.4) suggesting a single assessment dimension. Interrater reliability for each item (range 0.79 to 0.93) and internal consistency reliability among the items (Cronbach's alpha = 0.87) were high., Conclusions: This method for assessing resident physician QI proposals is supported by content and internal structure validity evidence. QIPAT-7 is a useful tool for assessing resident QI proposals. Future research should determine the reliability of QIPAT-7 scores in other residency and fellowship training programs. Correlations should also be made between assessment scores and criteria for QI proposal success such as implementation of QI proposals, resident scholarly productivity, and improved patient outcomes.

Background: Knowledge acquisition is a goal of residency and is measurable by in-training exams. Little is known about factors associated with medical knowledge acquisition., Objective: To examine associations of learning habits on medical knowledge acquisition., Design, Participants: Cohort study of all 195 residents who took the Internal Medicine In-Training Examination (IM-ITE) 421 times over 4 years while enrolled in the Internal Medicine Residency, Mayo Clinic, Rochester, MN., Measurements: Score (percent questions correct) on the IM-ITE adjusted for variables known or hypothesized to be associated with score using a random effects model., Results: When adjusting for demographic, training, and prior achievement variables, yearly advancement within residency was associated with an IM-ITE score increase of 5.1% per year (95%CI 4.1%, 6.2%; p < .001). In the year before examination, comparable increases in IM-ITE score were associated with attendance at two curricular conferences per week, score increase of 3.9% (95%CI 2.1%, 5.7%; p < .001), or self-directed reading of an electronic knowledge resource 20 minutes each day, score increase of 4.5% (95%CI 1.2%, 7.8%; p = .008). Other factors significantly associated with IM-ITE performance included: age at start of residency, score decrease per year of increasing age, -0.2% (95%CI -0.36%, -0.042%; p = .01), and graduation from a US medical school, score decrease compared to international medical school graduation, -3.4% (95%CI -6.5%, -0.36%; p = .03)., Conclusions: Conference attendance and self-directed reading of an electronic knowledge resource had statistically and educationally significant independent associations with knowledge acquisition that were comparable to the benefit of a year in residency training.

Objective: To determine the feasibility, efficacy, and outcomes of teaching Internal Jugular (IJ) central venous line placement (CVLP) to internal medicine residents in a hands-on training experience with adult patients., Subjects and Methods: Data were obtained from 47 residents during their 3-year residency program through questionnaires and a proprietary system that tracks resident procedures. Twenty-five postgraduate year (PGY) 2 residents at the Mayo Clinic in Rochester, Minn, were assigned to IJ-CVLP training in the cardiac catheterization laboratory from January 2001 to June 2001. Their experience, analyzed immediately after training and at completion of residency, was compared with that of 22 PGY-2 residents in the same class who were not assigned to IJ-CVLP training., Results: The median Likert scores of the residents' self-reported perception of independence in IJ-CVLP increased from 3.0 (mean +/- SD score, 2.8+/-1.4) before the intervention to 5.0 (4.4+/-0.9) after the intervention (P<.001, signed rank test). At graduation, trained residents had performed more IJ-CVLPs than the control residents (mean +/- SD, 17.8+/-8.4 vs 9.8+/-6.3, respectively; P<.001). Residents who received IJ-CVLP training, compared with those who did not, showed a significant increase in the mean percentage of IJ-CVLPs performed independently between PGY-1 (2.2%) and PGY-3 (31.2%) (P=.008)., Conclusions: Training internal medicine residents to perform IJ-CVLP is feasible in the cardiac catheterization laboratory with supervision from an attending cardiologist. Trained residents performed significantly more IJ-CVLPs independently during their third year compared with their first year of training. We believe this initiative may be implemented successfully in graduate medical education curriculums.