Search

Background Graduate medical education (GME) program leaders struggle to incorporate quality measures in the ambulatory care setting, leading to knowledge gaps on how to provide feedback to residents and programs. While nationally collected quality of care data are available, their reliability for individual resident learning and for GME program improvement is understudied. Objective To examine the reliability of the Healthcare Effectiveness Data and Information Set (HEDIS) clinical performance measures in family medicine and internal medicine GME programs and to determine whether HEDIS measures can inform residents and their programs with their quality of care. Methods From 2014 to 2017, we collected HEDIS measures from 566 residents in 8 family medicine and internal medicine programs under one sponsoring institution. Intraclass correlation was performed to establish patient sample sizes required for 0.70 and 0.80 reliability levels at the resident and program levels. Differences between the patient sample sizes required for reliable measurement and the actual patients cared for by residents were calculated. Results The highest reliability levels for residents (0.88) and programs (0.98) were found for the most frequently available HEDIS measure, colorectal cancer screening. At the GME program level, 87.5% of HEDIS measures had sufficient sample sizes for reliable measurement at alpha 0.7 and 75.0% at alpha 0.8. Most resident level measurements were found to be less reliable. Conclusions GME programs may reliably evaluate HEDIS performance pooled at the program level, but less so at the resident level due to patient volume.

PURPOSE Despite the importance of training in ambulatory care settings for residents to acquire important competencies, little is known about the organizational and environmental factors influencing the relative amount of time primary care residents train in ambulatory care during residency. The authors examined factors associated with postgraduate year 1 (PGY-1) residents' ambulatory care training time in Accreditation Council for Graduate Medical Education (ACGME)-accredited primary care programs. METHOD U.S.-accredited family medicine (FM) and internal medicine (IM) programs' 2016-2017 National Graduate Medical Education (GME) Census data from 895 programs within 550 sponsoring institutions (representing 13,077 PGY-1s) were linked to the 2016 Centers for Medicare and Medicaid Services Cost Reports and 2015-2016 Area Health Resource File. Multilevel regression models examined the association of GME program characteristics, sponsoring institution characteristics, geography, and environmental factors with PGY-1 residents' percentage of time spent in ambulatory care. RESULTS PGY-1 mean (standard deviation, SD) percent time spent in ambulatory care was 25.4% (SD, 0.4) for both FM and IM programs. In adjusted analyses (% increase [standard error, SE]), larger faculty size (0.03% [SE, 0.01], P < .001), sponsoring institution's receipt of Teaching Health Center (THC) funding (6.6% (SE, 2.7), P < .01), and accreditation warnings (4.8% [SE, 2.5], P < .05) were associated with a greater proportion of PGY-1 time spent in ambulatory care. Programs caring for higher proportions of Medicare beneficiaries spent relatively less time in ambulatory care (< 0.5% [SE, 0.2], P < .01). CONCLUSIONS Ambulatory care time for PGY-1s varies among ACGME-accredited primary care residency programs due to the complex context and factors primary care GME programs operate under. Larger ACGME-accredited FM and IM programs and those receiving federal THC GME funding had relatively more PGY-1 time spent in ambulatory care settings. These findings inform policies to increase resident exposure in ambulatory care, potentially improving learning, competency achievement, and primary care access.

Background Secure messages exchanged between patients and family medicine residents via an electronic health record (EHR) could be used to assess residents' clinical and communication skills, but the mechanism is not well understood. Objective To design and test a secure messaging competency assessment for family medicine residents in a patient-centered medical home (PCMH). Methods Using the existing literature and evidence-based guidelines, we designed an assessment tool to evaluate secure messaging competency for family medicine residents training in a PCMH. Core faculty performed 2-stage validity and reliability testing (n = 2 and n = 9, respectively). A series of randomly selected EHR secure messages (n = 45) were assessed from a sample of 10 residents across all years of training. Results The secure message assessment tool provided data on a set of competencies and a framework for resident feedback. Assessment showed 10% (n = 2) of residents at the novice level, 50% (n = 10) as progressing, and 40% (n = 8) as proficient. The most common deficiencies for residents' secure messages related to communication rather than clinical competencies (n = 37 [90%] versus n = 4 [10%]). Interrater reliability testing ranged from 60% to 78% agreement and 20% to 44% disagreement. Disagreement centered on interpersonal communication factors. After 2 stages of testing, the assessment using residents' secure messages was incorporated into our existing evaluation process. Conclusions Assessing family medicine residents' secure messaging for patient encounters closed an evaluation gap in our family medicine program, and offered residents feedback on their clinical and communication skills in a PCMH.

INTRODUCTION Competence in using an electronic health record (EHR) is considered a critical skill for physicians practicing in patient-centered medical homes (PCMHs), but few studies have examined the impact of EHR training for residents preparing to practice in PCMHs. This study explored the educational outcomes associated with comprehensive EHR training for family medicine residents. METHODS The PCMH EHR training consisted of case-based routine clinic visits delivered to 3 resident cohorts (N = 18). Participants completed an EHR competency self-assessment between 2011 and 2016 (N = 127), examining 6 EHR/PCMH core skills. We compared baseline characteristics for residents by low vs high exposure to EHR training. Multivariate regression estimated whether self-reported competencies improved over time and whether high PCMH EHR training exposure was associated with incremental improvement in self-reported competencies over time. RESULTS Residents completed an average of 8.2 sessions: low-exposure residents averaged 5.3 sessions (standard deviation = 1.5), and high-exposure residents averaged 9.0 sessions (standard deviation = 0.9). High-exposed residents had higher posttest scores at training completion (84.4 vs 70.7). Over time, adjusted mean scores (confidence interval) for both groups improved (p < 0.001) from 12.2 (9.6-14.8), with low-exposed residents having greater score improvement (p < 0.001) because of their much lower baseline scores. CONCLUSION Comprehensive training designed to improve EHR competencies among residents practicing in a PCMH resulted in improved assessment scores. Our findings indicate EHR training as part of resident exposure to the PCMH measurably improves self-assessed competencies, even among residents less engaged in EHR training.

Implementing an innovation, such as offering new types of patient-physician encounters through the patient-centered medical home (PCMH) model while maintaining Accreditation Council for Graduate Medical Education (ACGME) accreditation standards (e.g., patient encounter minimums for trainees), is challenging.In 2009, the Group Health Family Medicine Residency (GHFMR) received an ACGME Program Experimentation and Innovation Project (PEIP) exception that redefined the minimum Family Medicine Resident Review Committee requirement to 1,400 face-to-face visits and 250 electronic visits (1 electronic visit defined as 3 secure message or telephone encounters). The authors report GHFMR residents' continuity clinic encounters, specifically volume, from 2006 through 2013 via pre- and post-PCMH implementation. They discuss the implications for leaders of high-performing practices who desire to innovate while maintaining accreditation.Post-PCMH residents had 20% more overall patient contact. The largest change in care delivery method included a large increase in secure messages between patients and residents. Pre-PCMH residents had more face-to-face encounters; however, post-PCMH residents had more contact for all types of patient care encounters (face-to-face, secure messaging, and telephone) per hour of clinic time.The ACGME PEIP exception, allowing the incorporation of the PCMH, facilitated an increase in patient access and immersed residents in primary care innovation (namely, practicing in a PCMH model during graduate medical education training). The next steps are to assess the effect of the PCMH on resident learning and clinical outcomes and to continue residents' access to training that keeps pace with today's health care delivery needs.