Assessing competency in practice-based learning: a foundation for milestones in learning portfolio entries.

Background: Graduate medical education is undergoing a dramatic shift toward competency-based assessment of learners. Competency assessment requires clear definitions of competency and validated assessment methods. The purpose of this study is to identify criteria used by surgical educators to judge competence in Practice-Based Learning and Improvement (PBL&I) as demonstrated in learning portfolios.
Methods: A total of 6 surgical learning and instructional portfolio entries served as documents to be assessed by 3 senior surgical educators. These faculty members were asked to rate and then identify criteria used to assess PBL&I competency. Individual interviews and group discussions were conducted, recorded, and transcribed to serve as the study dataset. Analysis was performed using qualitative methodology to identify themes for the purpose of defining competence in PBL&I. The assessment themes derived are presented with narrative examples to describe the progression of competency.
Results: The collaborative coding process resulted in identification of 7 themes associated with competency in PBL&I related to surgical learning and instructional portfolio entries: (1) self-awareness regarding effect of actions; (2) identification and thorough description of learning goals; (3) cases used as catalyst for reflection; (4) reconceptualization with appropriate use and critique of cited literature; (5) communication skills/completeness of entry template; (6) description of future behavioral change; and (7) engagement in process--identifies as personally relevant.
Conclusions: The identified themes are consistent with and complement other criteria emerging from reflective practice literature and experiential learning theory. This study provides a foundation for further development of a tool for assessing learner portfolios consistent with the Accreditation Council for Graduate Medical Education's Next Accreditation System requirements.
(Copyright © 2014 Association of Program Directors in Surgery. Published by Elsevier Inc. All rights reserved.)