1. Measuring deliberate reflection in residents: validation and psychometric properties of a measurement tool
- Author
-
Richard H. Blum, Christine L. Mai, John D. Mitchell, Daniel Saddawi-Konefka, Jeffrey B. Cooper, George Shorten, and Audrey DunnGalvin
- Subjects
Reflection ,Reflective Practice ,Anesthesia Trainees ,Assessment ,REFLECT Rubric & Validation ,Special aspects of education ,LC8-6691 ,Medicine - Abstract
Abstract Purpose Reflective capacity is “the ability to understand critical analysis of knowledge and experience to achieve deeper meaning.” In medicine, there is little provision for post-graduate medical education to teach deliberate reflection. The feasibility, scoring characteristics, reliability, validation, and adaptability of a modified previously validated instrument was examined for its usefulness assessing reflective capacity in residents as a step toward developing interventions for improvement. Methods Third-year residents and fellows from four anesthesia training programs were administered a slightly modified version of the Reflection Evaluation for Learners’ Enhanced Competencies Tool (REFLECT) in a prospective, observational study at the end of the 2019 academic year. Six written vignettes of imperfect anesthesia situations were created. Subjects recorded their perspectives on two randomly assigned vignettes. Responses were scored using a 5-element rubric; average scores were analyzed for psychometric properties. An independent self-report assessment method, the Cognitive Behavior Survey: Residency Level (rCBS) was used to examine construct validity. Internal consistency (ICR, Cronbach’s alpha) and interrater reliability (weighted kappa) were examined. Pearson correlations were used between the two measures of reflective capacity. Results 46/136 invited subjects completed 2/6 randomly assigned vignettes. Interrater agreement was high (k = 0.85). The overall average REFLECT score was 1.8 (1–4 scale) with good distribution across the range of scores. ICR for both the REFLECT score (mean 1.8, sd 0.5; α = 0.92) and the reflection scale of the rCBS (mean 4.5, sd 1.1; α = 0.94) were excellent. There was a significant correlation between REFLECT score and the rCBS reflection scale (r = .44, p
- Published
- 2023
- Full Text
- View/download PDF