Search

Background: Critical reflection by faculty physicians on adverse patient events is important for changing physician's behaviors. However, there is little research regarding physician reflection on quality improvement (QI).Objective: To develop and validate a computerized case-based learning system (CBLS) to measure faculty physicians' reflections on adverse patient events.Design: Prospective validation study.Participants: Staff physicians in the Department of Medicine at Mayo Clinic Rochester.Main Measures: The CBLS was developed by Mayo Clinic information technology, medical education, and QI specialists. The reflection questionnaire, adapted from a previously validated instrument, contained eight items structured on five-point scales. Three cases, representing actual adverse events, were developed based on the most common error types: systems, medication, and diagnostic. In 2009, all Mayo Clinic hospital medicine, non-interventional cardiology, and pulmonary faculty were invited to participate. Faculty reviewed each case, determined the next management step, rated case generalizability and relevance, and completed the reflection questionnaire. Factor analysis and internal consistency reliability were calculated. Associations between reflection scores and characteristics of faculty and patient cases were determined.Key Results: Forty-four faculty completed 107 case reflections. The CBLS was rated as average to excellent in 95 of 104 (91.3%) completed satisfaction surveys. Factor analysis revealed two levels of reflection: Minimal and High. Internal consistency reliability was very good (overall Cronbach's α=0.77). Item mean scores ranged from 2.89 to 3.73 on a five-point scale. The overall reflection score was 3.41 (standard deviation 0.64). Reflection scores were positively associated with case generalizability (p=0.001), and case relevance (p=0.02).Conclusions: The CBLS is a valid method for stratifying faculty physicians' levels of reflection on adverse patient events. Reflection scores are associated with case generalizability and relevance, indicating that reflection improves with pertinent patient encounters. We anticipate that this instrument will be useful in future research on QI among low versus high-reflecting physicians. [ABSTRACT FROM AUTHOR]

Background: Continuing medical education (CME) often uses passive educational models including lectures. However, numerous studies have questioned the effectiveness of these less engaging educational strategies. Studies outside of CME suggest that engaged learning is associated with improved educational outcomes. However, measuring participants' engagement can be challenging. We developed and determined the validity evidence for a novel instrument to assess learner engagement in CME., Methods: We conducted a cross-sectional validation study at a large, didactic-style CME conference. Content validity evidence was established through review of literature and previously published engagement scales and conceptual frameworks on engagement, along with an iterative process involving experts in the field, to develop an eight-item Learner Engagement Instrument (LEI). Response process validity was established by vetting LEI items on item clarity and perceived meaning prior to implementation, as well as using a well-developed online platform with clear instructions. Internal structure validity evidence was based on factor analysis and calculating internal consistency reliability. Relations to other variables validity evidence was determined by examining associations between LEI and previously validated CME Teaching Effectiveness (CMETE) instrument scores. Following each presentation, all participants were invited to complete the LEI and the CMETE., Results: 51 out of 206 participants completed the LEI and CMETE (response rate 25%) Correlations between the LEI and the CMETE overall scores were strong (r = 0.80). Internal consistency reliability for the LEI was excellent (Cronbach's alpha = 0.96). To support validity to internal structure, a factor analysis was performed and revealed a two dimensional instrument consisting of internal and external engagement domains. The internal consistency reliabilities were 0.96 for the internal engagement domain and 0.95 for the external engagement domain., Conclusion: Engagement, as measured by the LEI, is strongly related to teaching effectiveness. The LEI is supported by robust validity evidence including content, response process, internal structure, and relations to other variables. Given the relationship between learner engagement and teaching effectiveness, identifying more engaging and interactive methods for teaching in CME is recommended.