Search

Background: Entrustable Professional Activities (EPAs) contain narrative 'entrustment roadmaps' designed to describe specific behaviors associated with different entrustment levels. However, these roadmaps were created using expert committee consensus, with little data available for guidance. Analysis of actual EPA assessment narrative comments using natural language processing may enhance our understanding of resident entrustment in actual practice., Methods: All text comments associated with EPA microassessments at a single institution were combined. EPA-entrustment level pairs (e.g. Gallbladder Disease-Level 1) were identified as documents. Latent Dirichlet Allocation (LDA), a common machine learning algorithm, was used to identify latent topics in the documents associated with a single EPA. These topics were then reviewed for interpretability by human raters., Results: Over 18 months, 1015 faculty EPA microassessments were collected from 64 faculty for 80 residents. LDA analysis identified topics that mapped 1:1 to EPA entrustment levels (Gammas >0.99). These LDA topics appeared to trend coherently with entrustment levels (words demonstrating high entrustment were consistently found in high entrustment topics, word demonstrating low entrustment were found in low entrustment topics)., Conclusions: LDA is capable of identifying topics relevant to progressive surgical entrustment and autonomy in EPA comments. These topics provide insight into key behaviors that drive different level of resident autonomy and may allow for data-driven revision of EPA entrustment maps., Competing Interests: Declaration of competing interest The authors report no conflicts of interest., (Copyright © 2020 Elsevier Inc. All rights reserved.)

Background: Longitudinal contact between faculty and residents facilitates greater faculty entrustment. The purpose of this study is to assess the relationship between faculty familiarity with residents and faculty entrustment., Materials and Methods: Researchers observed and rated entrustment behaviors using OpTrust, September 2015-June 2017 at Michigan Medicine. Faculty familiarity with resident was measured on a 1-4 scale (1 = not familiar, 4 = extremely familiar). ANOVA and Sidak adjusted multiple comparisons were used to assess the relationship between faculty familiarity and faculty entrustment., Results: 56 faculty and 73 residents were observed across 225 surgical cases. Faculty entrustment scores increased to 2.48 when resident familiarity was reported as "slightly familiar". Faculty entrustment scores for "moderately familiar" increased to 2.57. Faculty entrustment scores for "extremely familiar" increased to 2.84., Conclusions: We found a positive relationship between faculty familiarity and entrustment. These findings support greater continuity in faculty/resident relationships. Longitudinal contact allows learners to be granted progressive entrustment., Summary: This study demonstrates a positive relationship between faculty familiarity with residents and an increase in intraoperative entrustment. These findings support greater continuity in faculty/resident relationships., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Background: Lack of transparency and meaningful assessment in surgical residency has led to inconsistent intraoperative entrustment and highly variable trainee competence at graduation. The relationship between faculty entrustment and resident entrustability on clinical competency remains unclear. We sought to evaluate the dynamic between entrustment/entrustability and clinical competency in general surgery residency., Methods: Intraoperative observations were conducted across a 22-month period at an academic tertiary center. Entrustment/entrustability were measured using OpTrust. Clinical competencies were appraised via ACGME Milestones and Objective Structured Assessment of Technical Skill (OSATS) scores. Mixed effects linear regression was used to investigate the relationship among overall ACGME Milestone scores, OSATS domain scores, and overall OpTrust scores., Results: Overall OpTrust scores significantly correlated with overall Milestone scores and multiple OSATS score domains., Conclusions: OpTrust demonstrated a positive association between ACGME general surgery Milestones and OSATS scores. Overall, OpTrust may help optimize intraoperative faculty entrustment and resident entrustability, facilitating surgical trainee success during residency., Competing Interests: Declaration of competing interest The authors do not have any conflicts of interest to declare., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Background: Faculty entrustment decisions affect resident entrustability behaviors and surgical autonomy. The relationship between entrustability and autonomy is not well understood. This pilot study explores that relationship., Methods: 108 case observations were completed. Entrustment behaviors were rated using OpTrust. Residents completed a Zwisch self-assessment to measure surgical autonomy. Resident perceived autonomy was collected for 67 cases used for this pilot study., Results: Full entrustability was observed in 5 of the 108 observed cases. Residents in our study did not report full autonomy. Spearman's rank correlation coefficient identified that resident entrustability was positively correlated with perceived resident autonomy (ρ = 0.66, p < 0.05). Ordinal logistic regression assessed the relationship between resident entrustability and autonomy. The relationship persisted while controlling for PGY level, gender, and case complexity (OR = 8.42, SEM = 4.54, p < 0.000)., Conclusions: Resident entrustability is positively associated with perceived autonomy, yet full entrustability is not translating to the perception of full autonomy for residents., (Copyright © 2018 Elsevier Inc. All rights reserved.)

Background: Changes in the surgical training landscape have sparked an interest in developing new educational models anchored on entrustment assessment. We sought to optimize the validated OpTrust entrustment assessment tool by comparing ratings from short-course video reviews to previously validated intraoperative assessments., Methods: Entrustment assessment scores for video-based and 1-h (short-course) observations were compared to previously validated intraoperative assessment scores. Faculty and residents were surveyed for their perceptions related to operative observation., Results: There was a strong association between entrustment scores when comparing in-person to video-based observations (R 2  = 0.76-0.84, p < 0.01) as well as short-course to full-duration observations (R 2  = 0.65-0.76, p < 0.01). The majority of faculty and residents (>97%) felt observation did not negatively impact operative experience., Conclusions: Assessment of entrustment behaviors using short-course video review provides a feasible approach to intraoperative assessment. This latest application of OpTrust allows for the tool to be incorporated into surgical training programs across a variety of environments., (Copyright © 2017 Elsevier Inc. All rights reserved.)

Background: Increasing focus on more granular assessment in medical education has led to more lengthy instruments, with concern that the increased complexity undermines the utility of these tools. This study evaluated the relative contribution of individual questions in an assessment of resident performance and a faculty performance evaluation by residents., Methods: The authors performed factor analysis on the individual items in the resident assessment instrument (3,009 assessments of 71 residents) and faculty evaluations (7,328 evaluations of 61 faculty) collected from 2006 to 2012., Results: Factor analysis of the resident assessment tool revealed that 1 component was responsible for 96.6% of the variance. This component encompassed each question from the assessment form, and could also be termed "overall resident competency." Factor analysis of the attending evaluation form revealed 2 unique components, representing "clinical care" and "interpersonal skills," which accounted for 89.9% of variance., Conclusions: Three components accounted for 90% to 97% of the observed variance in our analysis. Factor analysis represents a useful strategy for analyzing the utility of data obtained from individual items in the assessment and evaluation instruments., (Copyright © 2015 Elsevier Inc. All rights reserved.)

Background: An operative anatomy course was developed within the construct of a surgical internship preparatory curriculum. This course provided fourth-year medical students matching into a surgical residency the opportunity to perform intern-level procedures on cadavers under the guidance of surgical faculty members., Methods: Senior medical students performed intern-level procedures on cadavers with the assistance of faculty surgeons. Students' confidence, anxiety, and procedural knowledge were evaluated both preoperatively and postoperatively. Preoperative and postoperative data were compared both collectively and based on individual procedures., Results: Student confidence and procedural knowledge significantly increased and anxiety significantly decreased when preoperative and postoperative data were compared (P < .05). Students reported moderate to significant improvement in their ability to perform a variety of surgical tasks., Conclusions: The consistent improvement in confidence, knowledge, and anxiety justifies further development of an operative anatomy course, with future assessment of the impact on performance in surgical residency., (Copyright © 2013 Elsevier Inc. All rights reserved.)

Background: This study compares a laparoscopic skill training protocol without proficiency targets to the same protocol with explicit targets and notification of progress., Methods: Fourteen surgery interns were randomized into 2 groups. The intervention group received task-specific proficiency criteria to guide practice. The control group did not. After training, participants were evaluated by blinded faculty during laparoscopic cholecystectomy., Results: The control group met significantly fewer of the 7 LapSim (Surgical Science Sweden AB, Gotëborg) targets (mean = 1.14) than the intervention group (mean = 7.00; P = .001) and significantly fewer of the 5 video trainer targets (mean = .86) than the intervention group (mean = 5.00; P = .001). Ratings of depth perception, bimanual dexterity, efficiency, tissue handling, autonomy, and overall competence were higher for the intervention group. Effect sizes ranged from medium to large (.394-.981), indicating an impact of the proficiency-based training protocol., Conclusions: Delineation of proficiency targets with reporting of progress improves interns' practice results, and appears to have a positive effect on their early operating room (OR) performance of laparoscopic cholecystectomy.

Background: The present study was undertaken to determine if psychomotor and visual-spatial abilities improve as a result of surgical training or are enhanced at baseline in those individuals choosing a surgical career., Methods: Medical students entering a surgical field and practicing surgeons performed a series of neuropsychologic tests. Performance was compared between surgeon groups, as well as with normative aged-matched controls., Results: An age-related decline was noted in the performance of all exercises, with the medical student group outperforming the midcareer surgeons, who in turn outperformed the senior surgeons. Interestingly, however, all 3 groups significantly outperformed their normative control groups on some or all tasks., Conclusions: Improved visual memory and psychomotor performance compared with normative controls appears to be present at baseline rather than resulting from surgical training. Decline in performance with age is observed, however, and this should be considered when an older surgeon is learning new visually complex procedures.

Background: Computer-aided simulators may increase the safety and efficiency of training in laparoscopic surgery. Before implementation of the Immersion LapSim (Gaithersburg, MD) simulator in our training curriculum, we wished to determine its construct validity (ie, whether the simulator could differentiate laparoscopic novices from trainees with greater experience)., Methods: Subjects were medical students (MS), residents (RES), and laparoscopic faculty (FAC). Subjects performed 10 repetitions of 6 LapSim tasks. The LapSim measured performance in 6 to 10 parameters per task, and performance was compared between groups. Post hoc t tests were used to make pair-wise comparisons among the 3 groups using the Bonferroni technique. Statistical significance was set at P < .05., Results: The degree of prior laparoscopic experience was reflected in performance on at least 1 parameter for each task. Several patterns of performance between MS, RES, and FAC were observed., Conclusions: The LapSim has performance parameters that reliably differentiate between subjects with varying laparoscopic experience. However, some performance parameters do not differentiate between groups. To accurately measure a trainee's skill level, only parameters that sensitively measure the true level of performance should be used.

Background: Despite equivalent performance as assessed by objective external measures, women tend to underestimate their abilities compared with their male counterparts., Methods: The difference in ordinal values from faculty and resident self-evaluations was calculated for each general and plastic surgery resident. Objective external performance measures were compared for female and male residents., Results: Male and female residents performed equivalently. All residents underestimated their abilities compared with faculty assessment; however, general surgery residents did so to a greater degree (P < .05). Female residents demonstrated a greater degree of underestimation than did their male colleagues; however, this was not statistically significant., Conclusions: Although female resident surgeons are generally confident in their abilities, this may be in contrast to the self-perception of many female medical students. Consideration of gender differences in self-perception may be important when providing feedback to female students and residents.

Background: No universally accepted method to measure professionalism exists. We developed an instrument to measure specific aspects of professionalism in surgical residents., Methods: Professionalism was deconstructed into 15 domains. Behavioral descriptors were determined for extreme and selected intermediate anchors. It became evident that residents could "go too far" in some professional behaviors. Therefore, although a 7-point continuous ordinal scale forms the framework, a score of 7 does not necessarily indicate the ideal. This characteristic minimizes the problem of inflated ratings., Results: The instrument was utilized by attending faculty to evaluate residents and also by residents as a self-evaluation. Calculated from ordinal values, mean (SD) of the ratings across domains for faculty evaluations of residents was 4.95 (0.38) while mean for self-evaluations was 4.95 (0.39). Reliability was high (coefficient alpha 0.85)., Conclusions: This instrument provides a means to measure professional behaviors during surgical residency. Repeated use will be required to thoroughly establish validity and reliability.