Search

ContextClinical supervisors make judgements about how much to trust learners with critical activities in patient care. Such decisions mediate trainees' opportunities for learning and competency development and thus are a critical component of education. As educators apply entrustment frameworks to assessment, it is important to determine how narrative feedback reflecting entrustment may also address learners' educational needs.MethodsIn this study, we used artificial intelligence (AI) and natural language processing (NLP) to identify characteristics of feedback tied to supervisors' entrustment decisions during direct observation encounters of clerkship medical students (3328 unique observations). Supervisors conducted observations of students and collaborated with them to complete an entrustment-based assessment in which they documented narrative feedback and assigned an entrustment rating. We trained a deep neural network (DNN) to predict entrustment levels from the narrative data and developed an explainable AI protocol to uncover the latent thematic features the DNN used to make its prediction.ResultsWe found that entrustment levels were associated with level of detail (specific steps for performing clinical tasks), feedback type (constructive versus reinforcing) and task type (procedural versus cognitive). In justifying both high and low levels of entrustment, supervisors detailed concrete steps that trainees performed (or did not yet perform) competently.ConclusionsFraming our results in the factors previously identified as influencing entrustment, we find a focus on performance details related to trainees' clinical competency as opposed to nonspecific feedback on trainee qualities. The entrustment framework reflected in feedback appeared to guide specific goal-setting, combined with details necessary to reach those goals. Our NLP methodology can also serve as a starting point for future work on entrustment and feedback as similar assessment datasets accumulate.

IntroductionTrust between supervisors and trainees mediates trainee participation and learning. A resident (postgraduate) trainee's understanding of their supervisor's trust can affect their perceptions of their patient care responsibilities, opportunities for learning, and overall growth as physicians. While the supervisor perspective of trust has been well studied, less is known about how resident trainees recognize supervisor trust and how it affects them.MethodsIn this qualitative study, 21 pediatric residents were interviewed at a single institution. Questions addressed their experiences during their first post-graduate year (PGY-1) on inpatient wards. Each interviewee was asked to describe three different patient care scenarios in which they perceived optimal, under-, and over-trust from their resident supervisor. Data were analyzed using thematic analysis.ResultsResidents recognized and interpreted their supervisor's trust through four factors: supervisor, task, relationship, and context. Optimal trust was associated with supervision balancing supervisor availability and resident independence, tasks affording participation in decision-making, trusting relationships with supervisors, and a workplace fostering appropriate autonomy and team inclusivity. The effects of supervisor trust on residents fell into three themes: learning experiences, attitudes and self-confidence, and identities and roles. Optimal trust supported learning via tailored guidance, confidence and lessened vulnerability, and a sense of patient ownership and team belonging.DiscussionUnderstanding how trainees recognize supervisor trust can enhance interventions for improving the dialogue of trust between supervisors and trainees. It is important for supervisors to be cognizant of their trainees' interpretations of trust because it affects how trainees understand their patient care roles, perceive autonomy, and approach learning.

We propose a dyadic Item Response Theory (dIRT) model for measuring interactions of pairs of individuals when the responses to items represent the actions (or behaviors, perceptions, etc.) of each individual (actor) made within the context of a dyad formed with another individual (partner). Examples of its use include the assessment of collaborative problem solving or the evaluation of intra-team dynamics. The dIRT model generalizes both Item Response Theory models for measurement and the Social Relations Model for dyadic data. The responses of an actor when paired with a partner are modeled as a function of not only the actor's inclination to act and the partner's tendency to elicit that action, but also the unique relationship of the pair, represented by two directional, possibly correlated, interaction latent variables. Generalizations are discussed, such as accommodating triads or larger groups. Estimation is performed using Markov-chain Monte Carlo implemented in Stan, making it straightforward to extend the dIRT model in various ways. Specifically, we show how the basic dIRT model can be extended to accommodate latent regressions, multilevel settings with cluster-level random effects, as well as joint modeling of dyadic data and a distal outcome. A simulation study demonstrates that estimation performs well. We apply our proposed approach to speed-dating data and find new evidence of pairwise interactions between participants, describing a mutual attraction that is inadequately characterized by individual properties alone.