Revisiting How We Assess Therapist Competence in Cognitive Therapy

We report on two approaches to addressing the problem of low reliability in the assessment of therapist competence in cognitive therapy (CT). In Study 1, we utilized ratings of a session from each of six therapists and investigated how CT experts evaluate specific therapist behaviors in making ratings on the Cognitive Therapy Scale (CTS). Departures from the consensus on judgments of these therapist behaviors were associated with more discrepant competence ratings. We take these results to suggest that, at least among experts, providing greater clarity about which therapist behaviors are relevant to each CTS item may enhance reliability. In Study 2, we utilize a sample of 14 therapists and pilot test a standardized assessment of therapist competence. The standardized assessment exhibited an intra-class correlation coefficient of 0.89 for a single rater. Standardized assessments of competence were related to four measures of therapist experience and qualifications, whereas the CTS was only related to two of these measures. With these promising initial results, we encourage future research to examine the predictive validity of standardized competence assessments in CT.