Search

BackgroundThe Mindfulness-Based Interventions: Teaching Assessment Criteria (MBI:TAC) is an important tool for assessing teacher skill and aspects of the fidelity of mindfulness-based interventions, but prior research on and implementation of the MBI:TAC has used video recordings, which can be difficult to obtain, share for assessments, and which increase privacy concerns for participants. Audio-only recordings might be a useful alternative, but their reliability is unknown.ObjectiveTo assess evaluator perception of the rating process and inter-rater reliability of MBI:TAC ratings using audio-only recordings.MethodsWe prepared audio-only files from video recordings of 21 previously rated Mindfulness-Based Stress Reduction teachers. Each audio recording was rated by 3 trained MBI:TAC assessors drawn from a pool of 12 who had previously participated in rating the video recordings. Teachers were rated by evaluators who had not viewed the video recording and did not know the teacher. We then conducted semi-structured interviews with evaluators.ResultsOn the 6 MBI:TAC domains, the intraclass correlation coefficients (ICCs) for audio recordings ranged from .53 to .69 using an average across 3 evaluators. Using a single rating resulted in lower ICCs (.27-.38). Bland-Altman plots showed audio ratings had little consistent bias compared to video recordings and agreed more closely for teachers with higher ratings. Qualitative analysis identified 3 themes: video recordings were particularly helpful when rating less skillful teachers, video recordings tended to provide a more complete picture for rating, and audio rating had some positive features.ConclusionsInter-rater reliability of the MBI:TAC using audio-only recordings was adequate for many research and clinical purposes, and reliability is improved when using an average across several evaluators. Ratings using audio-only recordings may be more challenging when rating less experienced teachers.

Assessing program or intervention fidelity/integrity is an important methodological consideration in clinical and educational research. These critical variables influence the degree to which outcomes can be attributed to the program and the success of the transition from research to practice and back again. Research in the Mindfulness-Based Program (MBP) field has been expanding rapidly over the last 20 years, but little attention has been given to how to assess intervention integrity within research and practice settings. The proliferation of different program forms, inconsistency in adhering to published curriculum guides, and variability of training levels and competency of trial teachers all pose grave risks to the sustainable development of the science of MBPs going forward. Three tools for assessing intervention integrity in the MBP field have been developed and researched to assess adherence and/or teaching competence: the Mindfulness-Based Cognitive Therapy-Adherence Scale (MBCT-AS), the Mindfulness-Based Relapse Prevention-Adherence and Competence Scale (MBRP-AC), and the Mindfulness-Based Interventions: Teaching Assessment Criteria (MBI:TAC). Further research is needed on these tools to better define their inter-rater reliability and their ability to measure elements of teaching competence that are important for participant outcomes. Research going forward needs to include systematic and consistent methods for demonstrating and verifying that the MBP was delivered as intended, both to ensure the rigor of individual studies and to enable different studies of the same MBP to be fairly and validly compared with each other. The critical variable of the teaching also needs direct investigation in future research. We recommend the use of the "Template for Intervention Description and Replication" (TIDieR) guidelines for addressing and reporting on intervention integrity during the various phases of the conduct of research and provide specific suggestions about h

Assessing program or intervention fidelity/integrity is an important methodological consideration in clinical and educational research. These critical variables influence the degree to which outcomes can be attributed to the program and the success of the transition from research to practice and back again. Research in the Mindfulness-Based Program (MBP) field has been expanding rapidly over the last 20 years, but little attention has been given to how to assess intervention integrity within research and practice settings. The proliferation of different program forms, inconsistency in adhering to published curriculum guides, and variability of training levels and competency of trial teachers all pose grave risks to the sustainable development of the science of MBPs going forward. Three tools for assessing intervention integrity in the MBP field have been developed and researched to assess adherence and/or teaching competence: the Mindfulness-Based Cognitive Therapy-Adherence Scale (MBCT-AS), the Mindfulness-Based Relapse Prevention-Adherence and Competence Scale (MBRP-AC), and the Mindfulness-Based Interventions: Teaching Assessment Criteria (MBI:TAC). Further research is needed on these tools to better define their inter-rater reliability and their ability to measure elements of teaching competence that are important for participant outcomes. Research going forward needs to include systematic and consistent methods for demonstrating and verifying that the MBP was delivered as intended, both to ensure the rigor of individual studies and to enable different studies of the same MBP to be fairly and validly compared with each other. The critical variable of the teaching also needs direct investigation in future research. We recommend the use of the "Template for Intervention Description and Replication" (TIDieR) guidelines for addressing and reporting on intervention integrity during the various phases of the conduct of research and provide specific suggestions ab