Search

Colonoscopy performance is typically assessed by a supervisor in the clinical setting. There are limitations of this approach, however, because it allows for rater bias and increases supervisor workload demand during the procedure. Video-based assessment of recorded procedures has been proposed as a complementary means by which to assess colonoscopy performance. This study sought to investigate the reliability, validity, and feasibility of video-based assessments of competence in performing colonoscopy compared with live assessment.Novice (50 previous colonoscopies), intermediate (50-500), and experienced (1000) endoscopists from 5 hospitals participated. Two views of each colonoscopy were videotaped: an endoscopic (intraluminal) view and a recording of the endoscopist's hand movements. Recorded procedures were independently assessed by 2 blinded experts using the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT), a validated procedure-specific assessment tool comprising a global rating scale (GRS) and checklist (CL). Live ratings were conducted by a non-blinded expert endoscopist. Outcomes included agreement between live and blinded video-based ratings of clinical colonoscopies, intra-rater reliability, inter-rater reliability and discriminative validity of video-based assessments, and perceived ease of assessment.Forty endoscopists participated (20 novices, 10 intermediates, and 10 experienced). There was good agreement between the live and video-based ratings (total, intra-class correlation [ICC] = 0.847; GRS, ICC = 0.868; CL, ICC = 0.749). Intra-rater reliability was excellent (total, ICC = 0.99; GRS, ICC = 0.99; CL, ICC = 0.98). Inter-rater reliability between the 2 blinded video-based raters was high (total, ICC = 0.91; GRS, ICC = 0.918; CL, ICC = 0.862). GiECAT total, GRS, and CL scores differed significantly among novice, intermediate, and experienced endoscopists (P .001). Video-based assessments were perceived as "fairly easy," although live assessments were rated as significantly easier (P .001).Video-based assessments of colonoscopy procedures using the GiECAT have strong evidence of reliability and validity. In addition, assessments using videos were feasible, although live assessments were easier.

Self-assessment is important for life-long learning and a recommended assessment method for endoscopy skills. Prior literature has not investigated self-assessment accuracy of colonoscopic competence in the clinical setting. This study aimed to determine the self-assessment accuracy of novice, intermediate, and experienced endoscopists.Novice (performed 50 previous colonoscopies), intermediate (50-500), and experienced (1000) endoscopists from 5 hospitals each performed a clinical colonoscopy. Video recordings of procedures were independently assessed by 2 blinded expert endoscopists by using the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT). Externally assessed and self-assessed GiECAT scores were defined as the mean of the 2 video-based ratings and as participants' own assigned ratings, respectively. Self-assessment accuracy between the externally assessed and self-assessed scores was evaluated by using absolute difference scores, intraclass correlation coefficients, and the Bland-Altman analysis.Twenty novice, 10 intermediate, and 10 experienced endoscopists participated. There was moderate agreement of externally assessed and self-assessed GiECAT scores, with an intraclass correlation coefficient of 0.65 (95% confidence interval, 0.44-0.80). The absolute difference scores among the 3 groups were significantly different (P = .002), with experienced endoscopists demonstrating a more accurate self-assessment ability compared with novices (P = .002). Bland-Altman plots suggest that novice and experienced endoscopists tend to overrate and underrate their clinical competence, respectively; no specific trends were associated with intermediates.Participants demonstrated moderate self-assessment accuracy of clinical competence. Endoscopist experience was positively associated with self-assessment accuracy; novices demonstrated lower self-assessment accuracy compared with experienced endoscopists. Moreover, novices tended to overestimate their performances. Novice endoscopists may benefit from targeted interventions to improve self-assessment accuracy.

Rigorously developed and validated direct observational assessment tools are required to support competency-based colonoscopy training to facilitate skill acquisition, optimize learning, and ensure readiness for unsupervised practice.To examine reliability and validity evidence of the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT) for colonoscopy for use within the clinical setting.Prospective, observational, multicenter validation study. Sixty-one endoscopists performing 116 colonoscopies were assessed using the GiECAT, which consists of a 7-item global rating scale (GRS) and 19-item checklist (CL). A second rater assessed procedures to determine interrater reliability by using intraclass correlation coefficients (ICCs). Endoscopists' first and second procedure scores were compared to determine test-retest reliability by using ICCs. Discriminative validity was examined by comparing novice, intermediate, and experienced endoscopists' scores. Concurrent validity was measured by correlating scores with colonoscopy experience, cecal and terminal ileal intubation rates, and physician global assessment.A total of 116 colonoscopies performed by 33 novice (50 previous procedures), 18 intermediate (50-500 previous procedures), and 10 experienced (1000 previous procedures) endoscopists from 6 Canadian hospitals.Interrater and test-retest reliability, discriminative, and concurrent validity.Interrater reliability was high (total: ICC=0.85; GRS: ICC=0.85; CL: ICC=0.81). Test-retest reliability was excellent (total: ICC=0.91; GRS: ICC=0.93; CL: ICC=0.80). Significant differences in GiECAT scores among novice, intermediate, and experienced endoscopists were noted (P.001). There was a significant positive correlation (P.001) between scores and number of previous colonoscopies (total: ρ=0.78, GRS: ρ=0.80, CL: Spearman's ρ=0.71); cecal intubation rate (total: ρ=0.81, GRS: Spearman's ρ=0.82, CL: Spearman's ρ=0.75); ileal intubation rate (total: Spearman's ρ=0.82, GRS: Spearman's ρ=0.82, CL: Spearman's ρ=0.77); and physician global assessment (total: Spearman's ρ=0.90, GRS: Spearman's ρ=0.94, CL: Spearman's ρ=0.77).Nonblinded assessments.This study provides evidence supporting the reliability and validity of the GiECAT for use in assessing the performance of live colonoscopies in the clinical setting.

Background Ensuring competence remains a seminal objective of endoscopy training programs, professional organizations, and accreditation bodies; however, no widely accepted measure of endoscopic competence currently exists. Objective By using Delphi methodology, we aimed to develop and establish the content validity of the Gastrointestinal Endoscopy Competency Assessment Tool for colonoscopy. Design An international panel of endoscopy experts rated potential checklist and global rating items for their importance as indicators of the competence of trainees learning to perform colonoscopy. After each round, responses were analyzed and sent back to the experts for further ratings until consensus was reached. Main Outcome Measurements Consensus was defined a priori as ≥80% of experts, in a given round, scoring ≥4 of 5 on all remaining items. Results Fifty-five experts agreed to be part of the Delphi panel: 43 gastroenterologists, 10 surgeons, and 2 endoscopy managers. Seventy-three checklist and 34 global rating items were generated through a systematic literature review and survey of committee members. An additional 2 checklist and 4 global rating items were added by Delphi panelists. Five rounds of surveys were completed before consensus was achieved, with response rates ranging from 67% to 100%. Seven global ratings and 19 checklist items reached consensus as good indicators of the competence of clinicians performing colonoscopy. Limitations Further validation required. Conclusion Delphi methodology allowed for the rigorous development and content validation of a new measure of endoscopic competence, reflective of practice across institutions. Although further evaluation is required, it is a promising step toward the objective assessment of competency for use in colonoscopy training, practice, and research.