Development and validation of an endoscopic submucosal dissection video assessment tool

Background: Despite a need for assessment of endoscopic submucosal dissection (ESD) skills in order to track progress and determine competence, there is no structured measure of assessing competency in ESD performance. The present study aims to develop and examine validity evidence for an assessment tool to evaluate the recorded performance of ESD for gastric neoplasms. Methods: The ESD video assessment tool (EVAT) was systematically developed by ESD experienced endoscopists. The EVAT consists of a 25-item global rating scale and 3-item checklist to assess competencies required to perform ESD. Five unedited videos were each evaluated by 2-blinded experienced ESD endoscopists to assess inter-rater reliability using intraclass correlation coefficients (ICC). Seventeen unedited videos in total were rated by 3 blinded experienced ESD endoscopists. Validity evidence for relationship to other variables was examined by comparing scores of inexperienced (fellows) and experienced endoscopists (attending staff), and by evaluating the relationship between the EVAT scores and ESD case experience. Internal consistency was evaluated using Cronbach’s alpha. Results: The inter-rater reliability for the total score was high at 0.87 (95% confidence interval 0.11 to 0.99). The total score [median, interquartile range (IQR)] was significantly different between the inexperienced (71, 63–77) and experienced group (95, 91–97) (P= 0.005). The total scores demonstrated high correlation with the number of ESD cases (Spearman’s ρ= 0.79, P< 0.01). The internal consistency was 0.97. Conclusions: This study provides preliminary validity evidence for the assessment of video-recorded ESD performances for gastric neoplasms using EVAT.