NCOG-04. ANALYSIS OF EXAMINER ERRORS ON COGNITIVE TESTING IN MULTI-SITE STUDIES

There is increasing interest in cognitive function as an outcome measure in brain tumor clinical trials. A 3-test battery has been recommended, including the Trail Making Test, Parts A and B (TMT A and TMT B), the Hopkins Verbal Learning Test-Revised (HVLT-R), and the Controlled Oral Word Association Test (COWAT). Use of this battery allows direct comparison and pooling of data across studies. We have used this battery with robust outcomes (e.g., Brown et al 2016). By necessity, psychometric examiners must be trained at many sites for multi-site studies, and some examiner error in testing and recording results is inevitable. To manage this, we re-score all cognitive test protocols for our studies prior to data entry. However, there is a growing interest in on-site data entry as an efficiency measure. We looked at error rates in cognitive testing using the aforementioned 3-test battery and standard training of examiners to inform quality-improvement measures and assess whether on-site entry would be prudent. In 500 consecutive exams, we found that 189 (37.8%) had at least one error. Of 274 total errors, 29 were not correctible upon review, leading to invalidation (1.3% of 2,277 tests administered). 48.5% of the errors would not have been detected with basic electronic on-site entry. Error rates (one or more errors/times given) for each test were TMT A 6%, TMT B 9%, COWAT 21.2%, HVLT-R Learning 7.5%, HVLT-R Delayed Recall 7.1% and HVLT-R Recognition 11.6%. The majority of the invalidating errors 26/29 (89.7%) were on the Trail Making Test.