Test-retest reproducibility of optic disk deterioration detected from stereophotographs by masked graders

Purpose To assess the reproducibility of determining whether an eye has developed optic disk deterioration by the Optic Disc Reading Center (ODRC) in the Ocular Hypertension Treatment Study (OHTS). Design Test-retest reproducibility study. Methods Masked, certified graders at the ODRC determine the occurrence of optic disk deterioration in OHTS by comparing baseline with follow-up stereoscopic optic disk photographs. To assess reproducibility, regradings were obtained annually by inserting masked “quality control” photographs into the usual ODRC reading stream. Results Agreement (kappa) ranged from 0.65 to 0.83 over 5 years. Specificity ranged from 98% to 100%, and sensitivity ranged from 64% to 81%. Conclusions The kappa statistic for test-retest agreement in OHTS is in the range considered good to excellent over 5 years. Consistency (specificity) in regrading optic disks that did not develop deterioration was particularly high. The sensitivity results show that detecting subtle deterioration of optic disks is challenging.