1. Evaluation of an AI system for the automated detection of glaucoma from stereoscopic optic disc photographs: the European Optic Disc Assessment Study
- Author
-
Nicolaas J. Reus, Sameer Trikha, Thomas W. Rogers, Hans G Lemij, Nicolas Jaccard, Koenraad A. Vermeer, and Francis Carbonaro
- Subjects
FOS: Computer and information sciences ,Optometrists ,genetic structures ,Computer Vision and Pattern Recognition (cs.CV) ,Optic Disk ,Computer Science - Computer Vision and Pattern Recognition ,Glaucoma ,Stereoscopy ,Sensitivity and Specificity ,Glaucomatous optic neuropathy ,Article ,law.invention ,03 medical and health sciences ,Deep Learning ,0302 clinical medicine ,Artificial Intelligence ,Predictive Value of Tests ,law ,Optic Nerve Diseases ,FOS: Electrical engineering, electronic engineering, information engineering ,Photography ,medicine ,Humans ,False Positive Reactions ,Diagnosis, Computer-Assisted ,Retrospective Studies ,Observer Variation ,Ophthalmologists ,business.industry ,Image and Video Processing (eess.IV) ,Significant difference ,Reproducibility of Results ,Electrical Engineering and Systems Science - Image and Video Processing ,medicine.disease ,eye diseases ,Large cohort ,Europe ,Ophthalmology ,medicine.anatomical_structure ,ROC Curve ,Photographic slides ,030221 ophthalmology & optometry ,Optometry ,Clinical Competence ,business ,Glaucoma, Open-Angle ,030217 neurology & neurosurgery ,Ai systems ,Optic disc - Abstract
Objectives: To evaluate the performance of a deep learning based Artificial Intelligence (AI) software for detection of glaucoma from stereoscopic optic disc photographs, and to compare this performance to the performance of a large cohort of ophthalmologists and optometrists. Methods: A retrospective study evaluating the diagnostic performance of an AI software (Pegasus v1.0, Visulytix Ltd., London UK) and comparing it to that of 243 European ophthalmologists and 208 British optometrists, as determined in previous studies, for the detection of glaucomatous optic neuropathy from 94 scanned stereoscopic photographic slides scanned into digital format. Results: Pegasus was able to detect glaucomatous optic neuropathy with an accuracy of 83.4% (95% CI: 77.5-89.2). This is comparable to an average ophthalmologist accuracy of 80.5% (95% CI: 67.2-93.8) and average optometrist accuracy of 80% (95% CI: 67-88) on the same images. In addition, the AI system had an intra-observer agreement (Cohen's Kappa, $\kappa$) of 0.74 (95% CI: 0.63-0.85), compared to 0.70 (range: -0.13-1.00; 95% CI: 0.67-0.73) and 0.71 (range: 0.08-1.00) for ophthalmologists and optometrists, respectively. There was no statistically significant difference between the performance of the deep learning system and ophthalmologists or optometrists. There was no statistically significant difference between the performance of the deep learning system and ophthalmologists or optometrists. Conclusion: The AI system obtained a diagnostic performance and repeatability comparable to that of the ophthalmologists and optometrists. We conclude that deep learning based AI systems, such as Pegasus, demonstrate significant promise in the assisted detection of glaucomatous optic neuropathy., Comment: 24 pages, 3 figures, 2 tables
- Published
- 2019