Gadi SRV, Mori Y, Misawa M, East JE, Hassan C, Repici A, Byrne MF, von Renteln D, Hewett DG, Wang P, Saito Y, Matsubayashi CO, Ahmad OF, Sharma P, Gross SA, Sengupta N, Mansour N, Cherubini A, Dinh NN, Xiao X, Mountney P, González-Bueno Puyal J, Little G, LaRocco S, Conjeti S, Seibt H, Zur D, Shimada H, Berzin TM, and Glissen Brown JR
Background and Aims: Multiple computer-aided detection (CADe) software programs have now achieved regulatory approval in the United States, Europe, and Asia and are being used in routine clinical practice to support colorectal cancer screening. There is uncertainty regarding how different CADe algorithms may perform. No objective methodology exists for comparing different algorithms. We aimed to identify priority scoring metrics for CADe evaluation and comparison., Methods: A modified Delphi approach was used. Twenty-five global leaders in CADe in colonoscopy, including endoscopists, researchers, and industry representatives, participated in an online survey over the course of 8 months. Participants generated 121 scoring criteria, 54 of which were deemed within the study scope and distributed for review and asynchronous e-mail-based open comment. Participants then scored criteria in order of priority on a 5-point Likert scale during ranking round 1. The top 11 highest priority criteria were re-distributed, with another opportunity for open comment, followed by a final round of priority scoring to identify the final 6 criteria., Results: Mean priority scores for the 54 criteria ranged from 2.25 to 4.38 after the first ranking round. The top 11 criteria after round 1 of ranking yielded mean priority scores ranging from 3.04 to 4.16. The final 6 highest priority criteria, including a tie for first-place ranking, were (1, tied) sensitivity (average, 4.16) and (1, tied) separate and independent validation of the CADe algorithm (average, 4.16); (3) adenoma detection rate (average, 4.08); (4) false-positive rate (average, 4.00); (5) latency (average, 3.84); and (6) adenoma miss rate (average, 3.68)., Conclusions: This is the first reported international consensus statement of priority scoring metrics for CADe in colonoscopy. These scoring criteria should inform CADe software development and refinement. Future research should validate these metrics on a benchmark video data set to develop a validated scoring instrument., Competing Interests: Disclosure The following authors disclosed financial relationships: Y. Mori: speaker fees from and consultant for Olympus Corporation; and ownership interest in Cybernet System Corp. M. Misawa: speaker fees from Olympus Corporation; ownership interest in Cybernet System Corp. J.E. East: consultant for and ownership interest in Satisfai Health; and speaker fees from Medtronic. C. Hassan: consultant for Odin, Fujifilm, Olympus, and NEC. A. Repici: consultant for Medtronic; consultant for and research support from Fujifilm; research support and speaker fees from Boston Scientific and Norgine; and research support from Olympus and Pentax. M.F. Byrne: chairman and founder: Satisfai Health. D. von Renteln: research funding from and consultant for ERBE, Pendopharm, and Fujifilm; research funding from Ventage and Pentax; and consultant for Boston Scientific. D.G. Hewett: consultant for and speaker fees from Olympus Australia Pty Ltd and Fresenius Kabi Pty Ltd; and speaker fees from Boston Scientific Pty Ltd. C. Ogawa Matsubayashi: employee and Vice President of Innovation for AI Medical Service Inc. O.F. Ahmad: speaker fees from Olympus Corporation and Medtronic. P. Sharma: consultant for Olympus Corporation, Boston Scientific, and Salix Pharmaceuticals, Cipla, Medtronic, Takeda, Samsung Bioepis, CDx, and Exact Science; and research support from Fujifilm, Erbe Medical, and Ironwood Pharmaceuticals. S.A. Gross: consultant for Cook, Medtronic, and Olympus. N. Mansour: consultant for Iterative Health. A. Cherubini: Senior Vice President of Science, AI, and Data for Cosmo Intelligent Medical Devices. N.N. Dinh: President of Cosmo Intelligent Medical Devices. X. Xiao: Co-Founder and Chief Scientist and Equity holder in Wision Ltd. P. Mountney: CEO of Odin Vision. J. González-Bueno Puyal: senior research scientist for Odin Vision. D. Zur: CEO of Magentiq Eye. H. Seibt: Global Product Manager Digital Solutions of Pentax Medical Europe. G. Little: Executive Director and Global Head of Marketing, Digital Health, for Olympus Corporation of the Americas. S. LaRocco: Vice President and Global Head, Customer Solutions Business Unit, for Olympus Corporation of the Americas. S. Conjeti: Global Product Lead for Olympus Europa SE & Co. H. Shimada: Clinical Marketing Manager for Fujifilm Healthcare Americas Corporation. T.M. Berzin: consultant for Wision AI, Medtronic, Boston Scientific, Magentiq Eye, and RSIP Vision. J.R.G. Brown: consultant for Medtronic and Olympus. All other authors disclosed no financial relationships. J.E. East is funded by the National Institute for Health ResearchOxford Biomedical Research Centre. This project was funded via the Duke Internal Medicine Residency Faculty Resident Research Grant. This project received no industry funding sources. The views expressed are those of the authors and not necessarily those of the National Health Service, the National Institute for Health Research, or the Department of Health., (Copyright © 2024 American Society for Gastrointestinal Endoscopy. Published by Elsevier Inc. All rights reserved.)