Deep Learning Empowers Endoscopic Detection and Polyps Classification: A Multiple-Hospital Study

The present study aimed to develop an AI-based system for the detection and classification of polyps using colonoscopy images. A total of about 256,220 colonoscopy images from 5000 colorectal cancer patients were collected and processed. We used the CNN model for polyp detection and the EfficientNet-b0 model for polyp classification. Data were partitioned into training, validation and testing sets, with a 70%, 15% and 15% ratio, respectively. After the model was trained/validated/tested, to evaluate its performance rigorously, we conducted a further external validation using both prospective (n = 150) and retrospective (n = 385) approaches for data collection from 3 hospitals. The deep learning model performance with the testing set reached a state-of-the-art sensitivity and specificity of 0.9709 (95% CI: 0.9646–0.9757) and 0.9701 (95% CI: 0.9663–0.9749), respectively, for polyp detection. The polyp classification model attained an AUC of 0.9989 (95% CI: 0.9954–1.00). The external validation from 3 hospital results achieved 0.9516 (95% CI: 0.9295–0.9670) with the lesion-based sensitivity and a frame-based specificity of 0.9720 (95% CI: 0.9713–0.9726) for polyp detection. The model achieved an AUC of 0.9521 (95% CI: 0.9308–0.9734) for polyp classification. The high-performance, deep-learning-based system could be used in clinical practice to facilitate rapid, efficient and reliable decisions by physicians and endoscopists.