Search

Background and Aim: Early whitish gastric neoplasms can be easily misdiagnosed; differential diagnosis of gastric whitish lesions remains a challenge. We aim to build a deep learning (DL) model to diagnose whitish gastric neoplasms and explore the effect of adding domain knowledge in model construction., Methods: We collected 4558 images from two institutions to train and test models. We first developed two sole DL models (1 and 2) using supervised and semi-supervised algorithms. Then we selected diagnosis-related features through literature research and developed feature-extraction models to determine features including boundary, surface, roundness, depression, and location. Then predictions of the five feature-extraction models and sole DL model were combined and inputted into seven machine-learning (ML) based fitting-diagnosis models. The optimal model was selected as ENDOANGEL-WD (whitish-diagnosis) and compared with endoscopists., Results: Sole DL 2 had higher sensitivity (83.12% vs 68.67%, Bonferroni adjusted P = 0.024) than sole DL 1. Adding domain knowledge, the decision tree performed best among the seven ML models, achieving higher specificity than DL 1 (84.38% vs 72.27%, Bonferroni adjusted P < 0.05) and higher accuracy than DL 2 (80.47%, Bonferroni adjusted P < 0.001) and was selected as ENDOANGEL-WD. ENDOANGEL-WD showed better accuracy compared with 10 endoscopists (75.70%, P < 0.001)., Conclusions: We developed a novel system ENDOANGEL-WD combining domain knowledge and traditional DL to detect gastric whitish neoplasms. Adding domain knowledge improved the performance of traditional DL, which provided a novel solution for establishing diagnostic models for other rare diseases potentially., (© 2024 Journal of Gastroenterology and Hepatology Foundation and John Wiley & Sons Australia, Ltd.)

Background: Double-balloon enteroscopy (DBE) is a standard method for diagnosing and treating small bowel disease. However, DBE may yield false-negative results due to oversight or inexperience. We aim to develop a computer-aided diagnostic (CAD) system for the automatic detection and classification of small bowel abnormalities in DBE., Design and Methods: A total of 5201 images were collected from Renmin Hospital of Wuhan University to construct a detection model for localizing lesions during DBE, and 3021 images were collected to construct a classification model for classifying lesions into four classes, protruding lesion, diverticulum, erosion & ulcer and angioectasia. The performance of the two models was evaluated using 1318 normal images and 915 abnormal images and 65 videos from independent patients and then compared with that of 8 endoscopists. The standard answer was the expert consensus., Results: For the image test set, the detection model achieved a sensitivity of 92% (843/915) and an area under the curve (AUC) of 0.947, and the classification model achieved an accuracy of 86%. For the video test set, the accuracy of the system was significantly better than that of the endoscopists (85% vs. 77 ± 6%, p < 0.01). For the video test set, the proposed system was superior to novices and comparable to experts., Conclusions: We established a real-time CAD system for detecting and classifying small bowel lesions in DBE with favourable performance. ENDOANGEL-DBE has the potential to help endoscopists, especially novices, in clinical practice and may reduce the miss rate of small bowel lesions., (© 2023. The Author(s).)

Background: White light (WL) and weak-magnifying (WM) endoscopy are both important methods for diagnosing gastric neoplasms. This study constructed a deep-learning system named ENDOANGEL-MM (multi-modal) aimed at real-time diagnosing gastric neoplasms using WL and WM data., Methods: WL and WM images of a same lesion were combined into image-pairs. A total of 4201 images, 7436 image-pairs, and 162 videos were used for model construction and validation. Models 1-5 including two single-modal models (WL, WM) and three multi-modal models (data fusion on task-level, feature-level, and input-level) were constructed. The models were tested on three levels including images, videos, and prospective patients. The best model was selected for constructing ENDOANGEL-MM. We compared the performance between the models and endoscopists and conducted a diagnostic study to explore the ENDOANGEL-MM's assistance ability., Results: Model 4 (ENDOANGEL-MM) showed the best performance among five models. Model 2 performed better in single-modal models. The accuracy of ENDOANGEL-MM was higher than that of Model 2 in still images, real-time videos, and prospective patients. (86.54 vs 78.85%, P = 0.134; 90.00 vs 85.00%, P = 0.179; 93.55 vs 70.97%, P < 0.001). Model 2 and ENDOANGEL-MM outperformed endoscopists on WM data (85.00 vs 71.67%, P = 0.002) and multi-modal data (90.00 vs 76.17%, P = 0.002), significantly. With the assistance of ENDOANGEL-MM, the accuracy of non-experts improved significantly (85.75 vs 70.75%, P = 0.020), and performed no significant difference from experts (85.75 vs 89.00%, P = 0.159)., Conclusions: The multi-modal model constructed by feature-level fusion showed the best performance. ENDOANGEL-MM identified gastric neoplasms with good accuracy and has a potential role in real-clinic., (© 2022. The Author(s) under exclusive licence to The International Gastric Cancer Association and The Japanese Gastric Cancer Association.)

Background and Aims: We aimed to develop and validate a deep learning-based system that covers various aspects of early gastric cancer (EGC) diagnosis, including detecting gastric neoplasm, identifying EGC, and predicting EGC invasion depth and differentiation status. Herein, we provide a state-of-the-art comparison of the system with endoscopists using real-time videos in a nationwide human-machine competition., Methods: This multicenter, prospective, real-time, competitive comparative, diagnostic study enrolled consecutive patients who received magnifying narrow-band imaging endoscopy at the Peking University Cancer Hospital from June 9, 2020 to November 17, 2020. The offline competition was conducted in Wuhan, China, and the endoscopists and the system simultaneously read patients' videos and made diagnoses. The primary outcomes were sensitivity in detecting neoplasms and diagnosing EGCs., Results: One hundred videos, including 37 EGCs and 63 noncancerous lesions, were enrolled; 46 endoscopists from 44 hospitals in 19 provinces in China participated in the competition. The sensitivity rates of the system for detecting neoplasms and diagnosing EGCs were 87.81% and 100%, respectively, significantly higher than those of endoscopists (83.51% [95% confidence interval [CI], 81.23-85.79] and 87.13% [95% CI, 83.75-90.51], respectively). Accuracy rates of the system for predicting EGC invasion depth and differentiation status were 78.57% and 71.43%, respectively, slightly higher than those of endoscopists (63.75% [95% CI, 61.12-66.39] and 64.41% [95% CI, 60.65-68.16], respectively)., Conclusions: The system outperformed endoscopists in identifying EGCs and was comparable with endoscopists in predicting EGC invasion depth and differentiation status in videos. This deep learning-based system could be a powerful tool to assist endoscopists in EGC diagnosis in clinical practice., (Copyright © 2022 American Society for Gastrointestinal Endoscopy. Published by Elsevier Inc. All rights reserved.)

Background: White light endoscopy is a pivotal first-line tool for the detection of gastric neoplasms. However, gastric neoplasms can be missed during upper gastrointestinal endoscopy due to the subtle nature of these lesions and varying skill among endoscopists. Here, we aimed to evaluate the effect of an artificial intelligence (AI) system designed to detect focal lesions and diagnose gastric neoplasms on reducing the miss rate of gastric neoplasms in clinical practice., Methods: This single-centre, randomised controlled, tandem trial was done at Renmin Hospital of Wuhan University, China. We recruited consecutive patients (≥18 years old) undergoing routine upper gastrointestinal endoscopy for screening, surveillance, or investigation of symptoms. Same-day tandem upper gastrointestinal endoscopy was done where patients first underwent either AI-assisted (AI-first) or routine (routine-first) white light endoscopy, followed immediately by the other procedure, with targeted biopsies for all detected lesions taken at the end of the second examination. Patients were randomly assigned (1:1) to the AI-first or routine-first group using a computer-generated random numerical series and block randomisation (block size of four). Endoscopists were not blinded to randomisation status, whereas patients and pathologists were. The primary endpoint was the miss rate of gastric neoplasms and the analysis was done per protocol. This trial is registered with the Chinese Clinical Trial Registry, ChiCTR2000034453, and has been completed., Findings: Between July 6, 2020, and Dec 11, 2020, 907 patients were randomly assigned to the AI-first group and 905 to the routine-first group. The gastric neoplasm miss rate was significantly lower in the AI-first group than in the routine-first group (6·1%, 95% CI 1·6-17·9 [3/49] vs 27·3%, 15·5-43·0 [12/44]; relative risk 0·224, 95% CI 0·068-0·744; p=0·015). The only reported adverse event was bleeding from a target lesion after biopsy., Interpretation: The use of an AI system during upper gastrointestinal endoscopy significantly reduced the gastric neoplasm miss rate. AI-assisted endoscopy has the potential to improve the yield of gastric neoplasms by endoscopists., Funding: The Project of Hubei Provincial Clinical Research Center for Digestive Disease Minimally Invasive Incision and the Hubei Province Major Science and Technology Innovation Project., Competing Interests: Declaration of interests We declare no competing interests., (Copyright © 2021 Elsevier Ltd. All rights reserved.)

Background: The study aimed to construct an intelligent difficulty scoring and assistance system (DSAS) for endoscopic retrograde cholangiopancreatography (ERCP) treatment of common bile duct (CBD) stones., Methods: 1954 cholangiograms were collected from three hospitals for training and testing the DSAS. The D-LinkNet34 and U-Net were adopted to segment the CBD, stones, and duodenoscope. Based on the segmentation results, the stone size, distal CBD diameter, distal CBD arm, and distal CBD angulation were estimated. The performance of segmentation and estimation was assessed by mean intersection over union (mIoU) and average relative error. A technical difficulty scoring scale, which was used for assessing the technical difficulty of CBD stone removal, was developed and validated. We also analyzed the relationship between scores evaluated by the DSAS and clinical indicators including stone clearance rate and need for endoscopic papillary large-balloon dilation (EPLBD) and lithotripsy., Results: The mIoU values of the stone, CBD, and duodenoscope segmentation were 68.35 %, 86.42 %, and 95.85 %, respectively. The estimation performance of the DSAS was superior to nonexpert endoscopists. In addition, the technical difficulty scoring performance of the DSAS was more consistent with expert endoscopists than two nonexpert endoscopists. A DSAS assessment score ≥ 2 was correlated with lower stone clearance rates and more frequent EPLBD., Conclusions: An intelligent DSAS based on deep learning was developed. The DSAS could assist endoscopists by automatically scoring the technical difficulty of CBD stone extraction, and guiding the choice of therapeutic approach and appropriate accessories during ERCP., Competing Interests: The authors declare that they have no conflicts of interest., (Thieme. All rights reserved.)

Computed tomography (CT) is the preferred imaging method for diagnosing 2019 novel coronavirus (COVID19) pneumonia. We aimed to construct a system based on deep learning for detecting COVID-19 pneumonia on high resolution CT. For model development and validation, 46,096 anonymous images from 106 admitted patients, including 51 patients of laboratory confirmed COVID-19 pneumonia and 55 control patients of other diseases in Renmin Hospital of Wuhan University were retrospectively collected. Twenty-seven prospective consecutive patients in Renmin Hospital of Wuhan University were collected to evaluate the efficiency of radiologists against 2019-CoV pneumonia with that of the model. An external test was conducted in Qianjiang Central Hospital to estimate the system's robustness. The model achieved a per-patient accuracy of 95.24% and a per-image accuracy of 98.85% in internal retrospective dataset. For 27 internal prospective patients, the system achieved a comparable performance to that of expert radiologist. In external dataset, it achieved an accuracy of 96%. With the assistance of the model, the reading time of radiologists was greatly decreased by 65%. The deep learning model showed a comparable performance with expert radiologist, and greatly improved the efficiency of radiologists in clinical practice.

Background and Aims: The quality of bowel preparation is an important factor that can affect the effectiveness of a colonoscopy. Several tools, such as the Boston Bowel Preparation Scale (BBPS) and Ottawa Bowel Preparation Scale, have been developed to evaluate bowel preparation. However, understanding the differences between evaluation methods and consistently applying them can be challenging for endoscopists. There are also subjective biases and differences among endoscopists. Therefore, this study aimed to develop a novel, objective, and stable method for the assessment of bowel preparation through artificial intelligence., Methods: We used a deep convolutional neural network to develop this novel system. First, we retrospectively collected colonoscopy images to train the system and then compared its performance with endoscopists via a human-machine contest. Then, we applied this model to colonoscopy videos and developed a system named ENDOANGEL to provide bowel preparation scores every 30 seconds and to show the cumulative ratio of frames for each score during the withdrawal phase of the colonoscopy., Results: ENDOANGEL achieved 93.33% accuracy in the human-machine contest with 120 images, which was better than that of all endoscopists. Moreover, ENDOANGEL achieved 80.00% accuracy among 100 images with bubbles. In 20 colonoscopy videos, accuracy was 89.04%, and ENDOANGEL continuously showed the accumulated percentage of the images for different BBPS scores during the withdrawal phase and prompted us for bowel preparation scores every 30 seconds., Conclusions: We provided a novel and more accurate evaluation method for bowel preparation and developed an objective and stable system-ENDOANGEL-that could be applied reliably and steadily in clinical settings., (Copyright © 2020 American Society for Gastrointestinal Endoscopy. Published by Elsevier Inc. All rights reserved.)

White light endoscopy is the most pivotal tool for detecting early gastric neoplasms. Previous artificial intelligence (AI) systems were primarily unexplainable, affecting their clinical credibility and acceptability. We aimed to develop an explainable AI named ENDOANGEL-ED (explainable diagnosis) to solve this problem. A total of 4482 images and 296 videos with focal lesions from 3279 patients from eight hospitals were used for training, validating, and testing ENDOANGEL-ED. A traditional sole deep learning (DL) model was trained using the same dataset. The performance of ENDOANGEL-ED and sole DL was evaluated on six levels: internal and external images, internal and external videos, consecutive videos, and man–machine comparison with 77 endoscopists in videos. Furthermore, a multi-reader, multi-case study was conducted to evaluate the ENDOANGEL-ED's effectiveness. A scale was used to compare the overall acceptance of endoscopists to traditional and explainable AI systems. The ENDOANGEL-ED showed high performance in the image and video tests. In man–machine comparison, the accuracy of ENDOANGEL-ED was significantly higher than that of all endoscopists in internal (81.10% vs. 70.61%, p < 0.001) and external videos (88.24% vs. 78.49%, p < 0.001). With ENDOANGEL-ED's assistance, the accuracy of endoscopists significantly improved (70.61% vs. 79.63%, p < 0.001). Compared with the traditional AI, the explainable AI increased the endoscopists' trust and acceptance (4.42 vs. 3.74, p < 0.001; 4.52 vs. 4.00, p < 0.001). In conclusion, we developed a real-time explainable AI that showed high performance, higher clinical credibility, and acceptance than traditional DL models and greatly improved the diagnostic ability of endoscopists. [ABSTRACT FROM AUTHOR]

Bleeding risk factors for gastroesophageal varices (GEV) detected by endoscopy in cirrhotic patients determine the prophylactical treatment patients will undergo in the following 2 years. We propose a methodology for measuring the risk factors. We create an artificial intelligence system (ENDOANGEL-GEV) containing six models to segment GEV and to classify the grades (grades 1–3) and red color signs (RC, RC0-RC3) of varices. It also summarizes changes in the above results with region in real time. ENDOANGEL-GEV is trained using 6034 images from 1156 cirrhotic patients across three hospitals (dataset 1) and validated on multicenter datasets with 11009 images from 141 videos (dataset 2) and in a prospective study recruiting 161 cirrhotic patients from Renmin Hospital of Wuhan University (dataset 3). In dataset 1, ENDOANGEL-GEV achieves intersection over union values of 0.8087 for segmenting esophageal varices and 0.8141 for gastric varices. In dataset 2, the system maintains fairly accuracy across images from three hospitals. In dataset 3, ENDOANGEL-GEV surpasses attended endoscopists in detecting RC of GEV and classifying grades (p < 0.001). When ranking the risk of patients combined with the Child‒Pugh score, ENDOANGEL-GEV outperforms endoscopists for esophageal varices (p < 0.001) and shows comparable performance for gastric varices (p = 0.152). Compared with endoscopists, ENDOANGEL-GEV may help 12.31% (16/130) more patients receive the right intervention. We establish an interpretable system for the endoscopic diagnosis and risk stratification of GEV. It will assist in detecting the first bleeding risk factors accurately and expanding the scope of quantitative measurement of diseases. [ABSTRACT FROM AUTHOR]

3D object detection is a key task in environmental awareness, which plays a vital role in autonomous driving safety. Lidars and cameras are widely used sensors that provide complementary information for accurate 3D detection. However, due to the domain difference between the two modalities, it is challenging to leverage their respective strengths and fuse them perfectly. In this paper, an end-to-end detector termed BF3D is proposed, which integrates with the semantic sampling module, geometric point-image mapping module, and bi-directional attention fusion module. Specifically, the semantic sampling module incorporates a novel downsampling strategy to preserve more foreground points and pixels. Additionally, the geometric point-image mapping module is developed to find geometric correlation pixels of the point and take advantage of the high density of image features. We also introduce a bi-directional attention fusion module to combine useful information from the two modalities by attention mechanism. Extensive experiments demonstrate that BF3D outperforms both single- and multi-modal 3D detectors. Codes are available at: https://github.com/hustzyj/BF3D. • A novel semantic sampling module is proposed to preserve more foreground points. • The geometric point-image mapping module is designed to improve the utilization of dense image features. • A bi-directional attention module fuses multi-modal features effectively. • Our method shows outstanding detection performance compared to both single- and multi-modal detectors. [ABSTRACT FROM AUTHOR]