Performance of explainable artificial intelligence in guiding the management of patients with a pancreatic cyst.

Background/objectives: Pancreatic cyst management can be distilled into three separate pathways - discharge, monitoring or surgery- based on the risk of malignant transformation. This study compares the performance of artificial intelligence (AI) models to clinical care for this task.
Methods: Two explainable boosting machine (EBM) models were developed and evaluated using clinical features only, or clinical features and cyst fluid molecular markers (CFMM) using a publicly available dataset, consisting of 850 cases (median age 64; 65 % female) with independent training (429 cases) and holdout test cohorts (421 cases). There were 137 cysts with no malignant potential, 114 malignant cysts, and 599 IPMNs and MCNs.
Results: The EBM and EBM with CFMM models had higher accuracy for identifying patients requiring monitoring (0.88 and 0.82) and surgery (0.66 and 0.82) respectively compared with current clinical care (0.62 and 0.58). For discharge, the EBM with CFMM model had a higher accuracy (0.91) than either the EBM model (0.84) or current clinical care (0.86). In the cohort of patients who underwent surgical resection, use of the EBM-CFMM model would have decreased the number of unnecessary surgeries by 59 % (n = 92), increased correct surgeries by 7.5 % (n = 11), identified patients who require monitoring by 122 % (n = 76), and increased the number of patients correctly classified for discharge by 138 % (n = 18) compared to clinical care.
Conclusions: EBM models had greater sensitivity and specificity for identifying the correct management compared with either clinical management or previous AI models. The model predictions are demonstrated to be interpretable by clinicians.
Competing Interests: Declaration of competing interest E.K.F. has educational grant support with GE Medical, Siemens Healthineers research grant support, HipGraphics Inc (co-founder and shareholder), Exact Sciences (consultant), Imaging Endpoints (consultant). A.M.L. is a consultant with Exact Sciences. K.W.K. is also a member of the Scientific Advisory Boards of Eisai-Morphotek, Syxmex-Inostics, CAGE, and NeoPhore. These companies, as well as other companies, have licensed technologies from Johns Hopkins University, on which K.W.K. is an inventor. These licenses and relationships are associated with equity or royalty payments to K.W.K. The terms of these arrangements are being managed by Johns Hopkins University in accordance with its conflict of interest policies. D.S.K. is a consultant and equity holder in PAIGE. AI. The terms of all these arrangements are being managed by Johns Hopkins University in accordance with its conflict of interest policies. The following patents are related to this work: Safe Sequencing System US201161476150P, Rapid Aneuploidy Detection US201261615535P, Mutations in pancreatic neoplasms US9976184B2, and Differential identification of pancreatic cysts US9637796B2.
(Copyright © 2024 The Authors. Published by Elsevier B.V. All rights reserved.)