An interpretable AI model for recurrence prediction after surgery in gastrointestinal stromal tumour: an observational cohort study.

Background: There are several models that predict the risk of recurrence following resection of localised, primary gastrointestinal stromal tumour (GIST). However, assessment of calibration is not always feasible and when performed, calibration of current GIST models appears to be suboptimal. We aimed to develop a prognostic model to predict the recurrence of GIST after surgery with both good discrimination and calibration by uncovering and harnessing the non-linear relationships among variables that predict recurrence.
Methods: In this observational cohort study, the data of 395 adult patients who underwent complete resection (R0 or R1) of a localised, primary GIST in the pre-imatinib era at Memorial Sloan Kettering Cancer Center (NY, USA) (recruited 1982-2001) and a European consortium (Spanish Group for Research in Sarcomas, 80 sites) (recruited 1987-2011) were used to train an interpretable Artificial Intelligence (AI)-based model called Optimal Classification Trees (OCT). The OCT predicted the probability of recurrence after surgery by capturing non-linear relationships among predictors of recurrence. The data of an additional 596 patients from another European consortium (Polish Clinical GIST Registry, 7 sites) (recruited 1981-2013) who were also treated in the pre-imatinib era were used to externally validate the OCT predictions with regard to discrimination (Harrell's C-index and Brier score) and calibration (calibration curve, Brier score, and Hosmer-Lemeshow test). The calibration of the Memorial Sloan Kettering (MSK) GIST nomogram was used as a comparative gold standard. We also evaluated the clinical utility of the OCT and the MSK nomogram by performing a Decision Curve Analysis (DCA).
Findings: The internal cohort included 395 patients (median [IQR] age, 63 [54-71] years; 214 men [54.2%]) and the external cohort included 556 patients (median [IQR] age, 60 [52-68] years; 308 men [55.4%]). The Harrell's C-index of the OCT in the external validation cohort was greater than that of the MSK nomogram (0.805 (95% CI: 0.803-0.808) vs 0.788 (95% CI: 0.786-0.791), respectively). In the external validation cohort, the slope and intercept of the calibration curve of the main OCT were 1.041 and 0.038, respectively. In comparison, the slope and intercept of the calibration curve for the MSK nomogram was 0.681 and 0.032, respectively. The MSK nomogram overestimated the recurrence risk throughout the entire calibration curve. Of note, the Brier score was lower for the OCT compared to the MSK nomogram (0.147 vs 0.564, respectively), and the Hosmer-Lemeshow test was insignificant (P = 0.087) for the OCT model but significant (P < 0.001) for the MSK nomogram. Both results confirmed the superior discrimination and calibration of the OCT over the MSK nomogram. A decision curve analysis showed that the AI-based OCT model allowed for superior decision making compared to the MSK nomogram for both patients with 25-50% recurrence risk as well as those with >50% risk of recurrence.
Interpretation: We present the first prognostic models of recurrence risk in GIST that demonstrate excellent discrimination, calibration, and clinical utility on external validation. Additional studies for further validation are warranted. With further validation, these tools could potentially improve patient counseling and selection for adjuvant therapy.
Funding: The NCI SPORE in Soft Tissue Sarcoma and NCI Cancer Center Support Grants.
Competing Interests: JMB reports personal medical consulting fees from PharmaMar, GSK, Novartis, Amgen, Bayer, Roche, Lilly, Tecnofarma, Asofarma, Boehringer Ingelheim, support for attending meetings from Pfizer, PharmaMar, grants to his institution from GSK, PharmaMar, Novartis, EISAI, Lilly, Bayer, Lixte Biotechnology, Karyopharm Therapeutics, Deciphera, Blueprint Medicines, Nektar, Forma therapeutics, Amgen, Daiichi Sankyo, Immix BioPharma, BMS, Pfizer, Celgene, Arog, Adaptimmune, Rain Therapeutics, InnibRx, Ayala Pharmaceuticals, Philogen, Cebiotex, PTC Therapeutics, Springworks Therapeutics, and is on the Boards for TRACON PHARMA, PHARMAMAR, BOERHINGER, outside the submitted work. PC reports grants to her institution from Pfizer/Array, Deciphera, Ningbo NewBay, consulting fees from Deciphera, Ningbo NewBay, and is on the Advisory board and Steering Committee for Ningbo NewBay, and on the Steering Committee for Deciphera (unpaid), outside the submitted work. PR reports consulting fees from Bristol Myers Squibb, Merck Sharp & Dohme, Novartis, Pierre Fabre, Sanofi, Merck, Philogen and Blueprint Medicine, payment or honoraria for lectures, presentations, speakers’ bureaus, manuscript writing or educational events from Bristol Myers Squibb, Merck Sharp & Dohme, Novartis, Pierre Fabre, Sanofi, Merck, Astra Zeneca, Philogen and Blueprint Medicine, outside the submitted work. PS reports payment or honoraria for lectures, presentations, speakers’ bureaus, manuscript writing or educational events from BMS, Gillead, support for attending meetings and/or travel from Novartis, BMS, MSD, is on the Advisory Board for Sandoz, is a Committee Member of the European Society of Medical Oncology and a Board Member of the Polish Society of Clinical Oncology, owns Celon Pharma stocks, and received drugs for noncomercial clinical trial from Immutep, outside the submitted work. WDT reports personal fess from Eli Lilly, EMD Serono, Mundipharma, C4 Therapeutics, Daiichi Sankyo, Deciphera, Adcendo, Ayala Pharmaceuticals, Kowa, Servier, Bayer Pharmaceuticals, Epizyme, Cogent, Medpacto, Foghorn Therapeutics, Amgen, AmMax Bio, Boehringer Ingelheim, BioAtla, Inhibrx. In addition, WDT has a patent Companion Diagnostic for CDK4 inhibitors—14/854,329 pending to MSKCC/SKI, and a patent Enigma and CDH18 as companion Diagnostics for CDK4 inhibition—SKI2016-021-03 pending to MSKCC/SKI, outside the submitted work. WDT is on the Scientific Advisory Boards for Certis Oncology Solutions and Innova Therapeutics and owns Certis Oncology Solutions and Atropos Therapeutics stocks. JW reports grants to her institution from the UCSF Noyce Initiative for Digital Transformation in Computational Biology & Health, Computational Innovator Postdoctoral Fellowship Award. All other authors declare no competing interests.
(© 2023 Published by Elsevier Ltd.)