1. Multi-modal AI for comprehensive breast cancer prognostication
- Author
-
Witowski, Jan, Zeng, Ken, Cappadona, Joseph, Elayoubi, Jailan, Chiru, Elena Diana, Chan, Nancy, Kang, Young-Joon, Howard, Frederick, Ostrovnaya, Irina, Fernandez-Granda, Carlos, Schnabel, Freya, Ozerdem, Ugur, Liu, Kangning, Steinsnyder, Zoe, Thakore, Nitya, Sadic, Mohammad, Yeung, Frank, Liu, Elisa, Hill, Theodore, Swett, Benjamin, Rigau, Danielle, Clayburn, Andrew, Speirs, Valerie, Vetter, Marcus, Sojak, Lina, Soysal, Simone Muenst, Baumhoer, Daniel, Choucair, Khalil, Zong, Yu, Daoud, Lina, Saad, Anas, Abdulsattar, Waleed, Beydoun, Rafic, Pan, Jia-Wern, Makmur, Haslina, Teo, Soo-Hwang, Pak, Linda Ma, Angel, Victor, Zilenaite-Petrulaitiene, Dovile, Laurinavicius, Arvydas, Klar, Natalie, Piening, Brian D., Bifulco, Carlo, Jun, Sun-Young, Yi, Jae Pak, Lim, Su Hyun, Brufsky, Adam, Esteva, Francisco J., Pusztai, Lajos, LeCun, Yann, and Geras, Krzysztof J.
- Subjects
Computer Science - Artificial Intelligence ,Computer Science - Computer Vision and Pattern Recognition ,Electrical Engineering and Systems Science - Image and Video Processing - Abstract
Treatment selection in breast cancer is guided by molecular subtypes and clinical characteristics. Recurrence risk assessment plays a crucial role in personalizing treatment. Current methods, including genomic assays, have limited accuracy and clinical utility, leading to suboptimal decisions for many patients. We developed a test for breast cancer patient stratification based on digital pathology and clinical characteristics using novel AI methods. Specifically, we utilized a vision transformer-based pan-cancer foundation model trained with self-supervised learning to extract features from digitized H&E-stained slides. These features were integrated with clinical data to form a multi-modal AI test predicting cancer recurrence and death. The test was developed and evaluated using data from a total of 8,161 breast cancer patients across 15 cohorts originating from seven countries. Of these, 3,502 patients from five cohorts were used exclusively for evaluation, while the remaining patients were used for training. Our test accurately predicted our primary endpoint, disease-free interval, in the five external cohorts (C-index: 0.71 [0.68-0.75], HR: 3.63 [3.02-4.37, p<0.01]). In a direct comparison (N=858), the AI test was more accurate than Oncotype DX, the standard-of-care 21-gene assay, with a C-index of 0.67 [0.61-0.74] versus 0.61 [0.49-0.73], respectively. Additionally, the AI test added independent information to Oncotype DX in a multivariate analysis (HR: 3.11 [1.91-5.09, p<0.01)]). The test demonstrated robust accuracy across all major breast cancer subtypes, including TNBC (C-index: 0.71 [0.62-0.81], HR: 3.81 [2.35-6.17, p=0.02]), where no diagnostic tools are currently recommended by clinical guidelines. These results suggest that our AI test can improve accuracy, extend applicability to a wider range of patients, and enhance access to treatment selection tools.
- Published
- 2024