Search

In this study, we introduce FEET, a standardized protocol designed to guide the development and benchmarking of foundation models. While numerous benchmark datasets exist for evaluating these models, we propose a structured evaluation protocol across three distinct scenarios to gain a comprehensive understanding of their practical performance. We define three primary use cases: frozen embeddings, few-shot embeddings, and fully fine-tuned embeddings. Each scenario is detailed and illustrated through two case studies: one in sentiment analysis and another in the medical domain, demonstrating how these evaluations provide a thorough assessment of foundation models' effectiveness in research applications. We recommend this protocol as a standard for future research aimed at advancing representation learning models., Comment: Findings paper presented at Machine Learning for Health (ML4H) symposium 2024, December 15-16, 2024, Vancouver, Canada, 11 pages

Background and purpose: IMPT faces challenges in lung cancer treatment, like maintaining plan robustness for moving tumors against setup, range errors, and interplay effects. Proton Arc Therapy (PAT) is an alternative to maintain target coverage, potentially improving organ at risk (OAR) sparing, reducing beam delivery time (BDT), and enhancing patient experience. We aim to perform a systematic plan comparison study between IMPT and ELSA-PAT to assess its potential for lung cancer treatment. Material and Methods: 14 Lung ELSA-PAT plans were compared retrospectively with IMPT plans. 4D worst-case minimax robust optimization was performed, including 84 scenarios (3%,3 mm). Dosimetry assessment included target (CTV) and important OARs, on nominal and worst-case scenarios. Most relevant normal tissue complication probabilities (NTCP), target coverage robustness against interplay effect and beam delivery time (BDT) were evaluated. Results: CTV D95% and D98% showed no significant difference in comparison. PAT demonstrated better conformality by 66% (p = 0.00012) but delivered a higher heart mean dose (HMD,23%). There was a 2% increase in NTCP 2-year mortality risk with PAT. Total BDT was comparable among techniques. IMPT was more robust than PAT against interplay effect, considering both D1% (1,0 $\pm$ 0.8 Gy vs 1.1 $\pm$ 1.4 Gy) and D98% bandwidths (0.9$\pm$0.9 Gy vs 1.1 $\pm$ 1.3 Gy). Interpretation: both techniques provide a similar level of dose coverage to the target volume. Although PAT improved dose conformality, higher HMD translated into increased heart toxicity, presumably due to chosen planning methodology and OAR proximity to target. Increased energy layers and spots raised PAT beam delivery time, although it could improve daily treatment workflow.