Search

Background: To address the numerous unmeet clinical needs, in recent years several Machine Learning models applied to medical images and clinical data have been introduced and developed. Even when they achieve encouraging results, they lack evolutionary progression, thus perpetuating their status as autonomous entities. We postulated that different algorithms which have been proposed in the literature to address the same diagnostic task, can be aggregated to enhance classification performance. We suggested a proof of concept to define an ensemble approach useful for integrating different algorithms proposed to solve the same clinical task. Methods: The proposed approach was developed starting from a public database consisting of radiomic features extracted from CT images relating to 535 patients suffering from lung cancer. Seven algorithms were trained independently by participants in the AI4MP working group on Artificial Intelligence of the Italian Association of Physics in Medicine to discriminate metastatic from non-metastatic patients. The classification scores generated by these algorithms are used to train SVM classifier. The Explainable Artificial Intelligence approach is applied to the final model. The ensemble model was validated following an 80–20 hold-out and leave-one-out scheme on the training set. Results: Compared to individual algorithms, a more accurate result was achieved. On the independent test the ensemble model achieved an accuracy of 0.78, a F1-score of 0.57 and a log-loss of 0.49. Shapley values representing the contribution of each algorithm to the final classification result of the ensemble model were calculated. This information represents an added value for the end user useful for evaluating the appropriateness of the classification result on a particular case. It also allows us to evaluate on a global level which methodological approaches of the individual algorithms are likely to have the most impact. Conclusion: Our proposal represents an innovative approach useful for integrating different algorithms that populate the literature and which lays the foundations for future evaluations in broader application scenarios. [ABSTRACT FROM AUTHOR]

[Display omitted] • AI4MP-Challenge is the first AIFM multicentric experience on machine learning. • The main objective is to improve knowledge and skills of medical physicists on machine learning. • Encountered pitfalls: violation of independence assumption, computation errors, data imbalance. • Providing both cross-validation and an independent test helps to detect implementation issue. • The exclusion of non-robust features does not allow to significantly increase model stability. A novel and unconventional approach to a machine learning challenge was designed to spread knowledge, identify robust methods and highlight potential pitfalls about machine learning within the Medical Physics community. A public dataset comprising 41 radiomic features and 535 patients was employed to assess the potential of radiomics in distinguishing between primary lung tumors and metastases. Each participant developed two classification models using: (i) all features (base model); (ii) only robust features (robust model). Both models were validated with cross-validation and on unseen data. The population stability index (PSI) was used as diagnostic metric for implementation issues. Performance was compared to reference. Base and robust models were compared in terms of performance and stability (coefficient of variation (CoV) of prediction probabilities). PSI detected potential implementation errors in 70 % of models. The dataset exhibited strong imbalance. The average Gmean (i.e. an appropriate metric for imbalance) among all participants was 0.67 ± 0.01, significantly higher than reference Gmean of 0.50 ± 0.04. Robust models performances were slightly worse than base models (p < 0.05). Regarding stability, robust models exhibited lower median CoV on training set only. AI4MP-Challenge models overperformed the reference, significantly improving the Gmean. Exclusion of less-robust features did not improve model robustness and it should be avoided when confounding effects are absent. Other methods, like harmonization or data augmentation, should be evaluated. This study demonstrated how the collaborative effort to foster knowledge on machine learning among medical physicists, through interactive sessions and exchange of information among participants, can result in improved models. [ABSTRACT FROM AUTHOR]

[-2]pro-prostate-specific antigen (2pPSA), a proform of PSA, is a new marker in patients at risk of prostate cancer. We explored the potential role of 2pPSA in the identification of patients with metastatic progression following radical prostatectomy for prostate cancer. Seventy-six patients with biochemical (PSA) recurrence following radical prostatectomy were studied retrospectively. Diagnostic imaging performed at the time of biochemical recurrence confirmed metastatic disease in 31 of the 76 patients. Serum samples were collected and stored at the time of imaging-confirmed metastatic progression or at the most recent procedure for patients with negative imaging. Median values of PSA, free PSA (fPSA), %fPSA, 2pPSA and prostate health index (PHI) were compared between metastatic and non-metastatic patients by the Mann-Whitney U test. The results of each test were then correlated with metastatic status by univariate and multivariate logistic regression analysis. PSA, fPSA, %fPSA, 2pPSA serum concentrations and PHI values were statistically significantly higher in patients with metastatic disease. Results of the multivariate analysis revealed that 2pPSA remained a statistically significant predictor of imaging-proven metastatic prostate cancer among patients with biochemical recurrence. At a cut-off value of 12.25 pg/ml, 2pPSA outperformed the other markers in terms of sensitivity and specificity (97 and 80%, respectively) with respect to imaging-confirmed metastatic progression. This is the first study suggesting that 2pPSA predicts diagnostic imaging-proven metastatic disease in previously resected prostate cancer patients with biochemical recurrence. Our results merit validation in a prospective study.