Search

[Display omitted] • AI4MP-Challenge is the first AIFM multicentric experience on machine learning. • The main objective is to improve knowledge and skills of medical physicists on machine learning. • Encountered pitfalls: violation of independence assumption, computation errors, data imbalance. • Providing both cross-validation and an independent test helps to detect implementation issue. • The exclusion of non-robust features does not allow to significantly increase model stability. A novel and unconventional approach to a machine learning challenge was designed to spread knowledge, identify robust methods and highlight potential pitfalls about machine learning within the Medical Physics community. A public dataset comprising 41 radiomic features and 535 patients was employed to assess the potential of radiomics in distinguishing between primary lung tumors and metastases. Each participant developed two classification models using: (i) all features (base model); (ii) only robust features (robust model). Both models were validated with cross-validation and on unseen data. The population stability index (PSI) was used as diagnostic metric for implementation issues. Performance was compared to reference. Base and robust models were compared in terms of performance and stability (coefficient of variation (CoV) of prediction probabilities). PSI detected potential implementation errors in 70 % of models. The dataset exhibited strong imbalance. The average Gmean (i.e. an appropriate metric for imbalance) among all participants was 0.67 ± 0.01, significantly higher than reference Gmean of 0.50 ± 0.04. Robust models performances were slightly worse than base models (p < 0.05). Regarding stability, robust models exhibited lower median CoV on training set only. AI4MP-Challenge models overperformed the reference, significantly improving the Gmean. Exclusion of less-robust features did not improve model robustness and it should be avoided when confounding effects are absent. Other methods, like harmonization or data augmentation, should be evaluated. This study demonstrated how the collaborative effort to foster knowledge on machine learning among medical physicists, through interactive sessions and exchange of information among participants, can result in improved models. [ABSTRACT FROM AUTHOR]

• A reliable feature extraction pipeline for multiparametric MRI data was described. • Image preprocessing is a fundamental step in radiomic MRI analysis. • A set of robust features respect to the image preprocessing technique and features extraction settings was founded. • The robustness of radiomic features and the impact on their predictive power were investigated. Analysis pipelines based on the computation of radiomic features on medical images are widely used exploration tools across a large variety of image modalities. This study aims to define a robust processing pipeline based on Radiomics and Machine Learning (ML) to analyze multiparametric Magnetic Resonance Imaging (MRI) data to discriminate between high-grade (HGG) and low-grade (LGG) gliomas. The dataset consists of 158 multiparametric MRI of patients with brain tumor publicly available on The Cancer Imaging Archive, preprocessed by the BraTS organization committee. Three different types of image intensity normalization algorithms were applied and 107 features were extracted for each tumor region, setting the intensity values according to different discretization levels. The predictive power of radiomic features in the LGG versus HGG categorization was evaluated by using random forest classifiers. The impact of the normalization techniques and of the different settings in the image discretization was studied in terms of the classification performances. A set of MRI-reliable features was defined selecting the features extracted according to the most appropriate normalization and discretization settings. The results show that using MRI-reliable features improves the performance in glioma grade classification (AUC = 0.93 ± 0.05) with respect to the use of raw (AUC = 0.88 ± 0.08) and robust features (AUC = 0.83 ± 0.08), defined as those not depending on image normalization and intensity discretization. These results confirm that image normalization and intensity discretization strongly impact the performance of ML classifiers based on radiomic features. Thus, special attention should be provided in the image preprocessing step before typical radiomic and ML analysis are carried out. [ABSTRACT FROM AUTHOR]