Towards effective machine learning in medical imaging analysis: A novel approach and expert evaluation of high-grade glioma 'ground truth' simulation on MRI

Purpose/objective(s) Gliomas are uniformly fatal brain tumours with significant neurological and quality of life detriment to patients. Improvement in outcomes has remained largely unchanged in nearly 20 years. MRI (magnetic resonance imaging) is often used in diagnosis and management. Machine learning analyses of large-scale MRI data are pivotal in advancing the diagnosis, management and improve outcomes in neuro-oncology. A common challenge to robust machine learning approaches is the lack of large ‘ground truth’ datasets in supervised learning for building classification and prediction models. The creation of these datasets relies on human-expert input and is time-consuming and subjective error-prone, limiting effective machine learning applications. Simulation of mechanistic aspects such as geometry, location and physical properties of brain tumours can generate large-scale ground-truth datasets allowing for comparison of analysis techniques in clinical applications. We aimed to develop a transparent and convenient method for building ‘ground truth’ presentations of simulated glioma lesions on anatomical MRI. Materials/methods The simulation workflow was created using the Feature Manipulation Engine (FME®), a data integration platform specializing in the spatial data processing. By compiling and integrating FME’s functions to read, integrate, transform, validate, save, and display MRI data, and experimenting with ways to manipulate the parameters concerning location, size, shape, and signal intensity with the presentations of glioma, we were able to generate simulated appearances of high-grade gliomas on gadolinium-based high-resolution 3D T1-weighted MRI (1 mm3). Data of patients with canonical high-grade tumours were used as real-world tumours for validating the accuracy of the simulation. Twenty raters who are experienced with brain tumour interpretation on MRI independently completed a survey, designed to distinguish simulated and real-world brain tumours. Sensitivity and specificity were calculated for assessing the performance of the approach with the binary classification of simulated vs real-world tumours. Correlation and regression were used in run time analysis, assessing the software toolset’s efficiency in producing different numbers of simulated lesions. Differences in the group means were examined using the non-parametric Kruskal-Wallis test. Results The simulation method was developed as an interpretable and useful workflow for the easy creation of tumour simulations and incorporation into 3D MRI. A linear increase in the running time and memory usage was observed with an increasing number of generated lesions. The respondents' accuracy rate ranged between 33.3 and 83.3 %. The sensitivity and specificity were low for a human expert to differentiate simulated lesions from real gliomas (0.43 and 0.58) or vice versa (0.65 and 0.62). The mean scores ranking the real-world gliomas did not differ between the simulated and real tumours. Conclusion The reliable and user-friendly software method can allow for robust simulation of high-grade glioma on MRI. Ongoing research efforts include optimizing the workflow for generating glioma datasets as well as adapting it to simulating additional MRI brain changes.