Search

Objective To evaluate the methodological rigor of radiomics-based studies using noninvasive imaging in ovarian setting. Methods Multiple medical literature archives (PubMed, Web of Science, and Scopus) were searched to retrieve original studies focused on computed tomography (CT), magnetic resonance imaging (MRI), ultrasound (US), or positron emission tomography (PET) radiomics for ovarian disorders’ assessment. Two researchers in consensus evaluated each investigation using the radiomics quality score (RQS). Subgroup analyses were performed to assess whether the total RQS varied according to first author category, study aim and topic, imaging modality, and journal quartile. Results From a total of 531 items, 63 investigations were finally included in the analysis. The studies were greatly focused (94%) on the field of oncology, with CT representing the most used imaging technique (41%). Overall, the papers achieved a median total RQS 6 (IQR, −0.5 to 11), corresponding to a percentage of 16.7% of the maximum score (IQR, 0–30.6%). The scoring was low especially due to the lack of prospective design and formal validation of the results. At subgroup analysis, the 4 studies not focused on oncological topic showed significantly lower quality scores than the others. Conclusions The overall methodological rigor of radiomics studies in the ovarian field is still not ideal, limiting the reproducibility of results and potential translation to clinical setting. More efforts towards a standardized methodology in the workflow are needed to allow radiomics to become a viable tool for clinical decision-making. Key Points • The 63 included studies using noninvasive imaging for ovarian applications were mostly focused on oncologic topic (94%). • The included investigations achieved a median total RQS 6 (IQR, −0.5 to 11), indicating poor methodological rigor. • The RQS was low especially due to the lack of prospective design and formal validation of the results.

Objectives We aimed to assess the performance of radiomics and machine learning (ML) for classification of non-cystic benign and malignant breast lesions on ultrasound images, compare ML’s accuracy with that of a breast radiologist, and verify if the radiologist’s performance is improved by using ML. Methods Our retrospective study included patients from two institutions. A total of 135 lesions from Institution 1 were used to train and test the ML model with cross-validation. Radiomic features were extracted from manually annotated images and underwent a multistep feature selection process. Not reproducible, low variance, and highly intercorrelated features were removed from the dataset. Then, 66 lesions from Institution 2 were used as an external test set for ML and to assess the performance of a radiologist without and with the aid of ML, using McNemar’s test. Results After feature selection, 10 of the 520 features extracted were employed to train a random forest algorithm. Its accuracy in the training set was 82% (standard deviation, SD, ± 6%), with an AUC of 0.90 (SD ± 0.06), while the performance on the test set was 82% (95% confidence intervals (CI) = 70–90%) with an AUC of 0.82 (95% CI = 0.70–0.93). It resulted in being significantly better than the baseline reference (p = 0.0098), but not different from the radiologist (79.4%, p = 0.815). The radiologist’s performance improved when using ML (80.2%), but not significantly (p = 0.508). Conclusions A radiomic analysis combined with ML showed promising results to differentiate benign from malignant breast lesions on ultrasound images. Key Points • Machine learning showed good accuracy in discriminating benign from malignant breast lesions • The machine learning classifier’s performance was comparable to that of a breast radiologist • The radiologist’s accuracy improved with machine learning, but not significantly

The aim of this study was to systematically review the literature and perform a meta-analysis of machine learning (ML) diagnostic accuracy studies focused on clinically significant prostate cancer (csPCa) identification on MRI. Multiple medical databases were systematically searched for studies on ML applications in csPCa identification up to July 31, 2019. Two reviewers screened all papers independently for eligibility. The area under the receiver operating characteristic curves (AUC) was pooled to quantify predictive accuracy. A random-effects model estimated overall effect size while statistical heterogeneity was assessed with the I2 value. A funnel plot was used to investigate publication bias. Subgroup analyses were performed based on reference standard (biopsy or radical prostatectomy) and ML type (deep and non-deep). After the final revision, 12 studies were included in the analysis. Statistical heterogeneity was high both in overall and in subgroup analyses. The overall pooled AUC for ML in csPCa identification was 0.86, with 0.81–0.91 95% confidence intervals (95%CI). The biopsy subgroup (n = 9) had a pooled AUC of 0.85 (95%CI = 0.79–0.91) while the radical prostatectomy one (n = 3) of 0.88 (95%CI = 0.76–0.99). Deep learning ML (n = 4) had a 0.78 AUC (95%CI = 0.69–0.86) while the remaining 8 had AUC = 0.90 (95%CI = 0.85–0.94). ML pipelines using prostate MRI to identify csPCa showed good accuracy and should be further investigated, possibly with better standardisation in design and reporting of results. • Overall pooled AUC was 0.86 with 0.81–0.91 95% confidence intervals. • In the reference standard subgroup analysis, algorithm accuracy was similar with pooled AUCs of 0.85 (0.79–0.91 95% confidence intervals) and 0.88 (0.76–0.99 95% confidence intervals) for studies employing biopsies and radical prostatectomy, respectively. • Deep learning pipelines performed worse (AUC = 0.78, 0.69–0.86 95% confidence intervals) than other approaches (AUC = 0.90, 0.85–0.94 95% confidence intervals).

• Interest in radiomics and machine learning is steadily increasing and is reflected both in research output and number of commercially available solutions. • Currently available commercial products using machine learning are often supported by limited evidence of clinical usefulness and studies are often of low methodological quality. • Ethical and regulatory issues remain open and hinder implementation of machine learning software packages in daily clinical practice.