Search

Machine learning models for radiology benefit from large-scale data sets with high quality labels for abnormalities. We curated and analyzed a chest computed tomography (CT) data set of 36,316 volumes from 19,993 unique patients. This is the largest multiply-annotated volumetric medical imaging data set reported. To annotate this data set, we developed a rule-based method for automatically extracting abnormality labels from free-text radiology reports with an average F-score of 0.976 (min 0.941, max 1.0). We also developed a model for multi-organ, multi-disease classification of chest CT volumes that uses a deep convolutional neural network (CNN). This model reached a classification performance of AUROC >0.90 for 18 abnormalities, with an average AUROC of 0.773 for all 83 abnormalities, demonstrating the feasibility of learning from unfiltered whole volume CT data. We show that training on more labels improves performance significantly: for a subset of 9 labels - nodule, opacity, atelectasis, pleural effusion, consolidation, mass, pericardial effusion, cardiomegaly, and pneumothorax - the model's average AUROC increased by 10% when the number of training labels was increased from 9 to all 83. All code for volume preprocessing, automated label extraction, and the volume abnormality prediction model is publicly available. The 36,316 CT volumes and labels will also be made publicly available pending institutional approval., Competing Interests: Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2020 Elsevier B.V. All rights reserved.)

Background: Preliminary work has demonstrated that background parenchymal enhancement (BPE) assessed by radiologists is predictive of future breast cancer in women undergoing high-risk screening MRI. Algorithmically assessed measures of BPE offer a more precise and reproducible means of measuring BPE than human readers and thus might improve the predictive performance of future cancer development., Purpose: To determine if algorithmically extracted imaging features of BPE on screening breast MRI in high-risk women are associated with subsequent development of cancer., Study Type: Case-control study., Population: In all, 133 women at high risk for developing breast cancer; 46 of these patients developed breast cancer subsequently over a follow-up period of 2 years., Field Strength/sequence: 5 T or 3.0 T T 1 -weighted precontrast fat-saturated and nonfat-saturated sequences and postcontrast nonfat-saturated sequences., Assessment: Automatic features of BPE were extracted with a computer algorithm. Subjective BPE scores from five breast radiologists (blinded to clinical outcomes) were also available., Statistical Tests: Leave-one-out crossvalidation for a multivariate logistic regression model developed using the automatic features and receiver operating characteristic (ROC) analysis were performed to calculate the area under the curve (AUC). Comparison of automatic features and subjective features was performed using a generalized regression model and the P-value was obtained. Odds ratios for automatic and subjective features were compared., Results: The multivariate model discriminated patients who developed cancer from the patients who did not, with an AUC of 0.70 (95% confidence interval: 0.60-0.79, P < 0.001). The imaging features remained independently predictive of subsequent development of cancer (P < 0.003) when compared with the subjective BPE assessment of the readers., Data Conclusion: Automatically extracted BPE measurements may potentially be used to further stratify risk in patients undergoing high-risk screening MRI., Level of Evidence: 3 Technical Efficacy: Stage 5 J. Magn. Reson. Imaging 2019;50:456-464., (© 2019 International Society for Magnetic Resonance in Medicine.)

Purpose: To determine whether a multivariate machine learning-based model using computer-extracted features of pre-treatment dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) can predict pathologic complete response (pCR) to neoadjuvant therapy (NAT) in breast cancer patients., Methods: Institutional review board approval was obtained for this retrospective study of 288 breast cancer patients at our institution who received NAT and had a pre-treatment breast MRI. A comprehensive set of 529 radiomic features was extracted from each patient's pre-treatment MRI. The patients were divided into equal groups to form a training set and an independent test set. Two multivariate machine learning models (logistic regression and a support vector machine) based on imaging features were trained to predict pCR in (a) all patients with NAT, (b) patients with neoadjuvant chemotherapy (NACT), and (c) triple-negative or human epidermal growth factor receptor 2-positive (TN/HER2+) patients who had NAT. The multivariate models were tested using the independent test set, and the area under the receiver operating characteristics (ROC) curve (AUC) was calculated., Results: Out of the 288 patients, 64 achieved pCR. The AUC values for predicting pCR in TN/HER+ patients who received NAT were significant (0.707, 95% CI 0.582-0.833, p < 0.002)., Conclusions: The multivariate models based on pre-treatment MRI features were able to predict pCR in TN/HER2+ patients.

In this study, we systematically investigate the impact of class imbalance on classification performance of convolutional neural networks (CNNs) and compare frequently used methods to address the issue. Class imbalance is a common problem that has been comprehensively studied in classical machine learning, yet very limited systematic research is available in the context of deep learning. In our study, we use three benchmark datasets of increasing complexity, MNIST, CIFAR-10 and ImageNet, to investigate the effects of imbalance on classification and perform an extensive comparison of several methods to address the issue: oversampling, undersampling, two-phase training, and thresholding that compensates for prior class probabilities. Our main evaluation metric is area under the receiver operating characteristic curve (ROC AUC) adjusted to multi-class tasks since overall accuracy metric is associated with notable difficulties in the context of imbalanced data. Based on results from our experiments we conclude that (i) the effect of class imbalance on classification performance is detrimental; (ii) the method of addressing class imbalance that emerged as dominant in almost all analyzed scenarios was oversampling; (iii) oversampling should be applied to the level that completely eliminates the imbalance, whereas the optimal undersampling ratio depends on the extent of imbalance; (iv) as opposed to some classical machine learning models, oversampling does not cause overfitting of CNNs; (v) thresholding should be applied to compensate for prior class probabilities when overall number of properly classified cases is of interest., (Copyright © 2018 Elsevier Ltd. All rights reserved.)

Purpose: To determine whether multivariate machine learning models of algorithmically assessed magnetic resonance imaging (MRI) features from breast cancer patients are associated with Oncotype DX (ODX) test recurrence scores., Methods: A set of 261 female patients with invasive breast cancer, pre-operative dynamic contrast enhanced magnetic resonance (DCE-MR) images and available ODX score at our institution was identified. A computer algorithm extracted a comprehensive set of 529 features from the DCE-MR images of these patients. The set of patients was divided into a training set and a test set. Using the training set we developed two machine learning-based models to discriminate (1) high ODX scores from intermediate and low ODX scores, and (2) high and intermediate ODX scores from low ODX scores. The performance of these models was evaluated on the independent test set., Results: High against low and intermediate ODX scores were predicted by the multivariate model with AUC 0.77 (95% CI 0.56-0.98, p < 0.003). Low against intermediate and high ODX score was predicted with AUC 0.51 (95% CI 0.41-0.61, p = 0.75)., Conclusion: A moderate association between imaging and ODX score was identified. The evaluated models currently do not warrant replacement of ODX with imaging alone.

Background and Purpose: Convolutional neural networks (CNNs) are commonly used for segmentation of brain tumors. In this work, we assess the effect of cross-institutional training on the performance of CNNs., Methods: We selected 44 glioblastoma (GBM) patients from two institutions in The Cancer Imaging Archive dataset. The images were manually annotated by outlining each tumor component to form ground truth. To automatically segment the tumors in each patient, we trained three CNNs: (a) one using data for patients from the same institution as the test data, (b) one using data for the patients from the other institution and (c) one using data for the patients from both of the institutions. The performance of the trained models was evaluated using Dice similarity coefficients as well as Average Hausdorff Distance between the ground truth and automatic segmentations. The 10-fold cross-validation scheme was used to compare the performance of different approaches., Results: Performance of the model significantly decreased (P < 0.0001) when it was trained on data from a different institution (dice coefficients: 0.68 ± 0.19 and 0.59 ± 0.19) as compared to training with data from the same institution (dice coefficients: 0.72 ± 0.17 and 0.76 ± 0.12). This trend persisted for segmentation of the entire tumor as well as its individual components., Conclusions: There is a very strong effect of selecting data for training on performance of CNNs in a multi-institutional setting. Determination of the reasons behind this effect requires additional comprehensive investigation., (© 2018 American Association of Physicists in Medicine.)