Search

Abstract Objectives Repeatability is crucial for ensuring the generalizability and clinical utility of radiomics-based prognostic models. This study aims to investigate the repeatability of radiomic feature (RF) and its impact on the cross-institutional generalizability of the prognostic model for predicting local recurrence-free survival (LRFS) and overall survival (OS) in esophageal squamous cell cancer (ESCC) receiving definitive (chemo) radiotherapy (dCRT). Methods Nine hundred and twelve patients from two hospitals were included as training and external validation sets, respectively. Image perturbations were applied to contrast-enhanced computed tomography to generate perturbed images. Six thousand five hundred ten RFs from different feature types, bin widths, and filters were extracted from the original and perturbed images separately to evaluate RF repeatability by intraclass correlation coefficient (ICC). The high-repeatable and low-repeatable RF groups grouped by the median ICC were further analyzed separately by feature selection and multivariate Cox proportional hazards regression model for predicting LRFS and OS. Results First-order statistical features were more repeatable than texture features (median ICC: 0.70 vs 0.42–0.62). RFs from LoG had better repeatability than that of wavelet (median ICC: 0.70–0.84 vs 0.14–0.64). Features with smaller bin widths had higher repeatability (median ICC of 8–128: 0.65–0.47). For both LRFS and OS, the performance of the models based on high- and low-repeatable RFs remained stable in the training set with similar C-index (LRFS: 0.65 vs 0.67, p = 0.958; OS: 0.64 vs 0.65, p = 0.651), while the performance of the model based on the low-repeatable group was significantly lower than that based on the high-repeatable group in the external validation set (LRFS: 0.61 vs 0.67, p = 0.013; OS: 0.56 vs 0.63, p = 0.013). Conclusions Applying high-repeatable RFs in modeling could safeguard the cross-institutional generalizability of the prognostic model in ESCC. Critical relevance statement The exploration of repeatable RFs in different diseases and different types of imaging is conducive to promoting the proper use of radiomics in clinical research. Key Points The repeatability of RFs impacts the generalizability of the radiomic model. The high-repeatable RFs safeguard the cross-institutional generalizability of the model. Smaller bin width helps improve the repeatability of RFs. Graphical Abstract

Abstract Background Early prediction of pathological complete response (pCR) is important for deciding appropriate treatment strategies for patients. In this study, we aimed to quantify the dynamic characteristics of dynamic contrast-enhanced magnetic resonance images (DCE-MRI) and investigate its value to improve pCR prediction as well as its association with tumor heterogeneity in breast cancer patients. Methods The DCE-MRI, clinicopathologic record, and full transcriptomic data of 785 breast cancer patients receiving neoadjuvant chemotherapy were retrospectively included from a public dataset. Dynamic features of DCE-MRI were computed from extracted phase-varying radiomic feature series using 22 CAnonical Time-sereis CHaracteristics. Dynamic model and radiomic model were developed by logistic regression using dynamic features and traditional radiomic features respectively. Various combined models with clinical factors were also developed to find the optimal combination and the significance of each components was evaluated. All the models were evaluated in independent test set in terms of area under receiver operating characteristic curve (AUC). To explore the potential underlying biological mechanisms, radiogenomic analysis was implemented on patient subgroups stratified by dynamic model to identify differentially expressed genes (DEGs) and enriched pathways. Results A 10-feature dynamic model and a 4-feature radiomic model were developed (AUC = 0.688, 95%CI: 0.635–0.741 and AUC = 0.650, 95%CI: 0.595–0.705) and tested (AUC = 0.686, 95%CI: 0.594–0.778 and AUC = 0.626, 95%CI: 0.529–0.722), with the dynamic model showing slightly higher AUC (train p = 0.181, test p = 0.222). The combined model of clinical, radiomic, and dynamic achieved the highest AUC in pCR prediction (train: 0.769, 95%CI: 0.722–0.816 and test: 0.762, 95%CI: 0.679–0.845). Compared with clinical-radiomic combined model (train AUC = 0.716, 95%CI: 0.665–0.767 and test AUC = 0.695, 95%CI: 0.656–0.714), adding the dynamic component brought significant improvement in model performance (train p

Objective: Knee replacement (KR) is the last-resort treatment for knee osteoarthritis. Although radiographic evidence of tibiofemoral joint has been widely adopted for prognostication, patellofemoral joint has gained little attention and may hold additional value for further improvements. We aimed to quantitatively analyse patellofemoral joint through radiomics analysis of lateral view radiographs for improved KR risk prediction. Design: From the Multicenter Osteoarthritis Study dataset, we retrospectively retrieved the initial-visit lateral left knee radiographs of 2943 patients aged 50 to 79. They were split into training and test cohorts at a 2:1 ratio. A comprehensive set of radiomic features were extracted within the best-performing subregion of patellofemoral joint and combined into a radiomics score (RadScore). A KR risk score, derived from Kellgren-Lawrence grade (KLG) of tibiofemoral joint and RadScore of patellofemoral joint, was developed by multivariate Cox regression and assessed using time-dependent area under receiver operating characteristic curve (AUC). Results: While patellofemoral osteoarthritis (PFOA) was insignificant during multivariate analysis, RadScore was identified as an independent risk factor (multivariate Cox p-value < 0.001) for KR. The subgroup analysis revealed that RadScore was particularly effective in predicting rapid progressor (KR occurrence before 30 months) among early- (KLG < 2) and mid-stage (KLG ​= ​2) patients. Combining two joints radiographic information, the AUC reached 0.89/0.87 for predicting 60-month KR occurrence. Conclusions: The RadScore of the patellofemoral joint on lateral radiographs emerges as an independent prognostic factor for improving KR prognosis prediction. The KR risk score could be instrumental in managing progressive knee osteoarthritis interventions.

Abstract Image perturbation is a promising technique to assess radiomic feature repeatability, but whether it can achieve the same effect as test–retest imaging on model reliability is unknown. This study aimed to compare radiomic model reliability based on repeatable features determined by the two methods using four different classifiers. A 191-patient public breast cancer dataset with 71 test–retest scans was used with pre-determined 117 training and 74 testing samples. We collected apparent diffusion coefficient images and manual tumor segmentations for radiomic feature extraction. Random translations, rotations, and contour randomizations were performed on the training images, and intra-class correlation coefficient (ICC) was used to filter high repeatable features. We evaluated model reliability in both internal generalizability and robustness, which were quantified by training and testing AUC and prediction ICC. Higher testing performance was found at higher feature ICC thresholds, but it dropped significantly at ICC = 0.95 for the test–retest model. Similar optimal reliability can be achieved with testing AUC = 0.7–0.8 and prediction ICC > 0.9 at the ICC threshold of 0.9. It is recommended to include feature repeatability analysis using image perturbation in any radiomic study when test–retest is not feasible, but care should be taken when deciding the optimal feature repeatability criteria.

Radiomics, which integrates the comprehensive characterization of imaging phenotypes with machine learning algorithms, is increasingly recognized for its potential in the diagnosis and prognosis of oncological conditions. However, the repeatability and reproducibility of radiomic features are critical challenges that hinder their widespread clinical adoption. This review aims to address the paucity of discussion regarding the factors that influence the reproducibility and repeatability of radiomic features and their subsequent impact on the application of radiomic models. We provide a synthesis of the literature on the repeatability and reproducibility of CT/MR-based radiomic features, examining sources of variation, the number of reproducible features, and the availability of individual feature repeatability indices. We differentiate sources of variation into random effects, which are challenging to control but can be quantified through simulation methods such as perturbation, and biases, which arise from scanner variability and inter-reader differences and can significantly affect the generalizability of radiomic model performance in diverse settings. Four suggestions for repeatability and reproducibility studies are suggested: (1) detailed reporting of variation sources, (2) transparent disclosure of calculation parameters, (3) careful selection of suitable reliability indices, and (4) comprehensive reporting of reliability metrics. This review underscores the importance of random effects in feature selection and harmonizing biases between development and clinical application settings to facilitate the successful translation of radiomic models from research to clinical practice.

Abstract Background The immunohistochemical test (IHC) of HER2 and HR can provide prognostic information and treatment guidance for invasive breast cancer patients. We aimed to develop noninvasive image signatures ISHER2 and ISHR of HER2 and HR, respectively. We independently evaluate their repeatability, reproducibility, and association with pathological complete response (pCR) to neoadjuvant chemotherapy. Methods Pre-treatment DWI, IHC receptor status HER2/HR, and pCR to neoadjuvant chemotherapy of 222 patients from the multi-institutional ACRIN 6698 trial were retrospectively collected. They were pre-separated for development, independent validation, and test–retest. 1316 image features were extracted from DWI-derived ADC maps within manual tumor segmentations. ISHER2 and ISHR were developed by RIDGE logistic regression using non-redundant and test–retest reproducible features relevant to IHC receptor status. We evaluated their association with pCR using area under receiver operating curve (AUC) and odds ratio (OR) after binarization. Their reproducibility was further evaluated using the test–retest set with intra-class coefficient of correlation (ICC). Results A 5-feature ISHER2 targeting HER2 was developed (AUC = 0.70, 95% CI 0.59 to 0.82) and validated (AUC = 0.72, 95% CI 0.58 to 0.86) with high perturbation repeatability (ICC = 0.92) and test–retest reproducibility (ICC = 0.83). ISHR was developed using 5 features with higher association with HR during development (AUC = 0.75, 95% CI 0.66 to 0.84) and validation (AUC = 0.74, 95% CI 0.61 to 0.86) and similar repeatability (ICC = 0.91) and reproducibility (ICC = 0.82). Both image signatures showed significant associations with pCR with AUC of 0.65 (95% CI 0.50 to 0.80) for ISHER2 and 0.64 (95% CI 0.50 to 0.78) for ISHER2 in the validation cohort. Patients with high ISHER2 were more likely to achieve pCR to neoadjuvant chemotherapy with validation OR of 4.73 (95% CI 1.64 to 13.65, P value = 0.006). Low ISHR patients had higher pCR with OR = 0.29 (95% CI 0.10 to 0.81, P value = 0.021). Molecular subtypes derived from the image signatures showed comparable pCR prediction values to IHC-based molecular subtypes (P value > 0.05). Conclusion Robust ADC-based image signatures were developed and validated for noninvasive evaluation of IHC receptors HER2 and HR. We also confirmed their value in predicting treatment response to neoadjuvant chemotherapy. Further evaluations in treatment guidance are warranted to fully validate their potential as IHC surrogates.

Abstract Purpose The study aimed to predict acute radiation esophagitis (ARE) with grade ≥ 2 for patients with locally advanced lung cancer (LALC) treated with intensity-modulated radiation therapy (IMRT) using multi-omics features, including radiomics and dosiomics. Methods 161 patients with stage IIIA−IIIB LALC who received chemoradiotherapy (CRT) or radiotherapy by IMRT with a prescribed dose from 45 to 70 Gy from 2015 to 2019 were enrolled retrospectively. All the toxicity gradings were given following the Common Terminology Criteria for Adverse Events V4.0. Multi-omics features, including radiomics, dosiomics (including dose−volume histogram dosimetric parameters), were extracted based on the planning CT image and three-dimensional dose distribution. All data were randomly divided into training cohorts (N = 107) and testing cohorts (N = 54). In the training cohorts, features with reliably high outcome relevance and low redundancy were selected under random patient subsampling. Four classification models (using clinical factors (CF) only, using radiomics features (RFs) only, dosiomics features (DFs) only, and the hybrid features (HFs) containing clinical factors, radiomics and dosiomics) were constructed employing the Ridge classifier using two-thirds of randomly selected patients as the training cohort. The remaining patient was treated as the testing cohort. A series of models were built with 30 times training–testing splits. Their performances were assessed using the area under the ROC curve (AUC) and accuracy. Results Among all patients, 51 developed ARE grade ≥ 2, with an incidence of 31.7%. Next, 8990 radiomics and 213 dosiomics features were extracted, and 3, 6, 12, and 13 features remained after feature selection in the CF, DF, RF and DF models, respectively. The RF and HF models achieved similar classification performance, with the training and testing AUCs of 0.796 ± 0.023 (95% confidence interval (CI [0.79, 0.80])/0.744 ± 0.044 (95% CI [0.73, 0.76]) and 0.801 ± 0.022 (95% CI [0.79, 0.81]) (p = 0.74), respectively. The model performances using CF and DF features were poorer, with training and testing AUCs of 0.573 ± 0.026 (95% CI [0.56, 0.58])/ 0.509 ± 0.072 (95% CI [0.48, 0.53]) and 0.679 ± 0.027 (95% CI [0.67, 0.69])/0.604 ± 0.041 (95% CI [0.53, 0.63]) compared with the above two models (p

Abstract Radiomic model reliability is a central premise for its clinical translation. Presently, it is assessed using test–retest or external data, which, unfortunately, is often scarce in reality. Therefore, we aimed to develop a novel image perturbation-based method (IPBM) for the first of its kind toward building a reliable radiomic model. We first developed a radiomic prognostic model for head-and-neck cancer patients on a training (70%) and evaluated on a testing (30%) cohort using C-index. Subsequently, we applied the IPBM to CT images of both cohorts (Perturbed-Train and Perturbed-Test cohort) to generate 60 additional samples for both cohorts. Model reliability was assessed using intra-class correlation coefficient (ICC) to quantify consistency of the C-index among the 60 samples in the Perturbed-Train and Perturbed-Test cohorts. Besides, we re-trained the radiomic model using reliable RFs exclusively (ICC > 0.75) to validate the IPBM. Results showed moderate model reliability in Perturbed-Train (ICC: 0.565, 95%CI 0.518–0.615) and Perturbed-Test (ICC: 0.596, 95%CI 0.527–0.670) cohorts. An enhanced reliability of the re-trained model was observed in Perturbed-Train (ICC: 0.782, 95%CI 0.759–0.815) and Perturbed-Test (ICC: 0.825, 95%CI 0.782–0.867) cohorts, indicating validity of the IPBM. To conclude, we demonstrated capability of the IPBM toward building reliable radiomic models, providing community with a novel model reliability assessment strategy prior to prospective evaluation.

BackgroundUsing high robust radiomic features in modeling is recommended, yet its impact on radiomic model is unclear. This study evaluated the radiomic model’s robustness and generalizability after screening out low-robust features before radiomic modeling. The results were validated with four datasets and two clinically relevant tasks.Materials and methodsA total of 1,419 head-and-neck cancer patients’ computed tomography images, gross tumor volume segmentation, and clinically relevant outcomes (distant metastasis and local-regional recurrence) were collected from four publicly available datasets. The perturbation method was implemented to simulate images, and the radiomic feature robustness was quantified using intra-class correlation of coefficient (ICC). Three radiomic models were built using all features (ICC > 0), good-robust features (ICC > 0.75), and excellent-robust features (ICC > 0.95), respectively. A filter-based feature selection and Ridge classification method were used to construct the radiomic models. Model performance was assessed with both robustness and generalizability. The robustness of the model was evaluated by the ICC, and the generalizability of the model was quantified by the train-test difference of Area Under the Receiver Operating Characteristic Curve (AUC).ResultsThe average model robustness ICC improved significantly from 0.65 to 0.78 (P< 0.0001) using good-robust features and to 0.91 (P< 0.0001) using excellent-robust features. Model generalizability also showed a substantial increase, as a closer gap between training and testing AUC was observed where the mean train-test AUC difference was reduced from 0.21 to 0.18 (P< 0.001) in good-robust features and to 0.12 (P< 0.0001) in excellent-robust features. Furthermore, good-robust features yielded the best average AUC in the unseen datasets of 0.58 (P< 0.001) over four datasets and clinical outcomes.ConclusionsIncluding robust only features in radiomic modeling significantly improves model robustness and generalizability in unseen datasets. Yet, the robustness of radiomic model has to be verified despite building with robust radiomic features, and tightly restricted feature robustness may prevent the optimal model performance in the unseen dataset as it may lower the discrimination power of the model.

Purpose: This study investigates the impact of lung function on radiation pneumonitis prediction using a dual-omics analysis method.Methods: We retrospectively collected data of 126 stage III lung cancer patients treated with chemo-radiotherapy using intensity-modulated radiotherapy, including pre-treatment planning CT images, radiotherapy dose distribution, and contours of organs and structures. Lung perfusion functional images were generated using a previously developed deep learning method. The whole lung (WL) volume was divided into function-wise lung (FWL) regions based on the lung perfusion functional images. A total of 5,474 radiomics features and 213 dose features (including dosiomics features and dose-volume histogram factors) were extracted from the FWL and WL regions, respectively. The radiomics features (R), dose features (D), and combined dual-omics features (RD) were used for the analysis in each lung region of WL and FWL, labeled as WL-R, WL-D, WL-RD, FWL-R, FWL-D, and FWL-RD. The feature selection was carried out using ANOVA, followed by a statistical F-test and Pearson correlation test. Thirty times train-test splits were used to evaluate the predictability of each group. The overall average area under the receiver operating characteristic curve (AUC), accuracy, precision, recall, and f1-score were calculated to assess the performance of each group.Results: The FWL-RD achieved a significantly higher average AUC than the WL-RD group in the training (FWL-RD: 0.927 ± 0.031, WL-RD: 0.849 ± 0.064) and testing cohorts (FWL-RD: 0.885 ± 0.028, WL-RD: 0.762 ± 0.053, p < 0.001). When using radiomics features only, the FWL-R group yielded a better classification result than the model trained with WL-R features in the training (FWL-R: 0.919 ± 0.036, WL-R: 0.820 ± 0.052) and testing cohorts (FWL-R: 0.862 ± 0.028, WL-R: 0.750 ± 0.057, p < 0.001). The FWL-D group obtained an average AUC of 0.782 ± 0.032, obtaining a better classification performance than the WL-D feature-based model of 0.740 ± 0.028 in the training cohort, while no significant difference was observed in the testing cohort (FWL-D: 0.725 ± 0.064, WL-D: 0.710 ± 0.068, p = 0.54).Conclusion: The dual-omics features from different lung functional regions can improve the prediction of radiation pneumonitis for lung cancer patients under IMRT treatment. This function-wise dual-omics analysis method holds great promise to improve the prediction of radiation pneumonitis for lung cancer patients.

PurposeTo investigate the role of different multi-organ omics-based prediction models for pre-treatment prediction of Adaptive Radiotherapy (ART) eligibility in patients with nasopharyngeal carcinoma (NPC).Methods and MaterialsPre-treatment contrast-enhanced computed tomographic and magnetic resonance images, radiotherapy dose and contour data of 135 NPC patients treated at Hong Kong Queen Elizabeth Hospital were retrospectively analyzed for extraction of multi-omics features, namely Radiomics (R), Morphology (M), Dosiomics (D), and Contouromics (C), from a total of eight organ structures. During model development, patient cohort was divided into a training set and a hold-out test set in a ratio of 7 to 3 via 20 iterations. Four single-omics models (R, M, D, C) and four multi-omics models (RD, RC, RM, RMDC) were developed on the training data using Ridge and Multi-Kernel Learning (MKL) algorithm, respectively, under 10-fold cross validation, and evaluated on hold-out test data using average area under the receiver-operator-characteristics curve (AUC). The best-performing single-omics model was first determined by comparing the AUC distribution across the 20 iterations among the four single-omics models using two-sided student t-test, which was then retrained using MKL algorithm for a fair comparison with the four multi-omics models.ResultsThe R model significantly outperformed all other three single-omics models (all p-value

The COVID-19 pandemic has posed a significant global public health threat with an escalating number of new cases and death toll daily. The early detection of COVID-related CXR abnormality potentially allows the early isolation of suspected cases. Chest X-Ray (CXR) is a fast and highly accessible imaging modality. Recently, a number of CXR-based AI models have been developed for the automated detection of COVID-19. However, most existing models are difficult to interpret due to the use of incomprehensible deep features in their models. Confronted with this, we developed an interpretable TSK fuzzy system in this study for COVID-19 detection using radiomics features extracted from CXR images. There are two main contributions. (1) When TSK fuzzy systems are applied to classification tasks, the commonly used binary label matrix of training samples is transformed into a soft one in order to learn a more discriminant transformation matrix and hence improve classification accuracy. (2) Based on the assumption that the samples in the same class should be kept as close as possible when they are transformed into the label space, the compactness class graph is introduced to avoid overfitting caused by label matrix relaxation. Our proposed model for a multi-categorical classification task (COVID-19 vs. No-Findings vs. Pneumonia) was evaluated using 600 CXR images from publicly available datasets and compared against five state-of-the-art AI models in aspects of classification accuracy. Experimental findings showed that our model achieved classification accuracy of over 83%, which is better than the state-of-the-art models, while maintaining high interpretability.