Search

Technological advancements have enabled the extended investigation, development, and application of computational approaches in various domains, including health care. A burgeoning number of diagnostic, predictive, prognostic, and monitoring biomarkers are continuously being explored to improve clinical decision making in neuro-oncology. These advancements describe the increasing incorporation of artificial intelligence (AI) algorithms, including the use of radiomics. However, the broad applicability and clinical translation of AI are restricted by concerns about generalisability, reproducibility, scalability, and validation. This Policy Review intends to serve as the leading resource of recommendations for the standardisation and good clinical practice of AI approaches in health care, particularly in neuro-oncology. To this end, we investigate the repeatability, reproducibility, and stability of AI in response assessment in neuro-oncology in studies on factors affecting such computational approaches, and in publicly available open-source data and computational software tools facilitating these goals. The pathway for standardisation and validation of these approaches is discussed with the view of trustworthy AI enabling the next generation of clinical trials. We conclude with an outlook on the future of AI-enabled neuro-oncology., Competing Interests: Declaration of interests NG reported Blue Earth Diagnostics honorarium and Telix Pharmaceuticals participation on Advisory Board. TCB reported consulting fees from Microvention and fees for speakers bureaus of Siemens Healthineers, Bayer, and Medtronic. HJWLA reported receiving grants from the National Institutes of Health, European Union, and the US Department of Veterans Affairs, and consulting fees from Onc.AI, Sphera, and Love Health. WLB reported receiving an honorarium from Stryker. PL reported receiving a grant from the German Research Foundation. RJ reported receiving a grant from AIRS Medical. JML reported receiving grants from the National Institutes of Health, Department of Defense, and GE Healthcare. JCT reported receiving grants from Novocure and Munich Surgical Imaging; royalties from Springer Publishing; consulting fees from Novartis; an honorarium from the Italian Society for Neurology; and support to attend meetings of the Belgian Association for Neurooncology and Glioma Meeting Athens 2023. MV reported receiving grants from Infuseon Therapeutics, Oncosynergy, and DeNovo Pharma; drug delivery device royalties; consulting fees from Servier Pharma and BioDexa Pharma; and holds patents for drug delivery devices. RYH reported consulting fees from Nuvation Bio and Vysioneet. All other authors declare no competing interests., (Copyright © 2024 Elsevier Ltd. All rights reserved, including those for text and data mining, AI training, and similar technologies.)

The development, application, and benchmarking of artificial intelligence (AI) tools to improve diagnosis, prognostication, and therapy in neuro-oncology are increasing at a rapid pace. This Policy Review provides an overview and critical assessment of the work to date in this field, focusing on diagnostic AI models of key genomic markers, predictive AI models of response before and after therapy, and differentiation of true disease progression from treatment-related changes, which is a considerable challenge based on current clinical care in neuro-oncology. Furthermore, promising future directions, including the use of AI for automated response assessment in neuro-oncology, are discussed., Competing Interests: Declaration of interests JML received grants from the National Institutes of Health (NIH; P01CA118816), Department of Defense, and GE Healthcare. WLB reported receiving an honorarium from Stryker. PL received a grant from the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation; 428090865/SPP 2177). TCB received consulting fees from Microvention; is on speakers bureaus for Siemens Healthineers, Bayer, and Medtronic; and received grants from the Medical Research Council (MR/W021684/1) and the Wellcome Engineering and Physical Sciences Research Council Centre for Medical Engineering (WT 203,148/Z/16/Z). HJWLA received grants from the NIH (U24CA194354, U01CA190234, U01CA209414, R35CA22052, and U54CA27451), the European Research Council (866504), and the US Department of Veterans Affairs; and consulting fees from Onc.AI, Sphera, and Love Health. JCT received grants from Novocure and Munich Surgical Imaging; royalties from Springer Publishing; consulting fees from Novartis; an honorarium from the Italian Society for Neurology; and support to attend meetings of the Belgian Association for Neurooncology and Glioma Meeting Athens 2023. RJ received a grant from AIRS Medical. MAV received grants from Infuseon Therapeutics, Oncosynergy, and DeNovo Pharma; consulting fees from Servier Pharma and BioDexa Pharma; and holds patents and received royalties for drug delivery devices. RYH reported consulting fees from Nuvation Bio and Vysioneet. NG received a grant from the DFG (428090865/SPP 2177); honorarium from Blue Earth Diagnostics; and participated on an advisory board for Telix Pharmaceuticals. CD received grants from the NIH (U24CA189523, UL1TR001878, and R01NS042645). SB received grants from the NIH (U01CA242871 and UL1TR001878). RRC received grants from the NIH. MI received grants from the Musella Foundation Grant and the R&D Pilot Award from the University of Wisconsin-Madison. PT received grants from the NIH (U01CA248226, R01CA264017, and U01CA248226), the Veterans Affairs (VA) Merit Award (1I01BX005842-01A2), the DOD/PRCRP Career Development Award (W81XWH-18-1-0404), the Dana Foundation David Mahoney Neuroimaging Program, the V Foundation Translational Research Award, the Johnson & Johnson WiSTEM2D Award, the Ohio Third Frontier Technology Validation Fund, the Musella Foundation Grant, and the R&D Pilot Award from the University of Wisconsin-Madison. All other authors declare no competing interests., (Copyright © 2024 Elsevier Ltd. All rights reserved, including those for text and data mining, AI training, and similar technologies.)

People diagnosed with cancer and their formal and informal caregivers are increasingly faced with a deluge of complex information, thanks to rapid advancements in the type and volume of diagnostic, prognostic, and treatment data. This commentary discusses the opportunities and challenges that the society faces as we integrate large volumes of data into regular cancer care., (©2024 The Authors; Published by the American Association for Cancer Research.)

Background and Purpose: Understanding sex-based differences in patients with glioblastoma is necessary for accurate personalized treatment planning to improve patient outcomes. Our purpose was to investigate sex-specific differences in molecular, clinical, and radiologic tumor parameters, as well as survival outcomes in patients with glioblastoma, isocitrate dehydrogenase-1 wild-type ( IDH1 -WT), grade 4., Materials and Methods: Retrospective data of 1832 patients with glioblastoma, IDH1 -WT with comprehensive information on tumor parameters was acquired from the Radiomics Signatures for Precision Oncology in Glioblastoma consortium. Data imputation was performed for missing values. Sex-based differences in tumor parameters, such as age, molecular parameters, preoperative Karnofsky performance score (KPS), tumor volumes, epicenter, and laterality were assessed through nonparametric tests. Spatial atlases were generated by using preoperative MRI maps to visualize tumor characteristics. Survival time analysis was performed through log-rank tests and Cox proportional hazard analyses., Results: Glioblastoma was diagnosed at a median age of 64 years in women compared with 61.9 years in men (false discovery rate [FDR] = 0.003). Men had a higher KPS (above 80) as compared with women (60.4% women versus 69.7% men, FDR = 0.044). Women had lower tumor volumes in enhancing (16.7 cm 3 versus 20.6 cm 3 in men, FDR = 0.001), necrotic core (6.18 cm 3 versus 7.76 cm 3 in men, FDR = 0.001), and edema regions (46.9 cm 3 versus 59.2 cm 3 in men, FDR = 0.0001). The right temporal region was the most common tumor epicenter in the overall population. Right as well as left temporal lobes were more frequently involved in men. There were no sex-specific differences in survival outcomes and mortality ratios. Higher age, unmethylated O6-methylguanine-DNA-methyltransferase promoter and undergoing subtotal resection increased the mortality risk in both men and women., Conclusions: Our study demonstrates significant sex-based differences in clinical and radiologic tumor parameters of patients with glioblastoma. Sex is not an independent prognostic factor for survival outcomes and the tumor parameters influencing patient outcomes are identical for men and women., (© 2024 by American Journal of Neuroradiology.)

In the medical diagnostics domain, pathology and histology are pivotal for the precise identification of diseases. Digital histopathology, enhanced by automation, facilitates the efficient analysis of massive amount of biopsy images produced on a daily basis, streamlining the evaluation process. This study focuses in Stain Color Normalization (SCN) within a Whole-Slide Image (WSI) cohort, aiming to reduce batch biases. Building on published graphical method, this research demonstrates a mathematical population or data-driven method that optimizes the dependency on the number of reference WSIs and corresponding aggregate sums, thereby increasing SCN process efficiency. This method expedites the analysis of color convergence 50-fold by using stain vector Euclidean distance analysis, slashing the requirement for reference WSIs by more than half. The approach is validated through a tripartite methodology: 1) Stain vector euclidean distances analysis, 2) Distance computation timing, and 3) Qualitative and quantitative assessments of SCN across cancer tumors regions of interest. The results validate the performance of data-driven SCN method, thus potential to enhance the precision and reliability of computational pathology analyses. This advancement is poised to enhance diagnostic processes, therapeutic strategies, and patient prognosis., Competing Interests: All authors confirm that there are no conflicts of interest., (© 2024 The Authors.)

Motivation: Software is vital for the advancement of biology and medicine. Impact evaluations of scientific software have primarily emphasized traditional citation metrics of associated papers, despite these metrics inadequately capturing the dynamic picture of impact and despite challenges with improper citation., Results: To understand how software developers evaluate their tools, we conducted a survey of participants in the Informatics Technology for Cancer Research (ITCR) program funded by the National Cancer Institute (NCI). We found that although developers realize the value of more extensive metric collection, they find a lack of funding and time hindering. We also investigated software among this community for how often infrastructure that supports more nontraditional metrics were implemented and how this impacted rates of papers describing usage of the software. We found that infrastructure such as social media presence, more in-depth documentation, the presence of software health metrics, and clear information on how to contact developers seemed to be associated with increased mention rates. Analysing more diverse metrics can enable developers to better understand user engagement, justify continued funding, identify novel use cases, pinpoint improvement areas, and ultimately amplify their software's impact. Challenges are associated, including distorted or misleading metrics, as well as ethical and security concerns. More attention to nuances involved in capturing impact across the spectrum of biomedical software is needed. For funders and developers, we outline guidance based on experience from our community. By considering how we evaluate software, we can empower developers to create tools that more effectively accelerate biological and medical research progress., Availability and Implementation: More information about the analysis, as well as access to data and code is available at https://github.com/fhdsl/ITCR_Metrics_manuscript_website., (© The Author(s) 2024. Published by Oxford University Press.)

Artificial intelligence (AI) shows potential to improve health care by leveraging data to build models that can inform clinical workflows. However, access to large quantities of diverse data is needed to develop robust generalizable models. Data sharing across institutions is not always feasible due to legal, security, and privacy concerns. Federated learning (FL) allows for multi-institutional training of AI models, obviating data sharing, albeit with different security and privacy concerns. Specifically, insights exchanged during FL can leak information about institutional data. In addition, FL can introduce issues when there is limited trust among the entities performing the compute. With the growing adoption of FL in health care, it is imperative to elucidate the potential risks. We thus summarize privacy-preserving FL literature in this work with special regard to health care. We draw attention to threats and review mitigation approaches. We anticipate this review to become a health-care researcher's guide to security and privacy in FL., Competing Interests: The authors declare no competing interests., (© 2024 The Authors.)

The Radiological Society of North of America (RSNA) and the Medical Image Computing and Computer Assisted Intervention (MICCAI) Society have led a series of joint panels and seminars focused on the present impact and future directions of artificial intelligence (AI) in radiology. These conversations have collected viewpoints from multidisciplinary experts in radiology, medical imaging, and machine learning on the current clinical penetration of AI technology in radiology and how it is impacted by trust, reproducibility, explainability, and accountability. The collective points-both practical and philosophical-define the cultural changes for radiologists and AI scientists working together and describe the challenges ahead for AI technologies to meet broad approval. This article presents the perspectives of experts from MICCAI and RSNA on the clinical, cultural, computational, and regulatory considerations-coupled with recommended reading materials-essential to adopt AI technology successfully in radiology and, more generally, in clinical practice. The report emphasizes the importance of collaboration to improve clinical deployment, highlights the need to integrate clinical and medical imaging data, and introduces strategies to ensure smooth and incentivized integration. Keywords: Adults and Pediatrics, Computer Applications-General (Informatics), Diagnosis, Prognosis © RSNA, 2024.

The translation of AI-generated brain metastases (BM) segmentation into clinical practice relies heavily on diverse, high-quality annotated medical imaging datasets. The BraTS-METS 2023 challenge has gained momentum for testing and benchmarking algorithms using rigorously annotated internationally compiled real-world datasets. This study presents the results of the segmentation challenge and characterizes the challenging cases that impacted the performance of the winning algorithms. Untreated brain metastases on standard anatomic MRI sequences (T1, T2, FLAIR, T1PG) from eight contributed international datasets were annotated in stepwise method: published UNET algorithms, student, neuroradiologist, final approver neuroradiologist. Segmentations were ranked based on lesion-wise Dice and Hausdorff distance (HD95) scores. False positives (FP) and false negatives (FN) were rigorously penalized, receiving a score of 0 for Dice and a fixed penalty of 374 for HD95. The mean scores for the teams were calculated. Eight datasets comprising 1303 studies were annotated, with 402 studies (3076 lesions) released on Synapse as publicly available datasets to challenge competitors. Additionally, 31 studies (139 lesions) were held out for validation, and 59 studies (218 lesions) were used for testing. Segmentation accuracy was measured as rank across subjects, with the winning team achieving a LesionWise mean score of 7.9. The Dice score for the winning team was 0.65 ± 0.25. Common errors among the leading teams included false negatives for small lesions and misregistration of masks in space. The Dice scores and lesion detection rates of all algorithms diminished with decreasing tumor size, particularly for tumors smaller than 100 mm3. In conclusion, algorithms for BM segmentation require further refinement to balance high sensitivity in lesion detection with the minimization of false positives and negatives. The BraTS-METS 2023 challenge successfully curated well-annotated, diverse datasets and identified common errors, facilitating the translation of BM segmentation across varied clinical environments and providing personalized volumetric reports to patients undergoing BM treatment., Competing Interests: Conflicts of Interest No conflicts of interest to disclose.

Pediatric tumors of the central nervous system are the most common cause of cancer-related death in children. The five-year survival rate for high-grade gliomas in children is less than 20%. Due to their rarity, the diagnosis of these entities is often delayed, their treatment is mainly based on historic treatment concepts, and clinical trials require multi-institutional collaborations. The MICCAI Brain Tumor Segmentation (BraTS) Challenge is a landmark community benchmark event with a successful history of 12 years of resource creation for the segmentation and analysis of adult glioma. Here we present the CBTN-CONNECT-DIPGR-ASNR-MICCAI BraTS-PEDs 2023 challenge, which represents the first BraTS challenge focused on pediatric brain tumors with data acquired across multiple international consortia dedicated to pediatric neuro-oncology and clinical trials. The BraTS-PEDs 2023 challenge focuses on benchmarking the development of volumentric segmentation algorithms for pediatric brain glioma through standardized quantitative performance evaluation metrics utilized across the BraTS 2023 cluster of challenges. Models gaining knowledge from the BraTS-PEDs multi-parametric structural MRI (mpMRI) training data will be evaluated on separate validation and unseen test mpMRI dataof high-grade pediatric glioma. The CBTN-CONNECT-DIPGR-ASNR-MICCAI BraTS-PEDs 2023 challenge brings together clinicians and AI/imaging scientists to lead to faster development of automated segmentation techniques that could benefit clinical trials, and ultimately the care of children with brain tumors.

Introduction: Glioblastoma (GBM) is a highly aggressive malignant tumor of the central nervous system that displays varying molecular and morphological profiles, leading to challenging prognostic assessments. Stratifying GBM patients according to overall survival (OS) from H&E-stained whole slide images (WSI) using advanced computational methods is challenging, but with direct clinical implications., Methods: This work is focusing on GBM (IDH-wildtype, CNS WHO Gr.4) cases, identified from the TCGA-GBM and TCGA-LGG collections after considering the 2021 WHO classification criteria. The proposed approach starts with patch extraction in each WSI, followed by comprehensive patch-level curation to discard artifactual content, i.e., glass reflections, pen markings, dust on the slide, and tissue tearing. Each patch is then computationally described as a feature vector defined by a pre-trained VGG16 convolutional neural network. Principal component analysis provides a feature representation of reduced dimensionality, further facilitating identification of distinct groups of morphology patterns, via unsupervised k-means clustering., Results: The optimal number of clusters, according to cluster reproducibility and separability, is automatically determined based on the rand index and silhouette coefficient, respectively. Our proposed approach achieved prognostic stratification accuracy of 83.33% on a multi-institutional independent unseen hold-out test set with sensitivity and specificity of 83.33%., Discussion: We hypothesize that the quantification of these clusters of morphology patterns, reflect the tumor's spatial heterogeneity and yield prognostic relevant information to distinguish between short and long survivors using a decision tree classifier. The interpretability analysis of the obtained results can contribute to furthering and quantifying our understanding of GBM and potentially improving our diagnostic and prognostic predictions., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2024 Baheti, Innani, Nasrallah and Bakas.)

Meningiomas are the most common primary intracranial tumors and can be associated with significant morbidity and mortality. Radiologists, neurosurgeons, neuro-oncologists, and radiation oncologists rely on brain MRI for diagnosis, treatment planning, and longitudinal treatment monitoring. However, automated, objective, and quantitative tools for non-invasive assessment of meningiomas on multi-sequence MR images are not available. Here we present the BraTS Pre-operative Meningioma Dataset, as the largest multi-institutional expert annotated multilabel meningioma multi-sequence MR image dataset to date. This dataset includes 1,141 multi-sequence MR images from six sites, each with four structural MRI sequences (T2-, T2/FLAIR-, pre-contrast T1-, and post-contrast T1-weighted) accompanied by expert manually refined segmentations of three distinct meningioma sub-compartments: enhancing tumor, non-enhancing tumor, and surrounding non-enhancing T2/FLAIR hyperintensity. Basic demographic data are provided including age at time of initial imaging, sex, and CNS WHO grade. The goal of releasing this dataset is to facilitate the development of automated computational methods for meningioma segmentation and expedite their incorporation into clinical practice, ultimately targeting improvement in the care of meningioma patients., (© 2024. The Author(s).)

Objective . The patterns of brain activity associated with different brain processes can be used to identify different brain states and make behavioural predictions. However, the relevant features are not readily apparent and accessible. Our aim is to design a system for learning informative latent representations from multichannel recordings of ongoing EEG activity. Approach : We propose a novel differentiable decoding pipeline consisting of learnable filters and a pre-determined feature extraction module. Specifically, we introduce filters parameterized by generalized Gaussian functions that offer a smooth derivative for stable end-to-end model training and allow for learning interpretable features. For the feature module, we use signal magnitude and functional connectivity estimates. Main results. We demonstrate the utility of our model on a new EEG dataset of unprecedented size (i.e. 721 subjects), where we identify consistent trends of music perception and related individual differences. Furthermore, we train and apply our model in two additional datasets, specifically for emotion recognition on SEED and workload classification on simultaneous task EEG workload. The discovered features align well with previous neuroscience studies and offer new insights, such as marked differences in the functional connectivity profile between left and right temporal areas during music listening. This agrees with the specialisation of the temporal lobes regarding music perception proposed in the literature. Significance . The proposed method offers strong interpretability of learned features while reaching similar levels of accuracy achieved by black box deep learning models. This improved trustworthiness may promote the use of deep learning models in real world applications. The model code is available athttps://github.com/SMLudwig/EEGminer/., (Creative Commons Attribution license.)

Tumor adaptation or selection is thought to underlie therapy resistance in glioma. To investigate longitudinal epigenetic evolution of gliomas in response to therapeutic pressure, we performed an epigenomic analysis of 132 matched initial and recurrent tumors from patients with IDH-wildtype (IDHwt) and IDH-mutant (IDHmut) glioma. IDHwt gliomas showed a stable epigenome over time with relatively low levels of global methylation. The epigenome of IDHmut gliomas showed initial high levels of genome-wide DNA methylation that was progressively reduced to levels similar to those of IDHwt tumors. Integration of epigenomics, gene expression, and functional genomics identified HOXD13 as a master regulator of IDHmut astrocytoma evolution. Furthermore, relapse of IDHmut tumors was accompanied by histologic progression that was associated with survival, as validated in an independent cohort. Finally, the initial cell composition of the tumor microenvironment varied between IDHwt and IDHmut tumors and changed differentially following treatment, suggesting increased neoangiogenesis and T-cell infiltration upon treatment of IDHmut gliomas. This study provides one of the largest cohorts of paired longitudinal glioma samples with epigenomic, transcriptomic, and genomic profiling and suggests that treatment of IDHmut glioma is associated with epigenomic evolution toward an IDHwt-like phenotype., Significance: Standard treatments are related to loss of DNA methylation in IDHmut glioma, resulting in epigenetic activation of genes associated with tumor progression and alterations in the microenvironment that resemble treatment-naïve IDHwt glioma., (©2023 The Authors; Published by the American Association for Cancer Research.)

Resection and whole brain radiotherapy (WBRT) are standard treatments for brain metastases (BM) but are associated with cognitive side effects. Stereotactic radiosurgery (SRS) uses a targeted approach with less side effects than WBRT. SRS requires precise identification and delineation of BM. While artificial intelligence (AI) algorithms have been developed for this, their clinical adoption is limited due to poor model performance in the clinical setting. The limitations of algorithms are often due to the quality of datasets used for training the AI network. The purpose of this study was to create a large, heterogenous, annotated BM dataset for training and validation of AI models. We present a BM dataset of 200 patients with pretreatment T1, T1 post-contrast, T2, and FLAIR MR images. The dataset includes contrast-enhancing and necrotic 3D segmentations on T1 post-contrast and peritumoral edema 3D segmentations on FLAIR. Our dataset contains 975 contrast-enhancing lesions, many of which are sub centimeter, along with clinical and imaging information. We used a streamlined approach to database-building through a PACS-integrated segmentation workflow., (© 2024. The Author(s).)

Glioblastoma is a highly heterogeneous disease, with variations observed at both phenotypical and molecular levels. Personalized therapies would be facilitated by non-invasive in vivo approaches for characterizing this heterogeneity. In this study, we developed unsupervised joint machine learning between radiomic and genomic data, thereby identifying distinct glioblastoma subtypes. A retrospective cohort of 571 IDH-wildtype glioblastoma patients were included in the study, and pre-operative multi-parametric MRI scans and targeted next-generation sequencing (NGS) data were collected. L21-norm minimization was used to select a subset of 12 radiomic features from the MRI scans, and 13 key driver genes from the five main signal pathways most affected in glioblastoma were selected from the genomic data. Subtypes were identified using a joint learning approach called Anchor-based Partial Multi-modal Clustering on both radiomic and genomic modalities. Kaplan-Meier analysis identified three distinct glioblastoma subtypes: high-risk, medium-risk, and low-risk, based on overall survival outcome (p < 0.05, log-rank test; Hazard Ratio = 1.64, 95% CI 1.17-2.31, Cox proportional hazard model on high-risk and low-risk subtypes). The three subtypes displayed different phenotypical and molecular characteristics in terms of imaging histogram, co-occurrence of genes, and correlation between the two modalities. Our findings demonstrate the synergistic value of integrated radiomic signatures and molecular characteristics for glioblastoma subtyping. Joint learning on both modalities can aid in better understanding the molecular basis of phenotypical signatures of glioblastoma, and provide insights into the biological underpinnings of tumor formation and progression., (© 2024. The Author(s).)

Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint-a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases., (© 2024. Springer Nature America, Inc.)

Filters are commonly used to enhance specific structures and patterns in images, such as vessels or peritumoral regions, to enable clinical insights beyond the visible image using radiomics. However, their lack of standardization restricts reproducibility and clinical translation of radiomics decision support tools. In this special report, teams of researchers who developed radiomics software participated in a three-phase study (September 2020 to December 2022) to establish a standardized set of filters. The first two phases focused on finding reference filtered images and reference feature values for commonly used convolutional filters: mean, Laplacian of Gaussian, Laws and Gabor kernels, separable and nonseparable wavelets (including decomposed forms), and Riesz transformations. In the first phase, 15 teams used digital phantoms to establish 33 reference filtered images of 36 filter configurations. In phase 2, 11 teams used a chest CT image to derive reference values for 323 of 396 features computed from filtered images using 22 filter and image processing configurations. Reference filtered images and feature values for Riesz transformations were not established. Reproducibility of standardized convolutional filters was validated on a public data set of multimodal imaging (CT, fluorodeoxyglucose PET, and T1-weighted MRI) in 51 patients with soft-tissue sarcoma. At validation, reproducibility of 486 features computed from filtered images using nine configurations × three imaging modalities was assessed using the lower bounds of 95% CIs of intraclass correlation coefficients. Out of 486 features, 458 were found to be reproducible across nine teams with lower bounds of 95% CIs of intraclass correlation coefficients greater than 0.75. In conclusion, eight filter types were standardized with reference filtered images and reference feature values for verifying and calibrating radiomics software packages. A web-based tool is available for compliance checking., (© RSNA, 2024 See also the editorial by Huisman and D'Antonoli in this issue.)

Purpose: To propose a new quality scoring tool, METhodological RadiomICs Score (METRICS), to assess and improve research quality of radiomics studies., Methods: We conducted an online modified Delphi study with a group of international experts. It was performed in three consecutive stages: Stage#1, item preparation; Stage#2, panel discussion among EuSoMII Auditing Group members to identify the items to be voted; and Stage#3, four rounds of the modified Delphi exercise by panelists to determine the items eligible for the METRICS and their weights. The consensus threshold was 75%. Based on the median ranks derived from expert panel opinion and their rank-sum based conversion to importance scores, the category and item weights were calculated., Result: In total, 59 panelists from 19 countries participated in selection and ranking of the items and categories. Final METRICS tool included 30 items within 9 categories. According to their weights, the categories were in descending order of importance: study design, imaging data, image processing and feature extraction, metrics and comparison, testing, feature processing, preparation for modeling, segmentation, and open science. A web application and a repository were developed to streamline the calculation of the METRICS score and to collect feedback from the radiomics community., Conclusion: In this work, we developed a scoring tool for assessing the methodological quality of the radiomics research, with a large international panel and a modified Delphi protocol. With its conditional format to cover methodological variations, it provides a well-constructed framework for the key methodological concepts to assess the quality of radiomic research papers., Critical Relevance Statement: A quality assessment tool, METhodological RadiomICs Score (METRICS), is made available by a large group of international domain experts, with transparent methodology, aiming at evaluating and improving research quality in radiomics and machine learning., Key Points: • A methodological scoring tool, METRICS, was developed for assessing the quality of radiomics research, with a large international expert panel and a modified Delphi protocol. • The proposed scoring tool presents expert opinion-based importance weights of categories and items with a transparent methodology for the first time. • METRICS accounts for varying use cases, from handcrafted radiomics to entirely deep learning-based pipelines. • A web application has been developed to help with the calculation of the METRICS score ( https://metricsscore.github.io/metrics/METRICS.html ) and a repository created to collect feedback from the radiomics community ( https://github.com/metricsscore/metrics )., (© 2024. The Author(s).)