Search

Background and objective: Micro-ultrasound (MUS) uses a high-frequency transducer with superior resolution to conventional ultrasound, which may differentiate prostate cancer from normal tissue and thereby allow targeted biopsy. Preliminary evidence has shown comparable sensitivity to magnetic resonance imaging (MRI), but consistency between users has yet to be described. Our objective was to assess agreement of MUS interpretation across multiple readers. Methods: After institutional review board approval, we prospectively collected MUS images for 57 patients referred for prostate biopsy after multiparametric MRI from 2022 to 2023. MUS images were interpreted by six urologists at four institutions with varying experience (range 2–6 yr). Readers were blinded to MRI results and clinical data. The primary outcome was reader agreement on the locations of suspicious lesions, measured in terms of Light’s κ and positive percent agreement (PPA). Reader sensitivity for identification of grade group (GG) ≥2 prostate cancer was a secondary outcome. Key findings and limitations: Analysis revealed a κ value of 0.30 (95% confidence interval [CI] 0.21–0.39). PPA was 33% (95% CI 25–42%). The mean patient-level sensitivity for GG ≥2 cancer was 0.66 ± 0.05 overall and 0.87 ± 0.09 when cases with anterior lesions were excluded. Readers were 12 times more likely to detect higher-grade cancers (GG ≥3), with higher levels of agreement for this subgroup (κ 0.41, PPA 45%). Key limitations include the inability to prospectively biopsy reader-delineated targets and the inability of readers to perform live transducer maneuvers. Conclusions and clinical implications: Inter-reader agreement on the location of suspicious lesions on MUS is lower than rates previously reported for MRI. MUS sensitivity for cancer in the anterior gland is lacking. Patient summary: The ability to find cancer on imaging scans can vary between doctors. We found that there was frequent disagreement on the location of prostate cancer when doctors were using a new high-resolution scan method called micro-ultrasound. This suggests that the performance of micro-ultrasound is not yet consistent enough to replace MRI (magnetic resonance imaging) for diagnosis of prostate cancer.

PURPOSE:To determine the inter-reader agreement of magnetic resonance imaging proton density fat fraction (PDFF) and its longitudinal change in a clinical trial of adults with nonalcoholic steatohepatitis (NASH). STUDY TYPE:We performed a secondary analysis of a placebo-controlled randomized clinical trial of a bile acid sequestrant in 45 adults with NASH. A six-echo spoiled gradient-recalled-echo magnitude-based fat quantification technique was performed at 3 T. Three independent readers measured MRI-PDFF by placing one primary and two additional regions of interest (ROIs) in each segment at both time points. Cross-sectional agreement between the three readers was evaluated using intra-class correlation coefficients (ICCs) and coefficients of variation (CV). Additionally, we used Bland-Altman analyses to examine pairwise agreement between the three readers at baseline, end of treatment (EOT), and for longitudinal change. RESULTS:Using all ROIs by all readers, mean PDFF at baseline, at EOT, and mean change in PDFF was 16.1%, 16.0%, and 0.07%, respectively. The 27-ROI PDFF measurements had 0.998 ICC and 1.8% CV at baseline, 0.998 ICC and 1.8% CV at EOT, and 0.997 ICC for longitudinal change. The 9-ROI PDFF measurements had corresponding values of 0.997 and 2.6%, 0.996 and 2.4%, and 0.994. Using 27 ROIs, the magnitude of the bias between readers for whole-liver PDFF measurement ranged from 0.03% to 0.06% points at baseline, 0.01% to 0.07% points at EOT, and 0.01% to 0.02% points for longitudinal change. CONCLUSION:Inter-reader agreement for measuring whole-liver PDFF and its longitudinal change is high. 9-ROI measurements have only slightly lower agreement than 27-ROI measurements.

To evaluate the diagnostic agreement between readers in VI-RADS interpretation to detect muscle-invasive bladder cancer (MIBC) preoperatively, we conducted a systematic review and meta-analysis of the available data. Scopus, PubMed, Web of Science, and Embase databases were systematically searched up to November 13, 2021. Case reports, review articles, editorials, and studies with insufficient data were eliminated. The Quality Appraisal of the Diagnostic Reliability Checklist was used to assess the risk of bias. The degree of agreement was determined by Cohen's kappa coefficient (κ) for comparison of data. The heterogeneity of these studies was explored using subgroup analysis and meta-regression analysis. The level of confidence was set at 0.05. All analyses were conducted in STATA 16.0. Overall, 19 eligible studies, consisting of 2439 participants, were included in this meta-analysis. The inter-reader agreement for VI-RADS in MIBC detection ranged from κ of 0.45 to 0.96 among included studies. The pooled inter-reader reliability was calculated as 0.76 [95% CI 0.73–0.80; I2 = 92.13%, Q(50) = 635.08, p < 0.01]. Sources of heterogeneity included magnetic strength, T2WI slice thickness, number of readers, sample size, study design, number of centers, year of publication, proportion of male patients, and mean age. There is substantial reliability in VI-RADS interpretation for MIBC among radiologists with various levels of expertise. The high degree of inter-reader agreement for MIBC detection supports the implementation of VI-RADS in routine clinical practice for the staging paradigm of bladder cancer. [ABSTRACT FROM AUTHOR]

Objective: To investigate the impact of an artificial intelligence (AI) software and quantitative ADC (qADC) on the inter-reader agreement, diagnostic performance, and reporting times of prostate biparametric MRI (bpMRI) for experienced and inexperienced readers. Materials and methods: A total of 170 multiparametric MRI (mpMRI) of patients with suspicion of prostate cancer (PCa) were retrospectively reviewed by one experienced and one inexperienced reader three times, following a wash-out period. First, only the bpMRI sequences, including T2-weighted imaging (T2WI), diffusion-weighted imaging (DWI) sequences, and apparent diffusion coefficient (ADC) maps, were used. Then, bpMRI and quantitative ADC values were used. Lastly, bpMRI and the AI software were used. Inter-reader agreement between the two readers and between each reader and the mpMRI original reports was calculated. Detection rates and reporting times were calculated for each group. Results: Inter-reader agreement with respect to mpMRI was moderate for bpMRI, Quantib, and qADC for both the inexperienced (weighted k of 0.42, 0.45, and 0.41, respectively) and the experienced radiologists (weighted k of 0.44, 0.46, and 0.42, respectively). Detection rate of PCa was similar between the inexperienced (0.24, 0.26, and 0.23) and the experienced reader (0.26, 0.27 and 0.27), for bpMRI, Quantib, and qADC, respectively. Reporting times were lower for Quantib (8.23, 7.11, and 9.87 min for the inexperienced reader and 5.62, 5.07, and 6.21 min for the experienced reader, for bpMRI, Quantib, and qADC, respectively). Conclusions: AI and qADC did not have a significant impact on the diagnostic performance of both readers. The use of Quantib was associated with lower reporting times. [ABSTRACT FROM AUTHOR]

Objectives: We aimed to systematically assess the inter-reader agreement of the Prostate Imaging Reporting and Data System Version (PI-RADS) v2.1 for the detection of prostate cancer (PCa). Methods: We included studies reporting inter-reader agreement of different radiologists that applied PI-RADS v2.1 for the detection of PCa. Quality assessment of the included studies was performed with the Guidelines for Reporting Reliability and Agreement Studies. The summary estimates of the inter-reader agreement were pooled with the random-effect model and categorized (from slight to almost perfect) according to the kappa (κ) value. Multiple subgroup analyses and meta-regression were performed to explore various clinical settings. Results: A total of 12 studies comprising 2475 patients were included. The pooled inter-reader agreement for whole gland was κ=0.65 (95% CI 0.56-0.73), and for transitional zone (TZ) lesions was κ=0.62 (95% CI 0.51-0.72). There was substantial heterogeneity presented throughout the studies (I²= 95.6%), and meta-regression analyses revealed that only readers' experience (<5 years vs. ≥5 years) was the significant factor associated with heterogeneity (P<0.01). In studies providing head-to-head comparison, there was no significant difference in inter-reader agreement between PI-RADS v2.1 and v2.0 for both the whole gland (0.64 vs. 0.57, p=0.37), and TZ (0.61 vs. 0.59, p=0.81). Conclusions: PI-RADS v2.1 demonstrated substantial inter-reader agreement among radiologists for whole gland and TZ lesions. However, the difference in agreement between PI-RADS v2.0 and v2.1 was not significant for the whole gland or the TZ. [ABSTRACT FROM AUTHOR]

Abstract Background To evaluate the inter- and intrareader agreement and reproducibility of the NI-RADS scoring system and lexicon with contrast-enhanced computed tomography (CECT) and contrast-enhanced magnetic resonance imaging (CEMRI). Methods This retrospective study included 97 CECT and CEMRI scans from 58 treated cases of head and neck squamous cell carcinoma (HNSCC) after the exclusion of head and neck cancers (HNCs) other than SCC and noncontrast and poor quality CT and MRI scans, with a total of 111 primary targets and 124 lymph node (LN) targets. Two experienced readers independently scored the likelihood of residual/recurrence for these targets based on the NI-RADS criteria and filled in report templates for NI-RADS lexicon diagnostic features. Inter- and intraobserver reproducibility was assessed with Cohen’s kappa, and the percent agreement was calculated. Results Almost perfect interreader agreement was found for the final NI-RADS category of the primary lesions and LNs, with K = 0.808 and 0.806, respectively. Better agreement was found for CT than for MRI (K = 0.843 and 0.77, respectively, P value 0.001). There was almost perfect agreement for excluding tissue enhancement (K = 0.826, 95% CI = 0.658–0.993, P value 0.001), with a percent agreement of 96.4%, and substantial agreement for discrete nodular and diffuse mucosal enhancement (K = 0.826, 95% CI = 0.658–0.993, P value 0.001), with a percent agreement of 96.4%. There was fair agreement for focal mucosal nonmass and deep ill-defined enhancement. The intrareader agreement was almost perfect for most of the rated features (K ranging from 0.802 to 1), with the exception of enlarging discrete nodule/mass and focal mucosal nonmass-like enhancement, which had substantial intraobserver agreement (K ranging from 0.768 to 0.786). Conclusion The individual features of NI-RADS show variable degrees of confidence; however, the overall NI-RADS category was not significantly affected.

Background: The Liver Imaging Reporting and Data System (LI‐RADS) was established for noninvasive diagnosis for hepatocellular carcinoma (HCC). However, whether training can improve readers' agreement with the expert consensus and inter‐reader agreement for final categories is still unclear. Purpose: To explore training effectiveness on readers' agreement with the expert consensus and inter‐reader agreement. Study Type: Prospective. Subjects: Seventy lesions in 61 patients at risk of HCC undergoing liver MRI; 20 visiting scholars. Field Strength/Sequence: 1.5 T or 3 T, Dual‐echo T1WI, Fast spin‐echo T2WI, SE‐EPI DWI, and Dynamic multiphase fast gradient‐echo T1WI. Assessment: Seventy lesions assigned LI‐RADS categories of LR1–LR5, LR‐M, and LR‐TIV by three radiologists in consensus were randomly selected, with 10 cases for each category. The consensus opinion was the standard reference. The third radiologist delivered the training. Twenty readers reviewed images independently and assigned each an LI‐RADS category both before and after the training. Statistical Tests: Accuracy, sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, receiver operating characteristic (ROC) analysis, simple and weighted kappa statistics, and Fleiss kappa statistics. Results: Before and after training: readers' AUC (areas under ROC) for LR‐1–LR‐5, LR‐M, and LR‐TIV were 0.898 vs. 0.913, 0.711 vs. 0.876, 0.747 vs. 0.860, 0.724 vs. 0.815, 0.844 vs. 0.895, 0.688 vs. 0.873, and 0.720 vs. 0.948, respectively, and all improved significantly (P < 0.05), except LR‐1(P = 0.25). Inter‐reader agreement between readers for LR‐1–LR‐5, LR‐M, LR‐TIV were 0.725 vs. 0.751, 0.325 vs. 0.607, 0.330 vs. 0.559, 0.284 vs. 0.488, 0.447 vs. 0.648, 0.229 vs. 0.589, and 0.362 vs. 0.852, respectively, and all increased significantly (P < 0.05). For training effectiveness on both AUC and inter‐reader agreement, LR‐TIV, LR‐M, and LR‐2 improved most, and LR‐1 made the least. Data Conclusion: This study shows LI‐RADS training could improve reader agreement with the expert consensus and inter‐reader agreement for final categories. Level of Evidence: 2 Technical Efficacy Stage: 2 [ABSTRACT FROM AUTHOR]

Purpose: To assess inter-reader agreement of key features from the SAR-AGA recommendations for the interpretation and reporting of MRE in adult patients with CD, focusing on the impact of radiologist experience on inter-reader agreement of CD phenotypes. Methods: Two experienced and two less-experienced radiologists retrospectively evaluated 99 MRE in CD patients (50 initial MRE, 49 follow-up MRE) performed from 1/1/2019 to 3/20/2020 for the presence of active bowel inflammation (stomach, proximal small bowel, ileum, colon), stricture, probable stricture, penetrating disease, and perianal disease. The MRE protocol did not include dedicated perianal sequences. Inter-rater agreement was determined for each imaging feature using prevalence-adjusted bias-adjusted kappa and compared by experience level. Results: All readers had almost-perfect inter-reader agreement (κ > 0.90) for penetrating disease, abscess, and perianal abscess in all 99 CD patients. All readers had strong inter-reader agreement (κ: 0.80–0.90) in 99 CD patients for active ileum inflammation, proximal small bowel inflammation, and stricture. Less-experienced readers had significantly lower inter-reader agreement for active ileum inflammation on initial than follow-up MRE (κ 0.68 versus 0.96, p = 0.018) and for strictures on follow-up than initial MRE (κ 0.76 versus 1.0, p = 0.027). Experienced readers had significantly lower agreement for perianal fistula on follow-up than initial MRE (κ: 0.55 versus 0.92, p = 0.008). Conclusion: There was strong to almost-perfect inter-reader agreement for key CD phenotypes described in the SAR-AGA consensus recommendations including active ileum and proximal small bowel inflammation, stricture, penetrating disease, abscess, and perianal abscess. Areas of lower inter-reader agreement could be targeted for future education efforts to further standardize CD MRE reporting. Dedicated perianal sequences should be included on follow-up MRE. [ABSTRACT FROM AUTHOR]

Background & Aim: The American College of Radiology Ultrasound Liver Imaging Reporting and Data System (ACR US LI-RADS) Visualization Score conveys the expected level of sensitivity of screening and surveillance ultrasound exams in patients at risk for hepatocellular carcinoma (HCC). We sought to determine inter-reader agreement of the Visualization Score which is currently unknown. Methods: Consecutive 6998 ultrasound HCC screening and surveillance studies in 3115 patients from 2017 to 2020 were retrospectively retrieved. Of these, 6154 (87.9%) studies were Visualization A (No or minimal limitations), 709 (10.1%) were Visualization B (Moderate limitations), and 135 (1.9%) were Visualization C (Severe limitations). Randomly sampled 90 studies, with 30 studies in each Visualization category, were included for analysis. Nine radiologists (3 senior attendings, 3 junior attendings and 3 body imaging fellows) blinded to the original categorization independently reviewed each study and assigned a Visualization Score. Intraclass correlation coefficient (ICC) was used to quantify inter-reader agreement. Results: ICC among all 9 radiologists was 0.70 (95% CI 0.63–0.77). ICCs among senior attendings, junior attendings and body imaging fellows were 0.68 (CI 0.58–0.76), 0.72 (CI 0.62–0.80) and 0.76 (CI 0.68–0.83), respectively. Subgroup analysis by liver parenchyma was further performed. ICC was highest in the patient group with normal liver parenchyma (0.69, CI 0.56–0.81), followed by steatosis (0.66, CI 0.54–0.79) and cirrhosis (0.58, CI 0.43–0.73), respectively. Conclusions: US LI-RADS Visualization Score is a reliable tool with good inter-reader agreement that can be used to indicate the expected level of sensitivity of a screening and surveillance ultrasound examination for detecting focal liver observations. [ABSTRACT FROM AUTHOR]

Purpose: To investigate the inter-reader agreement in assessing high-resolution computed tomography (HRCT) features of coronavirus disease 2019 (COVID-19) pneumonia. Method: Seventy-seven consecutive patients (mean age, 64 ± 15 years) with mild COVID-19 pneumonia that underwent HRCT were retrospectively included. Three radiologists [two devoted to thoracic imaging (R1, R2), and one generalist (R3)] on a per-examination basis independently assessed ground-glass opacity (GGO), consolidation, and crazy-paving pattern. The extent of each feature (total feature score, TFS) was semi-quantitatively assessed, and each TFS summed up to obtain total lung score (TLS). Presence of organizing pneumonia (OP) pattern was also recorded. The inter-reader agreement was calculated with Cohen's Kappa (k) and Free-Marginal Multirater k. Multivariable analysis was run to determine whether imaging features were predictive of short-term evolution to severe disease (need for ventilation). Results: Most features showed substantial inter-reader agreement, including TLS > 6 (k = 0.69), which was an independent predictor of short-term occurrence of severe disease, regardless of the reader (OR 9–53.19). Consolidation TFS > 2 and OP pattern showed substantial and moderate agreement, respectively, only when comparing R1 and R2. Consolidation TFS > 2 and OP pattern were independent predictors of severe disease for R2 (OR 4.87) and R1 (OR 6), respectively. Conclusions: The inter-reader agreement for most HRCT features of COVID-19 pneumonia ranges moderate-to-substantial, though it depends on readers' experience in the case of consolidation and OP pattern. [ABSTRACT FROM AUTHOR]

Aims: Studies in various cancer types have demonstrated discordance between results from different programmed death‐ligand 1 (PD‐L1) assays. Here, we compare the reproducibility and analytical concordance of four clinically developed assays for assessing PD‐L1‐positivity in tumour‐infiltrating immune cells in the tumour area (PD‐L1‐IC‐positivity) in triple‐negative breast cancer (TNBC). Methods and results: Primary TNBC resection specimens (n = 30) were selected based on their PD‐L1‐IC‐positivity per VENTANA SP142 (<1%: 15 cases; 1–5%: seven cases; >5%: eight cases). Serial histological sections were stained for PD‐L1 using VENTANA SP142, VENTANA SP263, DAKO 22C3 and DAKO 28‐8. PD‐L1‐IC‐positivity and tumour cell expression (≥1 versus <1%) were scored by trained readers from seven sites using online virtual microscopy. The adjusted mean of PD‐L1‐IC‐positivity for SP263 (7.8%) was significantly higher than those for the other three assays (3.7–4.9%). Differences in adjusted means were statistically significant between SP263 and the other three assays (P < 0.0001) but not between the three remaining assays when excluding SP263 (P = 0.0961–0.6522). Intra‐class correlation coefficients revealed moderate‐to‐strong inter‐reader agreement for each assay (0.460–0.805) and poor‐to‐strong inter‐assay agreement for each reader (0.298–0.678) on PD‐L1‐IC‐positivity. Conclusions: In this first multicentre study of different PD‐L1 assays in TNBC, we show that PD‐L1‐IC‐positivity for SP142, 22C3 and 28‐8 was reproducible and analytically concordant, indicating that these three assays may be analytically interchangeable. The relevance of the higher PD‐L1‐IC‐positivity for SP263 should be further investigated. [ABSTRACT FROM AUTHOR]

Background and Objective: Pathological features in non-muscle-invasive bladder cancer specimens are pivotal in determining correct patients' therapeutic management. Sparse data exist regarding the importance of second opinion performed by an expert uropathologist. This study aimed to assess the importance of a second opinion by an expert uropathologist in the management of bladder cancer., Methods: The study relied on 272 bladder cancer specimens from 231 patients seeking a pathology second opinion after transurethral resection of the bladder for a clinical suspicion of bladder cancer, relapse, or second-look procedure. Pathology second opinion was offered by an experienced fellowship-trained uropathologist. Discrepancies were recorded considering primary tumor staging, the presence of muscularis propria, and the presence of histological variants. Cases were categorized as no significant discordance, major discordance without management change, and major discordance with management change according to the European Urology Association (EAU) guidelines., Key Findings and Limitations: Among 272 second opinion cases, 39% (108/272) had major discordance and 28% (75/272) had major discordance with change in management according to the EAU guidelines. Upstaging and downstaging were reported in 66 (24%) patients. Improper identification of the presence of muscularis propria was found in 46 (17%) cases, of which 11 (4%) were deemed clinically relevant. Differences regarding the presence of histological variants were diagnosed in 40 cases (15%), resulting in eight (3%) changes in clinical management. In ten specimens (4%), multiple clinically relevant discrepancies were found., Conclusions and Clinical Implications: The second opinion evaluation changed the clinical management in 25% of the cases. These results support the importance of specimen review by an expert uropathologist as a major driver in the correct bladder cancer management., Patient Summary: We investigated the importance of a second opinion performed by an expert uropathologist in the management of bladder cancer. We found that 25% had their treatment plan changed due to the revised pathological report., (Copyright © 2024 European Association of Urology. Published by Elsevier B.V. All rights reserved.)

Agreement measures are useful tools to both compare different evaluations of the same diagnostic outcomes and validate new rating systems or devices. Cohen's kappa (κ) certainly is the most popular agreement method between two raters, and proved its effectiveness in the last sixty years. In spite of that, this method suffers from some alleged issues, which have been highlighted since the 1970s; moreover, its value is strongly dependent on the prevalence of the disease in the considered sample. This work introduces a new agreement index, the informational agreement (IA), which seems to avoid some of Cohen's kappa's flaws, and separates the contribution of the prevalence from the nucleus of agreement. These goals are achieved by modelling the agreement-in both dichotomous and multivalue ordered-categorical cases-as the information shared between two raters through the virtual diagnostic channel connecting them: the more information exchanged between the raters, the higher their agreement. In order to test its fair behaviour and the effectiveness of the method, IA has been tested on some cases known to be problematic for κ, in the machine learning context and in a clinical scenario to compare ultrasound (US) and automated breast volume scanner (ABVS) in the setting of breast cancer imaging. Graphical Abstract To evaluate the agreement between the two raters [Formula: see text] and [Formula: see text] we create an agreement channel, based on Shannon Information Theory, that directly connects the random variables X and Y, that express the raters outcomes. They are the terminals of the chain X⇔ diagnostic test performed by [Formula: see text] ⇔ patient condition[Formula: see text] ⇔ diagnostic test performed by [Formula: see text] ⇔ Y. [ABSTRACT FROM AUTHOR]

We compared 3 Tesla (3T) compressed sensing (CS)-MRI of different pulse sequences with various acceleration factors to standard fast spin-echo (FSE) sequences in terms of time, quality, and inter-reader agreement. Each sequence was qualitatively ranked and then qualitatively scored for blurring, artifact, low contrast detection, noise pattern, signal-to-noise ratio, and overall quality. The CS-MRI sequences demonstrated very good overall quality compared with routine FSE sequences with overall good inter-reader agreement. [ABSTRACT FROM AUTHOR]

Despite being one of the most common indications for surgery, data on the types of meniscus tear that should be treated surgically are limited. Improving patient selection requires agreement on meniscus tear description. This study evaluates a simple MRI tear classification system for inter-observer agreement. Knee MRI examinations from 57 subjects from the Osteoarthritis Initiative cohort were reviewed by two sub-specialty trained, musculoskeletal radiologists. Based on two pulse sequences, each meniscus was classified by: tear or no tear; location of tear in anterior, middle or posterior third or multiple thirds; and displaced or non-displaced radial, horizontal, longitudinal or complex tear pattern. A tear was defined as signal abnormality extending to the surface on at least two images and displacement as more than 2 mm of extrusion or separation measured orthogonal to the tear plane. Kappa, weighted Kappa and percentage agreement were calculated. For the medial meniscus, Kappa and percentage agreement estimates were, respectively: the presence of tear, 0.79 and 89.5%; tear with displacement, 0.70 (weighted Kappa) and 66.0%; tear description, 0.47 and 61.4%; tear location, 0.64 and 79.0%. For the lateral meniscus, estimates were: the presence of tear, 0.75 and 89.5%; tear with displacement, 0.81 (weighted Kappa) and 86.0%; tear description, 0.56 and 78.9%; tear location, 0.74 and 87.7%. The strength of agreement between readers was moderate to substantial underscoring the challenge of meniscus tear classification. [ABSTRACT FROM AUTHOR]

Purpose: To assess inter-radiologist agreement using the Society of Abdominal Radiology-American Gastroenterological Association (SAR-AGA) consensus recommendations for reporting CT/MR enterography exams in pediatric and young adult small bowel Crohn disease (CD).Methods: Institutional review board approval was obtained for this HIPAA-compliant retrospective investigation; the requirement for informed consent was waived. 25 CT and 25 MR enterography exams performed in children and young adults (age range: 6-23 years) between January 2015 and April 2017 with a distribution of ileal CD severity (phenotype) were identified: normal or chronic CD without active inflammation (40%), active inflammatory CD (20%), stricturing CD (20%), and penetrating CD (20%). Five fellowship-trained pediatric radiologists, blinded to one another, documented key imaging findings and standardized impressions based on SAR-AGA consensus recommendations. Inter-radiologist agreement was evaluated using Fleiss' multi-rater kappa statistic (κ) with 95% confidence intervals (CI).Results: Inter-radiologist agreement was moderate for all key imaging findings except presence of ulcerations (κ 0.37 [95% CI 0.28-0.46]) and sacculations (κ 0.31 [95% CI 0.23-0.40]). Agreement for standardized impressions was substantial for stricturing disease (κ 0.79 [95% CI 0.70-0.87]) and moderate for presence of inflammation (κ 0.49 [95% CI 0.44-0.56]) and penetrating disease (κ 0.58 [95% CI 0.49-0.67]). No significant difference in agreement was found between CT and MRI.Conclusions: Agreement among five pediatric radiologists was moderate to substantial for SAR-AGA standardized impressions and fair to moderate for key imaging findings of pediatric and young adult CD. [ABSTRACT FROM AUTHOR]

Background: In 2018, a new system was proposed for classifying and reporting post-treatment adult brain tumor on magnetic resonance imaging, named as Brain Tumor Reporting and Data System (BT-RADS), that needs a validation by means of agreement studies., Methods: A retrospective study was designed with the aim of identifying contrast-enhanced magnetic resonance imaging (MRI) of adult patients on follow-up for primary brain tumor at Fondazione Policlinico Campus Bio-Medico. Four radiologists (2 radiology residents, 1 general radiologist, 1 neuroradiologist) read and scored each study using the BT-RADS scoring tool, blinded to the MRI original report. Interobserver agreement and Fleiss' k were calculated to assess the level of diagnostic agreement. It was assessed how many times the assignment of different scoring of BT-RADS would have led to a different patient management., Results: The total number of patients included in the study was 23 with 147 MRIs and a total of 588 BT-RADS scores retrospectively evaluated. The two most frequent tumor types were astrocytoma grade 4 (62%) and oligodendroglioma grade 3 (21%). The overall agreement rate for all 4 radiologists was 82% with a Fleiss' k of 0.70. The overall agreement rate between general radiologist and neuroradiologist was 91% with a Fleiss' k of 0.86. The overall agreement rate between 2 radiology residents and neuroradiologist was 80% with a Fleiss' k of 0.66. Astrocytoma grade 3 (k: 0.51) and oligodendroglioma grade 2 (k: 0.32) showed a poor agreement while higher values of agreement were found for astrocytoma grade 4 (k: 0.70), astrocytoma grade 2 (k: 0.78) and oligodendroglioma grade 3 (k: 0.78). All the radiologists agreed on BT-RADS assignment in 70% patients, three radiologists agreed in 17% and two radiologists agree in 13%. In no cases there was a complete disagreement among the readers. In 18% of cases the discrepancy in the estimated BT-RADS would have led to a different follow-up management., Conclusions: BT-RADS can be considered a valid tool for neuroradiologists and radiologists even with little experience in the interpretation of patients' images during follow-up for adult primary brain tumors supporting standardized interpretation, reporting and clinical management., Competing Interests: Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://qims.amegroups.com/article/view/10.21037/qims-22-850/coif). The special issue “Imaging of Aging and Age-Related Disorders” was commissioned by the editorial office without any funding or sponsorship. CAM served as the unpaid Guest Editor of the issue and serves as an unpaid editorial board member of Quantitative Imaging in Medicine and Surgery. The authors have no other conflicts of interest to declare., (2023 Quantitative Imaging in Medicine and Surgery. All rights reserved.)

Objective Compare the spike detection performance of three skilled humans and three computer algorithms. Methods 40 prolonged EEGs, 35 containing reported spikes, were evaluated. Spikes and sharp waves were marked by the humans and algorithms. Pairwise sensitivity and false positive rates were calculated for each human–human and algorithm-human pair. Differences in human pairwise performance were calculated and compared to the range of algorithm versus human performance differences as a type of statistical Turing test. Results 5474 individual spike events were marked by the humans. Mean, pairwise human sensitivities and false positive rates were 40.0%, 42.1%, and 51.5%, and 0.80, 0.97, and 1.99/min. Only the Persyst 13 (P13) algorithm was comparable to humans – 43.9% and 1.65/min. Evaluation of pairwise differences in sensitivity and false positive rate demonstrated that P13 met statistical noninferiority criteria compared to the humans. Conclusion Humans had only a fair level of agreement in spike marking. The P13 algorithm was statistically noninferior to the humans. Significance This was the first time that a spike detection algorithm and humans performed similarly. The performance comparison methodology utilized here is generally applicable to problems in which skilled human performance is the desired standard and no external gold standard exists. [ABSTRACT FROM AUTHOR]

The article presents a review performed to evaluate the scientific evidence about variability in audiologists' interpretation of auditory brainstem response (ABR) tests and discusses 2 types of reader reliability namely inter-reader reliability or agreement and intra-reader reliability or agreement. It cites the importance of understanding the possible sources of variability and factors that may influence audiologists' decision making when reading ABR waves to reduce possible errors.

Background: The American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) was introduced to standardize the ultrasound characterization of thyroid nodules. Studies have shown that ACR-TIRADS reduces unnecessary biopsies and improves consistency of imaging recommendations. Despite its widespread adoption, there are few studies to date assessing the inter-reader agreement amongst radiology trainees with limited ultrasound experience. We hypothesize that in PGY-4 radiology residents with no prior exposure to ACR TI-RADS, a statistically significant improvement in inter-reader reliability can be achieved with a one hour training session., Aim: To evaluate the inter-reader agreement of radiology residents in using ACR TI-RADS before and after training., Methods: A single center retrospective cohort study evaluating 50 thyroid nodules in 40 patients of varying TI-RADS levels was performed. Reference standard TI-RADS scores were established through a consensus panel of three fellowship-trained staff radiologists with between 1 and 14 years of clinical experience each. Three PGY-4 radiology residents (trainees) were selected as blinded readers for this study. Each trainee had between 4 to 5 mo of designated ultrasound training. No trainee had received specialized TI-RADS training prior to this study. Each of the readers independently reviewed the 50 testing cases and assigned a TI-RADS score to each case before and after TI-RADS training performed 6 wk apart. Fleiss kappa was used to measure the pooled inter-reader agreement. The relative diagnostic performance of readers, pre- and post-training, when compared against the reference standard., Results: There were 33 females and 7 males with a mean age of 56.6 ± 13.6 years. The mean nodule size was 19 ± 14 mm (range from 5 to 63 mm). A statistically significant superior inter-reader agreement was found on the post-training assessment compared to the pre-training assessment for the following variables: 1. "Shape" (k of 0.09 [slight] pre-training vs 0.67 [substantial] post-training, P < 0.001), 2. "Echogenic foci" (k of 0.28 [fair] pre-training vs 0.45 [moderate] post-training, P = 0.004), 3. 'TI-RADS level' (k of 0.14 [slight] pre-training vs 0.36 [fair] post-training, P < 0.001) and 4. 'Recommendations' (k of 0.36 [fair] pre-training vs 0.50 [moderate] post-training, P = 0.02). No significant differences between the pre- and post-training assessments were found for the variables 'composition', 'echogenicity' and 'margins'. There was a general trend towards improved pooled sensitivity with TI-RADS levels 1 to 4 for the post-training assessment while the pooled specificity was relatively high (76.6%-96.8%) for all TI-RADS level., Conclusion: Statistically significant improvement in inter-reader agreement in the assigning TI-RADS level and recommendations after training is observed. Our study supports the use of dedicated ACR TI-RADS training in radiology residents., Competing Interests: Conflict-of-interest statement: The authors have no conflict of interest to declare., (©The Author(s) 2022. Published by Baishideng Publishing Group Inc. All rights reserved.)

Simple Summary: In our systematic review and meta-analysis of eight observational studies including a total of 1016 patients, we demonstrated excellent pooled inter-observer agreement among Genito-Urinary radiologists when adopting the novel Vesical Imaging-Reporting and Data System (VI-RADS) criteria in the pre-trans-urethral resection of bladder tumor (TURBT) assessment for non-muscle (NMIBC) vs. muscle-invasive bladder cancer (MIBC) detection. The Vesical Imaging-Reporting and Data System (VI-RADS) has been introduced to provide preoperative bladder cancer staging and has proved to be reliable in assessing the presence of muscle invasion in the pre-TURBT (trans-urethral resection of bladder tumor). We aimed to assess through a systematic review and meta-analysis the inter-reader variability of VI-RADS criteria for discriminating non-muscle vs. muscle invasive bladder cancer (NMIBC, MIBC). PubMed, Web of Science, Cochrane, and Embase were searched up until 30 July 2020. The Quality Appraisal of Diagnostic Reliability (QAREL) checklist was utilized to assess the quality of included studies and a pooled measure of inter-rater reliability (Cohen's Kappa [κ] and/or Intraclass correlation coefficients (ICCs)) was calculated. Further sensitivity analysis, subgroup analysis, and meta-regression were conducted to investigate the contribution of moderators to heterogeneity. In total, eight studies between 2018 and 2020, which evaluated a total of 1016 patients via 21 interpreting genitourinary (GU) radiologists, met inclusion criteria and were critically examined. No study was considered to be significantly flawed with publication bias. The pooled weighted mean κ estimate was 0.83 (95%CI: 0.78–0.88). Heterogeneity was present among the studies (Q = 185.92, d.f. = 7, p < 0.001; I2 = 92.7%). Meta-regression analyses showed that the relative % of MIBC diagnosis and cumulative reader's experience to influence the estimated outcome (Coeff: 0.019, SE: 0.007; p= 0.003 and 0.036, SE: 0.009; p = 0.001). In the present study, we confirm excellent pooled inter-reader agreement of VI-RADS to discriminate NMIBC from MIBC underlying the importance that standardization and reproducibility of VI-RADS may confer to multiparametric magnetic resonance (mpMRI) for preoperative BCa staging. [ABSTRACT FROM AUTHOR]