Search

Artificial intelligence (AI) and machine learning (ML) in pediatric urology is gaining increased popularity and credibility. However, the literature lacks standardization in reporting and there are areas for methodological improvement, which incurs difficulty in comparison between studies and may ultimately hurt clinical implementation of these models. The "STandardized REporting of Applications of Machine learning in UROlogy" (STREAM-URO) framework provides methodological instructions to improve transparent reporting in urology and APPRAISE-AI in a critical appraisal tool which provides quantitative measures for the quality of AI studies. The adoption of these will allow urologists and developers to ensure consistency in reporting, improve comparison, develop better models, and hopefully inspire clinical translation. In this article, we have applied STREAM-URO framework and APPRAISE-AI tool to the pediatric hydronephrosis literature. By doing this, we aim to describe best practices on ML reporting in urology with STREAM-URO and provide readers with a critical appraisal tool for ML quality with APPRAISE-AI. By applying these to the pediatric hydronephrosis literature, we provide some tutorial for other readers to employ these in developing and appraising ML models. We also present itemized recommendations for adequate reporting, and critically appraise the quality of ML in pediatric hydronephrosis insofar. We provide examples of strong reporting and highlight areas for improvement. There were 8 ML models applied to pediatric hydronephrosis. The 26-item STREAM-URO framework is provided in Appendix A and 24-item APPRAISE-AI tool is provided in Appendix B. Across the 8 studies, the median compliance with STREAM-URO was 67 % and overall study quality was moderate. The highest scoring APPRAISE-AI domains in pediatric hydronephrosis were clinical relevance and reporting quality, while the worst were methodological conduct, robustness of results, and reproducibility. If properly conducted and reported, ML has the potential to impact the care we provide to patients in pediatric urology. While AI is exciting, the paucity of strong evidence limits our ability to translate models to practice. The first step toward this goal is adequate reporting and ensuring high quality models, and STREAM-URO and APPRAISE-AI can facilitate better reporting and critical appraisal, respectively. [ABSTRACT FROM AUTHOR]

Purpose: Vesicoureteral reflux grading from voiding cystourethrograms is highly subjective with low reliability. We aimed to demonstrate improved reliability for vesicoureteral reflux grading with simple and machine learning approaches using ureteral tortuosity and dilatation on voiding cystourethrograms. Materials and Methods: Voiding cystourethrograms were collected from our institution for training and 5 external data sets for validation. Each voiding cystourethrogram was graded by 5-7 raters to determine a consensus vesicoureteral reflux grade label and inter- and intra-rater reliability was assessed. Each voiding cystourethrogram was assessed for 4 features: ureteral tortuosity, proximal, distal, and maximum ureteral dilatation. The labels were then assigned to the combination of the 4 features. A machine learning-based model, qVUR, was trained to predict vesicoureteral reflux grade from these features and model performance was assessed by AUROC (area under the receiver-operator-characteristic). Results: A total of 1,492 kidneys and ureters were collected from voiding cystourethrograms resulting in a total of 8,230 independent gradings. The internal inter-rater reliability for vesicoureteral reflux grading was 0.44 with a median percent agreement of 0.71 and low intra-rater reliability. Higher values for each feature were associated with higher vesicoureteral reflux grade. qVUR performed with an accuracy of 0.62 (AUROC[0.84) with stable performance across all external data sets. The model improved vesicoureteral reflux grade reliability by 3.6-fold compared to traditional grading (P < .001). Conclusions: In a large pediatric population from multiple institutions, we show that machine learning-based assessment for vesicoureteral reflux improves reliability compared to current grading methods. qVUR is generalizable and robust with similar accuracy to clinicians but the added prognostic value of quantitative measures warrants further study. [ABSTRACT FROM AUTHOR]

Identifying patients with low left ventricular ejection fraction (LVEF) and monitoring LVEF responses to treatment are important clinical goals. Can a deep-learning algorithm predict pediatric LVEF within clinically acceptable error? The study authors wanted to fine-tune an adult deep-learning algorithm to calculate LVEF in pediatric patients. A priori, their objective was to refine the algorithm to perform LVEF calculation with a mean absolute error (MAE) ≤5%. A quaternary pediatric hospital A convenience sample (n = 321) of echocardiograms from newborns to 18 years old with normal cardiac anatomy or hemodynamically insignificant anomalies. Echocardiograms were chosen from a group of healthy controls with known normal LVEF (n = 267) and a dilated cardiomyopathy patient group with reduced LVEF (n = 54). The artificial intelligence model EchoNet-Dynamic was tested on this data set and then retrained, tested, and further validated to improve LVEF calculation. The gold standard value was LVEF calculated by clinical experts. In a random subset of subjects (n = 40) not analyzed prior to selection of the final model, EchoNet-Dynamic calculated LVEF with a MAE of 8.39%, R2 = 0.47 without, and MAE 4.47%, R2 = 0.87 with fine-tuning. Bland-Altman analysis suggested that the model slightly underestimates LVEF (bias = –2.42%). The 95% limits of agreement between actual and calculated values were –12.32% to 7.47%. The fine-tuned model calculates LVEF in a range of pediatric patients within clinically acceptable error. Potential advantages include reducing operator error in LVEF calculation and supporting independent LVEF assessment by inexperienced users. [Display omitted] [ABSTRACT FROM AUTHOR]

The objectivity of vesicoureteral reflux (VUR) grading has come into question for low inter-rater reliability. Using quantitative image features to aid in VUR grading may make it more consistent. To develop a novel quantitative approach to the assignment of VUR from voiding cystourethrograms (VCUG) alone. An online dataset of VCUGs was abstracted and individual renal units were graded as low-grade (I-III) or high-grade (IV–V). We developed an image analysis and machine learning workflow to automatically calculate and normalize the ureteropelvic junction (UPJ) width, ureterovesical junction (UVJ) width, maximum ureter width, and tortuosity of the ureter based on three simple user annotations. A random forest classifier was trained to distinguish between low-vs high-grade VUR. An external validation cohort was generated from the institutional imaging repository. Discriminative capability was quantified using receiver-operating-characteristic and precision–recall curve analysis. We used Shapley Additive exPlanations to interpret the model's predictions. 41 renal units were abstracted from an online dataset, and 44 renal units were collected from the institutional imaging repository. Significant differences observed in UVJ width, UPJ width, maximum ureter width, and tortuosity between low- and high-grade VUR. A random-forest classifier performed favourably with an accuracy of 0.83, AUROC of 0.90 and AUPRC of 0.89 on leave-one-out cross-validation, and accuracy of 0.84, AUROC of 0.88 and AUPRC of 0.89 on external validation. Tortuosity had the highest feature importance, followed by maximum ureter width, UVJ width, and UPJ width. We deployed this tool as a web-application, qVUR (quantitative VUR), where users are able to upload any VCUG for automated grading using the model generated here (https://akhondker.shinyapps.io/qVUR/). This study provides the first step towards creating an automated and more objective standard for determining the significance of VUR features. Our findings suggest that tortuosity and ureter dilatation are predictors of high-grade VUR. Moreover, this proof-of-concept model was deployed in a simple-to-use web application. Grading of VUR using quantitative metrics is possible, even in non-standardized datasets of VCUG. Machine learning methods can be applied to objectively grade VUR in the future. [Display omitted] [ABSTRACT FROM AUTHOR]

The concepts of fragility index (FI) and fragility quotient (FQ) have been previously described. PlumX metrics encompass online "footprints" of research in addition to traditional citations. Herein we explore PlumX metrics against the quality of BBD literature. To explore altmetrics against the quality of bladder and bowel dysfunction (BBD) literature. A literature search was conducted using Pubmed, Medline, Embase for BBD and related terms. A total of 54,045 abstracts were screened, followed by 693 full text reviews and data extraction from 126. Studies were included if they reported on 2 groups being compared, had dichotomous outcomes, and had significant results. The median FI score was 4 (0–500) and there were 20 studies which had a FI of 0. The FQ had a median of 0.04 (0–0.32). PlumX usage was 263 ± 540, captures were 45 ± 60 and social media attention was 2 ± 2. Overall, 42% of papers were clinical trials (RCTs). When compared to other study designs, we noted a significant difference in PlumX captures (57 ± 72 RCT vs. 35 ± 47 other; p = 0.03). RCTs had higher usage, social media engagement and citations however, the differences were not significant. H-Index had a significant correlation with FI (p = 0.036), however correlations for PlumX usage and captures, while modestly positive (0.04–0.10) for the FI and FQ, were not significant. A comparison of FI and FQ by topic can be reviewed in the Summary Table. When considering the FI and FQ robustness indicators of the BBD literature, we found similarities when compared to other studies. It was reported that overall, the hydronephrosis literature was fragile with many studies requiring only a few events to nullify significance, regardless of the study design. Similarly, in a review of pediatric vesicoureteral reflux (VUR) clinical trials, results were also fragile. When comparing fragility measures to altmetric variables we noted that despite the growing popularity of altmetrics, citation counts, and h-indices remain the traditional measures to monitor research consumption. There has been a reported correlation between manuscript citation counts, author h-index, altmetrics measures in several specialties and across many domains of research including medical sciences, arts, and the humanities, however in the present study only weak correlations were noted. The body of BBD comparative studies is fragile in keeping with other pediatric urology literature populations. Despite fragile results, RCTs generate slightly moreattention as measured by select PlumX metrics. These results suggest the need for including fragility measures in our literature, aiming to focus attention towards more robust articles. Table 1 Comparison of Fragility Index and Fragility Quotient by BBD topic. Table 1 Topic Fragility Index Fragility Quotient Nocturnal Enuresis 5 (1) 0.05 (0.06) Urinary Tract Infection 4.5 (7) 0.044 (0.08) Overactive Bladder 2 (5) 0.013 (0.03) Dysfunctional Voiding 2 (5) 0.029 (0.06) Incontinence 4 (7) 0.045 (0.06) Values in table show median and interquartile range. There were no statistically significant differences using Kruskal–Wallis (KW) test. [ABSTRACT FROM AUTHOR]