Application of STREAM-URO and APPRAISE-AI reporting standards for artificial intelligence studies in pediatric urology: A case example with pediatric hydronephrosis.

Introduction: Artificial intelligence (AI) and machine learning (ML) in pediatric urology is gaining increased popularity and credibility. However, the literature lacks standardization in reporting and there are areas for methodological improvement, which incurs difficulty in comparison between studies and may ultimately hurt clinical implementation of these models. The "STandardized REporting of Applications of Machine learning in UROlogy" (STREAM-URO) framework provides methodological instructions to improve transparent reporting in urology and APPRAISE-AI in a critical appraisal tool which provides quantitative measures for the quality of AI studies. The adoption of these will allow urologists and developers to ensure consistency in reporting, improve comparison, develop better models, and hopefully inspire clinical translation.
Methods: In this article, we have applied STREAM-URO framework and APPRAISE-AI tool to the pediatric hydronephrosis literature. By doing this, we aim to describe best practices on ML reporting in urology with STREAM-URO and provide readers with a critical appraisal tool for ML quality with APPRAISE-AI. By applying these to the pediatric hydronephrosis literature, we provide some tutorial for other readers to employ these in developing and appraising ML models. We also present itemized recommendations for adequate reporting, and critically appraise the quality of ML in pediatric hydronephrosis insofar. We provide examples of strong reporting and highlight areas for improvement.
Results: There were 8 ML models applied to pediatric hydronephrosis. The 26-item STREAM-URO framework is provided in Appendix A and 24-item APPRAISE-AI tool is provided in Appendix B. Across the 8 studies, the median compliance with STREAM-URO was 67 % and overall study quality was moderate. The highest scoring APPRAISE-AI domains in pediatric hydronephrosis were clinical relevance and reporting quality, while the worst were methodological conduct, robustness of results, and reproducibility.
Conclusions: If properly conducted and reported, ML has the potential to impact the care we provide to patients in pediatric urology. While AI is exciting, the paucity of strong evidence limits our ability to translate models to practice. The first step toward this goal is adequate reporting and ensuring high quality models, and STREAM-URO and APPRAISE-AI can facilitate better reporting and critical appraisal, respectively.
Competing Interests: Conflict of interest STREAM-URO framework and APPRAISE-AI tool were developed across multiple institutions, with principal developers at the University of Toronto (AK, JCCK, LE, MR, GSK, AJL). Articles were assessed by multiple raters and STREAM-URO compliance was assessed by a rater outside of its development (IA). GSK reports advisory, consultant, or trial work with Merck, BMS, EMD Serono, Pfizer, Janssen, Ferring, Theralase, Verity, TerSera, Knight Therapeutics, PhotoCure, and Astra Zeneca. There are no other financial or monetary conflicts of interest to disclose, and funders had no role in the conception, development, or decision to publish this manuscript.
(Copyright © 2024 Journal of Pediatric Urology Company. Published by Elsevier Ltd. All rights reserved.)