Automated Calculator for the Pediatric Sequential Organ Failure Assessment Score: Development and External Validation in a Single-Center 7-Year Cohort, 2015-2021.

Objectives: The pediatric Sequential Organ Failure Assessment (pSOFA) score summarizes severity of organ dysfunction and can be used to predict in-hospital mortality. Manual calculation of the pSOFA score is time-consuming and prone to human error. An automated method that is open-source, flexible, and scalable for calculating the pSOFA score directly from electronic health record data is desirable.
Design: Single-center, retrospective cohort study.
Setting: Quaternary 40-bed PICU.
Patients: All patients admitted to the PICU between 2015 and 2021 with ICU stay of at least 24 hours.
Interventions: None.
Measurements and Main Results: We used 77 records to evaluate the automated score. The automated algorithm had an overall accuracy of 97%. The algorithm calculated the respiratory component of two cases incorrectly. An expert human annotator had an initial accuracy of 75% at the patient level and 95% at the component level. An untrained human annotator with general clinical research experience had an overall accuracy of 16% and component-wise accuracy of 67%. Weighted kappa for agreement between the automated method and the expert annotator's initial score was 0.92 (95% CI, 0.88-0.95), and between the untrained human annotator and the automated score was 0.50 (95% CI, 0.36-0.61). Data from 9146 patients (in-hospital mortality 3.6%) were included to validate externally the discriminability of the automated pSOFA score. The admission-day pSOFA score had an area under the receiver operating characteristic curve of 0.79 (95% CI, 0.77-0.82).
Conclusions: The developed automated algorithm calculates pSOFA score with high accuracy and is more accurate than a trained expert rater and nontrained data abstracter. pSOFA's performance for predicting in-hospital mortality was lower in our cohort than it was for the originally derived score.
Competing Interests: The authors have disclosed that they do not have any potential conflicts of interest.
(Copyright © 2024 by the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies.)