Comparing Expert Reported Outcomes to National Surgical Quality Improvement Program Risk Calculator-Predicted Outcomes: Do Reporting Standards Differ?

Introduction: Expert-reported outcomes and complications may not reflect the standardized coding that can be provided by independent, third-party evaluations. The goal of this article is to compare expert-reported complications with standardized coding by the National Surgical Quality Improvement Program (NSQIP). The procedures evaluated were laparoscopic radical nephrectomy (LRN), robot-assisted radical prostatectomy (RARP), and radical cystectomy (RC).
Methods: The 10 largest LRN, RARP, and RC series were reviewed for reported complications. An index patient was derived from each series using patient demographic data. Index patients were entered into the NSQIP surgical risk calculator (SRC), which provides 11 predicted outcomes based on inputted data. SRC-predicted outcomes were compared with available complication rates in each series.
Results: Across the 30 studies, 172 out of 330 (52%) of NSQIP-provided outcome types were presented within expert manuscripts. Death and venous thromboembolism (VTE) were the most commonly reported (27 and 23 studies, respectively), whereas urinary tract infection (UTI) (9) and pneumonia (10) were the least commonly presented. Comorbidities and follow-up duration were reported in 8 out of 30 and 17 out of 30 studies, respectively. For LRN, the median number of reported outcomes was 3 (range 1-5). LRN experts demonstrated a shorter mean length of stay (LOS) (2.5 days, SD=1.7) (p<0.001). In RARP studies, a median of 7.5 (3-11) outcomes was reported. Experts outperformed NSQIP RARP predictions in serious complications (p<0.001), any complication (p<0.001), surgical site infection (p=0.025), UTI (p<0.001), and VTE (p=0.002). RC manuscripts reported a median of 7 (2-11) outcomes. RC experts had higher rates of serious complications (p<0.001), reoperation (p<0.001), and death (p<0.001) than predicted by SRC.
Conclusion: The level of standardization in reporting of outcomes differs between expert series and NSQIP, thus making comparisons difficult.