Validity of the AHRQ Patient Safety Indicator for Postoperative Physiologic and Metabolic Derangement Based on a National Sample of Medical Records

OBJECTIVE The Agency for Healthcare Research and Quality Patient Safety Indicator (PSI) 10, "Postoperative Physiologic and Metabolic Derangement" (PPMD), uses administrative data to detect postoperative acute kidney injury (AKI) requiring dialysis and diabetes-related complications. We sought to evaluate the indicator's criterion validity. RESEARCH DESIGN We conducted a retrospective cross-sectional study of hospitalization records flagged positive and negative by PSI 10 from a diverse set of 35 hospitals between February 1, 2006 and June 30, 2009. Trained nurse abstractors reviewed medical records. We determined the indicator's sensitivity, specificity, and positive and negative predictive values. RESULTS Of 94 records flagged by PSI 10 (87 for AKI, 7 for diabetic complications, 1 for both), 69 (73%) involved an accurately coded event; 60 (64%; 95% CI, 46%-79%) represented true PPMD from a clinical perspective. Two of 8 records flagged for diabetic complications were true events. Nineteen false positives involved preoperative renal failure. Three of 230 records flagged negative (enriched with questionably negative records) represented true PPMD. The indicator's sensitivity was 66% (20%-94%), specificity 99.9% (99.5%-100%), and negative predictive value 99.9% (99.4%-100%). Considering dialysis access procedures tantamount to dialysis and excluding records with lower urinary tract obstruction might increase the sensitivity and positive predictive value to 98% (87%-100%) and 72% (50%-87%), respectively. CONCLUSIONS PSI 10 mostly concerns AKI and currently has moderate criterion validity, which might improve with increased use of "present on admission" coding, abandonment of the diabetes criteria, and adjustments to the indicator specifications regarding dialysis access and urinary tract obstruction.