1. Postoperative delirium prediction using machine learning models and preoperative electronic health record data
- Author
-
Bishara, Andrew, Chiu, Catherine, Whitlock, Elizabeth L, Douglas, Vanja C, Lee, Sei, Butte, Atul J, Leung, Jacqueline M, and Donovan, Anne L
- Subjects
Biomedical and Clinical Sciences ,Clinical Sciences ,Brain Disorders ,Clinical Research ,Machine Learning and Artificial Intelligence ,Patient Safety ,Prevention ,Neurodegenerative ,Neurosciences ,Networking and Information Technology R&D (NITRD) ,Mental Health ,Acquired Cognitive Impairment ,4.1 Discovery and preclinical testing of markers and technologies ,Good Health and Well Being ,Aged ,Cohort Studies ,Delirium ,Electronic Health Records ,Female ,Humans ,Machine Learning ,Male ,Middle Aged ,Postoperative Complications ,Predictive Value of Tests ,Preoperative Period ,Reproducibility of Results ,Retrospective Studies ,Postoperative delirium ,Delirium prevention ,Risk prediction model ,Machine learning ,Geriatric surgery ,Medical Physiology ,Anesthesiology ,Clinical sciences - Abstract
BackgroundAccurate, pragmatic risk stratification for postoperative delirium (POD) is necessary to target preventative resources toward high-risk patients. Machine learning (ML) offers a novel approach to leveraging electronic health record (EHR) data for POD prediction. We sought to develop and internally validate a ML-derived POD risk prediction model using preoperative risk features, and to compare its performance to models developed with traditional logistic regression.MethodsThis was a retrospective analysis of preoperative EHR data from 24,885 adults undergoing a procedure requiring anesthesia care, recovering in the main post-anesthesia care unit, and staying in the hospital at least overnight between December 2016 and December 2019 at either of two hospitals in a tertiary care health system. One hundred fifteen preoperative risk features including demographics, comorbidities, nursing assessments, surgery type, and other preoperative EHR data were used to predict postoperative delirium (POD), defined as any instance of Nursing Delirium Screening Scale ≥2 or positive Confusion Assessment Method for the Intensive Care Unit within the first 7 postoperative days. Two ML models (Neural Network and XGBoost), two traditional logistic regression models ("clinician-guided" and "ML hybrid"), and a previously described delirium risk stratification tool (AWOL-S) were evaluated using the area under the receiver operating characteristic curve (AUC-ROC), sensitivity, specificity, positive likelihood ratio, and positive predictive value. Model calibration was assessed with a calibration curve. Patients with no POD assessments charted or at least 20% of input variables missing were excluded.ResultsPOD incidence was 5.3%. The AUC-ROC for Neural Net was 0.841 [95% CI 0. 816-0.863] and for XGBoost was 0.851 [95% CI 0.827-0.874], which was significantly better than the clinician-guided (AUC-ROC 0.763 [0.734-0.793], p
- Published
- 2022