Search

We tested the value of adding data from the operating room to models predicting in-hospital death. We assessed model performance using two metrics, the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (AUPRC), to illustrate the differences in information they convey in the setting of class imbalance. Data was collected on 74,147 patients who underwent major noncardiac surgery and 112 unique features were extracted from electronic health records. Sets of features were incrementally added to models using logistic regression, naïve Bayes, random forest, and gradient boosted machine methods. AUROC increased as more features were added, but changes were small for some modeling approaches. In contrast, AUPRC, which reflects positive predicted value, exhibited improvements across all models. Using AUPRC highlighted the added value of intraoperative data, not seen consistently with AUROC, and that with class imbalance AUPRC may serve as the more clinically relevant criterion.

About 1 in 10 adults are reported to exhibit clinical depression and the associated personal, societal, and economic costs are significant. In this study, we applied the MTERMS NLP system and machine learning classification algorithms to identify patients with depression using discharge summaries. Domain experts reviewed both the training and test cases, and classified these cases as depression with a high, intermediate, and low confidence. For depression cases with high confidence, all of the algorithms we tested performed similarly, with MTERMS' knowledge-based decision tree slightly better than the machine learning classifiers, achieving an F-measure of 89.6%. MTERMS also achieved the highest F-measure (70.6%) on intermediate confidence cases. The RIPPER rule learner was the best performing machine learning method, with an F-measure of 70.0%, and a higher precision but lower recall than MTERMS. The proposed NLP-based approach was able to identify a significant portion of the depression cases (about 20%) that were not on the coded diagnosis list.