Search

Growing evidence has shown that applying machine learning models to large clinical data sources may exceed clinician performance in suicide risk stratification. However, many existing prediction models either suffer from "temporal bias" (a bias that stems from using case-control sampling) or require training on all available patient visit data. Here, we adopt a "landmark model" framework that aligns with clinical practice for prediction of suicide-related behaviors (SRBs) using a large electronic health record database. Using the landmark approach, we developed models for SRB prediction (regularized Cox regression and random survival forest) that establish a time-point (e.g., clinical visit) from which predictions are made over user-specified prediction windows using historical information up to that point. We applied this approach to cohorts from three clinical settings: general outpatient, psychiatric emergency department, and psychiatric inpatients, for varying prediction windows and lengths of historical data. Models achieved high discriminative performance (area under the Receiver Operating Characteristic curve 0.74-0.93 for the Cox model) across different prediction windows and settings, even with relatively short periods of historical data. In short, we developed accurate, dynamic SRB risk prediction models with the landmark approach that reduce bias and enhance the reliability and portability of suicide risk prediction models., Competing Interests: Declaration of Competing Interest Dr. Smoller is a member of the the Scientific Advisory Board of Sensorium Therapeutics (with equity), and has received grant support from Biogen, Inc. He is PI of a collaborative study of the genetics of depression and bipolar disorder sponsored by 23andMe for which 23andMe provides analysis time as in-kind support but no payments., (Copyright © 2023 The Author(s). Published by Elsevier B.V. All rights reserved.)

Importance: Half of the people who die by suicide make a health care visit within 1 month of their death. However, clinicians lack the tools to identify these patients., Objective: To predict suicide attempts within 1 and 6 months of presentation at an emergency department (ED) for psychiatric problems., Design, Setting, and Participants: This prognostic study assessed the 1-month and 6-month risk of suicide attempts among 1818 patients presenting to an ED between February 4, 2015, and March 13, 2017, with psychiatric problems. Data analysis was performed from May 1, 2020, to November 19, 2021., Main Outcomes and Measures: Suicide attempts 1 and 6 months after presentation to the ED were defined by combining data from electronic health records (EHRs) with patient 1-month (n = 1102) and 6-month (n = 1220) follow-up surveys. Ensemble machine learning was used to develop predictive models and a risk score for suicide., Results: A total of 1818 patients participated in this study (1016 men [55.9%]; median age, 33 years [IQR, 24-46 years]; 266 Hispanic patients [14.6%]; 1221 non-Hispanic White patients [67.2%], 142 non-Hispanic Black patients [7.8%], 64 non-Hispanic Asian patients [3.5%], and 125 non-Hispanic patients of other race and ethnicity [6.9%]). A total of 137 of 1102 patients (12.9%; weighted prevalence) attempted suicide within 1 month, and a total of 268 of 1220 patients (22.0%; weighted prevalence) attempted suicide within 6 months. Clinicians' assessment alone was little better than chance at predicting suicide attempts, with externally validated area under the receiver operating characteristic curve (AUC) of 0.67 for the 1-month model and 0.60 for the 6-month model. Prediction accuracy was slightly higher for models based on EHR data (1-month model: AUC, 0.71; 6 month model: AUC, 0.65) and was best using patient self-reports (1-month model: AUC, 0.76; 6-month model: AUC, 0.77), especially when patient self-reports were combined with EHR and/or clinician data (1-month model: AUC, 0.77; and 6 month model: AUC, 0.79). A model that used only 20 patient self-report questions and an EHR-based risk score performed similarly well (1-month model: AUC, 0.77; 6 month model: AUC, 0.78). In the best 1-month model, 30.7% (positive predicted value) of the patients classified as having highest risk (top 25% of the sample) made a suicide attempt within 1 month of their ED visit, accounting for 64.8% (sensitivity) of all 1-month attempts. In the best 6-month model, 46.0% (positive predicted value) of the patients classified at highest risk made a suicide attempt within 6 months of their ED visit, accounting for 50.2% (sensitivity) of all 6-month attempts., Conclusions and Relevance: This prognostic study suggests that the ability to identify patients at high risk of suicide attempt after an ED visit for psychiatric problems improved using a combination of patient self-reports and EHR data.

Objective: The purpose of this article was to determine whether longitudinal historical data, commonly available in electronic health record (EHR) systems, can be used to predict patients' future risk of suicidal behavior., Method: Bayesian models were developed using a retrospective cohort approach. EHR data from a large health care database spanning 15 years (1998-2012) of inpatient and outpatient visits were used to predict future documented suicidal behavior (i.e., suicide attempt or death). Patients with three or more visits (N=1,728,549) were included. ICD-9-based case definition for suicidal behavior was derived by expert clinician consensus review of 2,700 narrative EHR notes (from 520 patients), supplemented by state death certificates. Model performance was evaluated retrospectively using an independent testing set., Results: Among the study population, 1.2% (N=20,246) met the case definition for suicidal behavior. The model achieved sensitive (33%-45% sensitivity), specific (90%-95% specificity), and early (3-4 years in advance on average) prediction of patients' future suicidal behavior. The strongest predictors identified by the model included both well-known (e.g., substance abuse and psychiatric disorders) and less conventional (e.g., certain injuries and chronic conditions) risk factors, indicating that a data-driven approach can yield more comprehensive risk profiles., Conclusions: Longitudinal EHR data, commonly available in clinical settings, can be useful for predicting future risk of suicidal behavior. This modeling approach could serve as an early warning system to help clinicians identify high-risk patients for further screening. By analyzing the full phenotypic breadth of the EHR, computerized risk screening approaches may enhance prediction beyond what is feasible for individual clinicians.