Search

Background Opioid misuse screening in hospitals is resource-intensive and rarely done. Many hospitalized patients are never offered opioid treatment. An automated approach leveraging routinely captured electronic health record (EHR) data may be easier for hospitals to institute. We previously derived and internally validated an opioid classifier in a separate hospital setting. The aim is to externally validate our previously published and open-source machine-learning classifier at a different hospital for identifying cases of opioid misuse. Methods An observational cohort of 56,227 adult hospitalizations was examined between October 2017 and December 2019 during a hospital-wide substance use screening program with manual screening. Manually completed Drug Abuse Screening Test served as the reference standard to validate a convolutional neural network (CNN) classifier with coded word embedding features from the clinical notes of the EHR. The opioid classifier utilized all notes in the EHR and sensitivity analysis was also performed on the first 24 h of notes. Calibration was performed to account for the lower prevalence than in the original cohort. Results Manual screening for substance misuse was completed in 67.8% (n = 56,227) with 1.1% (n = 628) identified with opioid misuse. The data for external validation included 2,482,900 notes with 67,969 unique clinical concept features. The opioid classifier had an AUC of 0.99 (95% CI 0.99–0.99) across the encounter and 0.98 (95% CI 0.98–0.99) using only the first 24 h of notes. In the calibrated classifier, the sensitivity and positive predictive value were 0.81 (95% CI 0.77–0.84) and 0.72 (95% CI 0.68–0.75). For the first 24 h, they were 0.75 (95% CI 0.71–0.78) and 0.61 (95% CI 0.57–0.64). Conclusions Our opioid misuse classifier had good discrimination during external validation. Our model may provide a comprehensive and automated approach to opioid misuse identification that augments current workflows and overcomes manual screening barriers.

The Centers for Medicare & Medicaid Services Electronic Health Record Meaningful Use Incentive Program requires physicians to document body mass index (BMI) and a follow-up treatment plan for adult patients with BMI ≥ 25. To examine the effect of a best practice alert on physician documentation of obesity-related care and referrals to weight management treatment, in a cluster-randomized design, 14 primary care clinics at an academic medical center were randomized to best practice alert intervention (n = 7) or comparator (n = 7). The alert was triggered when both height and weight were entered and BMI was ≥30. Both intervention and comparator clinics could document meaningful use by selecting a nutrition education handout within the alert. Intervention clinics could also select a referral option from the list of clinic and community-based weight management programs embedded in the alert. Main outcomes were proportion of eligible patients with (1) obesity-related documentation and (2) referral. There were 26,471 total primary care encounters with 12,981 unique adult patients with BMI ≥ 30 during the 6-month study period. Documentation doubled (17 to 33%) with implementation of the alert. However, intervention clinics were not significantly more likely to refer patients to weight management than comparator clinics (2.8 vs. 1.3%, p = 0.07). Although the alert was associated with increased physician meaningful use compliance, it was not an effective strategy for improving patient access to weight management services. Further research is needed to understand system-level characteristics that influence obesity management in primary care.