Search

Background: Prediction models have demonstrated a range of applications across medicine, including using electronic health record (EHR) data to identify hospital readmission and mortality risk. Large language models (LLMs) can transform unstructured EHR text into structured features, which can then be integrated into statistical prediction models, ensuring that the results are both clinically meaningful and interpretable., Objective: This study aims to compare the classification decisions made by clinical experts with those generated by a state-of-the-art LLM, using terms extracted from a large EHR data set of individuals with mental health disorders seen in emergency departments (EDs)., Methods: Using a dataset from the EHR systems of more than 50 health care provider organizations in the United States from 2016 to 2021, we extracted all clinical terms that appeared in at least 1000 records of individuals admitted to the ED for a mental health-related problem from a source population of over 6 million ED episodes. Two experienced mental health clinicians (one medically trained psychiatrist and one clinical psychologist) reached consensus on the classification of EHR terms and diagnostic codes into categories. We evaluated an LLM's agreement with clinical judgment across three classification tasks as follows: (1) classify terms into "mental health" or "physical health", (2) classify mental health terms into 1 of 42 prespecified categories, and (3) classify physical health terms into 1 of 19 prespecified broad categories., Results: There was high agreement between the LLM and clinical experts when categorizing 4553 terms as "mental health" or "physical health" (κ=0.77, 95% CI 0.75-0.80). However, there was still considerable variability in LLM-clinician agreement on the classification of mental health terms (κ=0.62, 95% CI 0.59-0.66) and physical health terms (κ=0.69, 95% CI 0.67-0.70)., Conclusions: The LLM displayed high agreement with clinical experts when classifying EHR terms into certain mental health or physical health term categories. However, agreement with clinical experts varied considerably within both sets of mental and physical health term categories. Importantly, the use of LLMs presents an alternative to manual human coding, presenting great potential to create interpretable features for prediction models., (© Nicholas C Cardamone, Mark Olfson, Timothy Schmutte, Lyle Ungar, Tony Liu, Sara W Cullen, Nathaniel J Williams, Steven C Marcus. Originally published in JMIR Medical Informatics (https://medinform.jmir.org).)

Objective: Youths who start behavioral health treatment often stop before completing a therapeutic course of care. To increase treatment engagement and quality of care, the Evidence-Based Practice and Innovation Center in Philadelphia has incentivized use of evidence-based practices (EBPs) for mental health care of youths. The authors aimed to compare treatment outcomes between youths who received EBP care and those who did not., Methods: Using EBP-specific billing codes and propensity score matching, the authors compared treatment retention among youths who received trauma-focused cognitive-behavioral therapy (TF-CBT; N=413) or parent-child interaction therapy (PCIT; N=90) relative to matched samples of youths in standard outpatient therapy (N=503)., Results: Youths with a minimum of one session of TF-CBT or PCIT attended a second session at higher rates than did youths in the matched control group (TF-CBT: 96% vs. 68%, p<0.01; PCIT: 94% vs. 69%, respectively, p<0.01). On average, these returning youths attended more sessions in the EBP groups than in the control group (TF-CBT: 15.9 vs. 11.5 sessions, p<0.01; PCIT: 11.2 vs. 6.9 sessions, p<0.01)., Conclusions: These findings indicate that, in addition to improving quality of care, EBP implementation helps address the major challenge that most youths who engage with treatment are not retained long enough for care to have therapeutic effects. Future research should examine the mechanisms through which EBPs can improve treatment retention., Competing Interests: The authors report no financial relationships with commercial interests.

Background: Despite the ubiquity of multilevel sampling, design, and analysis in mental health implementation trials, few resources are available that provide reference values of design parameters (e.g., effect size, intraclass correlation coefficient [ICC], and proportion of variance explained by covariates [covariate R 2 ]) needed to accurately determine sample size. The aim of this study was to provide empirical reference values for these parameters by aggregating data on implementation and clinical outcomes from multilevel implementation trials, including cluster randomized trials and individually randomized repeated measures trials, in mental health. The compendium of design parameters presented here represents plausible values that implementation scientists can use to guide sample size calculations for future trials., Method: We searched NIH RePORTER for all federally funded, multilevel implementation trials addressing mental health populations and settings from 2010 to 2020. For all continuous and binary implementation and clinical outcomes included in eligible trials, we generated values of effect size, ICC, and covariate R 2 at each level via secondary analysis of trial data or via extraction of estimates from analyses in published research reports. Effect sizes were calculated as Cohen d ; ICCs were generated via one-way random effects ANOVAs; covariate R 2 estimates were calculated using the reduction in variance approach., Results: Seventeen trials were eligible, reporting on 53 implementation and clinical outcomes and 81 contrasts between implementation conditions. Tables of effect size, ICC, and covariate R 2 are provided to guide implementation researchers in power analyses for designing multilevel implementation trials in mental health settings, including two- and three-level cluster randomized designs and unit-randomized repeated-measures designs., Conclusions: Researchers can use the empirical reference values reported in this study to develop meaningful sample size determinations for multilevel implementation trials in mental health. Discussion focuses on the application of the reference values reported in this study., Competing Interests: The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: NJW, NCC, and SCM have no conflicts of interest to declare. RSB is principal at Implementation Science & Practice, LLC. She is currently an appointed member of the National Advisory Mental Health Council and the NASEM study, “Blueprint for a national prevention infrastructure for behavioral health disorders,” and serves on the scientific advisory board for AIM Youth Mental Health Foundation and the Klingenstein Third Generation Foundation. She has received consulting fees from United Behavioral Health and OptumLabs. She previously served on the scientific and advisory board for Optum Behavioral Health and has received royalties from Oxford University Press. All reported activities are outside of the submitted work., (© The Author(s) 2024.)

Background: Medications for opioid use disorder (MOUD) are among the best tools we have to combat the opioid epidemic. Yet, use of MOUD among people with opioid use disorder (OUD) remains low. Interventions to increase MOUD access in the United States have largely focused on improving organizational capacity and addressing funding barriers, yet stigma toward MOUD may inhibit uptake even where MOUD is readily available. Substance use disorder (SUD) treatment counselors likely have considerable influence on a client's choice to initiate and adhere to MOUD, but beliefs that counselors convey about MOUD in interaction with clients are understudied. The current study explores what advantages and disadvantages that SUD treatment counselors communicate about buprenorphine, methadone, and naltrexone., Methods: From June to December 2021, we surveyed counselors from publicly-funded SUD treatment agencies under a municipality-wide mandate to offer MOUD to all clients with OUD. Counselors were asked to describe, in a free-response format, the most important advantages and disadvantages to communicate to their clients about taking buprenorphine, methadone, and naltrexone. Counselor responses were coded for one or more advantage and disadvantage., Results: A total of 271 SUD counselors from 29 agencies completed the survey, generating 1,995 advantages and disadvantages across three types of MOUD. The most frequently reported advantage across all three types of MOUD was their ability to reduce cravings and illicit drug use. The most frequently reported disadvantage related to the potential for some types of MOUD to develop long-term medication dependence., Conclusions: As the availability and variety of MOUD treatment options continue to expand, it is important that SUD counselors are equipped with evidence-based recommendations for OUD care. We identified misalignments with the MOUD-prescribing evidence base and stigmatizing language toward MOUD within counselors' responses, highlighting the potential to refine training materials for MOUD and mitigate stigmatizing beliefs., Competing Interests: Declarations Competing interests. The authors declare that there is no conflict of interest.

Background: The harms of opioid use disorder (OUD) and HIV infection disproportionately impact marginalized populations, especially people experiencing homelessness and people who inject drugs (PWID). Mobile OUD service delivery models are emerging to increase access and reduce barriers to OUD care. While there is growing interest in these models, there is limited research about the services they provide, how they operate, and what barriers they face. We characterize the capacity, barriers, and sustainment of mobile OUD care services in a large city with a high incidence of OUD and HIV., Methods: From May to August 2022, we conducted semi-structured interviews with leadership from all seven mobile OUD care units (MOCU) providing a medication for OUD or other substance use disorder services in Philadelphia. We surveyed leaders about their unit's services, staffing, operating location, funding sources, and linkages to care. Leaders were asked to describe their clinical approach, treatment process, and the barriers and facilitators to their operations. Interview recordings were coded using rapid qualitative analysis., Results: MOCUs are run by small, multidisciplinary teams, typically composed of a clinician, one or two case managers, and a peer recovery specialist or outreach worker. MOCUs provide a range of services, including medications for OUD, wound care, medical services, case management, and screening for infectious diseases. No units provide methadone, but all units provide naloxone, six write prescriptions for buprenorphine, and one unit dispenses buprenorphine. The most frequently reported barriers include practical challenges of working on a MOCU (e.g. lack of space, safety), lack of community support, and patients with substantial medical and psychosocial needs. Interviewees reported concerns about funding and specifically as it relates to providing their staff with adequate pay. The most frequently reported facilitators include positive relationships with the community, collaboration with other entities (e.g. local nonprofits, the police department, universities), and having non-clinical staff (e.g. outreach workers, peer recovery specialists) on the unit., Conclusions: MOCUs provide life-saving services and engage marginalized individuals with OUD. These findings highlight the challenges and complexities of caring for PWID and demonstrate a need to strengthen collaborations between MOCU providers and the treatment system. Policymakers should consider programmatic funding for permanent mobile OUD care services., (© 2023. The Author(s).)

Background: In response to the COVID-19 pandemic, we launched the Penn Medicine Coping First Aid program to provide psychosocial supports to our health system community. Our approach leveraged lay health worker volunteers trained in principles of Psychological First Aid to deliver coaching services through a centralized virtual platform., Methods: We emailed all (n = 408) first year housestaff (i.e., residents and fellows) with an invitation to schedule a session with a resilience coach. We compared the mental health concerns, symptoms, and Psychological First Aid techniques recorded in (n = 67) first year housestaff sessions with (n = 91) sessions of other employees in the health system., Results: Between June and November 2020, forty-six first year housestaff attended at least one resilience coaching session. First year housestaff most commonly presented with feelings of anxiety and sadness and shared concerns related to the availability of social support. Resilience coaches most frequently provided practical assistance and ensured safety and comfort to first year housestaff. First year housestaff reported fewer physical or mental health symptoms and held shorter sessions with resilience coaches than non-housestaff., Conclusions: This work offers insights on how to address psychosocial functioning through low-intensity interventions delivered by lay personnel. More research is needed to understand the efficacy of this program and how best to engage housestaff in wellness and resilience programs throughout training, both during and beyond COVID-19., (© 2023. BioMed Central Ltd., part of Springer Nature.)

Background: Vending machines for harm reduction (VMHR) are an innovative approach to deliver life-saving materials, information, and treatment for hard-to-reach populations, particularly for persons who inject drugs. The current study explores stakeholders' perspectives on the feasibility and acceptability of VMHR in Philadelphia., Methods: From October 2021 to February 2022, we conducted 31 semi-structured interviews with potential end users, staff, and leadership at a local federally qualified health center, and community members. Trained coders extracted themes from interview transcripts across four key domains: materials and logistics, location, access, and community introduction., Results: Interviewees from all stakeholder groups endorsed using VMHR to provide supplies for wound care, fentanyl test strips, naloxone, and materials to connect individuals to treatment and other services. Dispensing syringes and medications for opioid use disorder were commonly endorsed by health center staff but were more controversial among potential end users. Even within stakeholder groups, views varied with respect to where to locate the machines, but most agreed that the machine should be placed in the highest drug use areas. Across stakeholder groups, interviewees suggested several strategies to introduce and gain community acceptance of VMHR, including community education, one-on-one conversations with community members, and coupling the machine with safe disposal of syringes and information to link individuals to treatment., Conclusions: Stakeholders were generally receptive to VMHR. The current study findings are consistent with qualitative analyses from outside of the USA and contribute new ideas regarding the anticipated community response and best methods for introducing these machines to a community. With thoughtful planning and design, VMHR could be a feasible and acceptable modality to reduce death and disease transmission associated with the opioid and HIV epidemics in cities like Philadelphia., (© 2023. The Author(s).)

Background: Self-report measures can improve evidence-based assessment practices in substance use disorder treatment, but many measures are burdensome and costly, limiting their utility in community practice and non-specialty healthcare settings. This systematic review identified and evaluated the psychometric properties of brief, free, and readily accessible self-report measures of substance use and related factors., Methods: We searched two electronic databases (PsycINFO and PubMed) in May 2021 for published literature on scales, measures, or instruments related to substance use, substance use treatment, and recovery, and extracted the names of all measures. Measures were included if they were: (1) brief (25 items or fewer), (2) freely accessible in a ready-to-use format, and (3) had published psychometric data., Results: An initial search returned 411 measures, of which 73 (18%) met criteria for inclusion. Included measures assessed a variety of substances (e.g., alcohol, nicotine, opioids, cannabinoids, cocaine) and measurement domains (e.g., use, severity, expectancies, withdrawal). Among these measures, 14 (19%) were classified as psychometrically "excellent," 27 (37%) were rated as "good," 32 (44%) were "adequate.", Conclusions: Despite the shift toward evidence-based assessment in substance use disorder treatment in the last twenty years, key areas of public health concern are lacking pragmatic, psychometrically valid measures. Among the brief measures we reviewed, less than a fifth met criteria for psychometric "excellence" and most of these instruments fell into one measurement domain: screening for problematic substance use. Future research should focus both on improving the evidence base for existing brief self-report measures and creating new low-burden measures for specific substances and treatment constructs., (Copyright © 2022 Elsevier B.V. All rights reserved.)

Introduction: Despite their well-established effectiveness, medications for opioid use disorder (MOUD) are widely underutilized across the United States. In the context of a large publicly funded behavioral health system, we examined the relationship between a range of implementation barriers and a substance use disorder treatment agency's level of adoption of MOUD., Methods: We surveyed leadership of publicly funded substance use disorder treatment centers in Philadelphia about the significance of barriers to implementing MOUD related to their workforce, organization, funding, regulations, and beliefs about MOUD's efficacy and safety. We queried leaders on the percentage of their patients with opioid use disorder who receive MOUD and examined associations between implementation barriers and MOUD adoption., Results: Ratings of regulatory, organizational, or funding barriers of respondents who led high MOUD adopting agencies (N = 20) were indistinguishable from those who led agencies that were low adopting of MOUD (N = 23). In contrast, agency leaders who denied MOUD-belief or workforce barriers were significantly more likely to lead high-MOUD-adopting organizations., Conclusions: These findings suggest that leadership beliefs about MOUD may be a key factor of the organizational decision to adopt and should be a target of implementation efforts to increase direct provision of these medications., (Copyright © 2022. Published by Elsevier Inc.)