Search

BACKGROUND: Implementing evidence-based practices (EBPs) within service systems is critical to population-level health improvements - but also challenging, especially for complex behavioral health interventions in low-resource settings. Mis-implementation refers to poor outcomes from an EBP implementation effort; mis-implementation outcomes are an important, but largely untapped, source of information about how to improve knowledge exchange. AIMS AND OBJECTIVES: We present mis-implementation cases from three pragmatic trials of behavioral health EBPs in U.S. Federally Qualified Health Centers (FQHCs). METHODS: We adapted the Consolidated Framework for Implementation Research and its Outcomes Addendum into a framework for mis-implementation and used it to structure the case summaries with information about the EBP and trial, mis-implementation outcomes, and associated determinants (barriers and facilitators). We compared the three cases to identify shared and unique mis-implementation factors. FINDINGS: Across cases, there was limited adoption and fidelity to the interventions, which led to eventual discontinuation. Barriers contributing to mis-implementation included intervention complexity, low buy-in from overburdened providers, lack of alignment between providers and leadership, and COVID-19-related stressors. Mis-implementation occurred earlier in cases that experienced both patient- and provider-level barriers, and that were conducted during the COVID-19 pandemic. DISCUSSION AND CONCLUSION: Multi-level determinants contributed to EBP mis-implementation in FQHCs, limiting the ability of these health systems to benefit from knowledge exchange. To minimize mis-implementation, knowledge exchange strategies should be designed around common, core barriers but also flexible enough to address a variety of site-specific contextual factors and should be tailored to relevant audiences such as providers, patients, and/or leadership.

BACKGROUND: Financial barriers in substance use disorder service systems have limited the widespread adoption-i.e., provider-level reach-of evidence-based practices (EBPs) for youth substance use disorders. Reach is essential to maximizing the population-level impact of EBPs. One promising, but rarely studied, type of implementation strategy for overcoming barriers to EBP reach is financing strategies, which direct financial resources in various ways to support implementation. We evaluated financing strategies for the Adolescent Community Reinforcement Approach (A-CRA) EBP by comparing two US federal grant mechanisms, organization-focused and state-focused grants, on organization-level A-CRA reach outcomes. METHOD: A-CRA implementation took place through organization-focused and state-focused grantee cohorts from 2006 to 2021. We used a quasi-experimental, mixed-method design to compare reach between treatment organizations funded by organization-focused versus state-focused grants (164 organizations, 35 states). Using administrative training records, we calculated reach as the per-organization proportion of trained individuals who received certification in A-CRA clinical delivery and/or supervision by the end of grant funding. We tested differences in certification rate by grant type using multivariable linear regression models that controlled for key covariates (e.g., time), and tested threats to internal validity from our quasi-experimental design through a series of sensitivity analyses. We also drew on interviews and surveys collected from the treatment organizations and (when relevant) interviews with state administrators to identify factors that influenced reach. RESULTS: The overall certification rates were 27 percentage points lower in state-focused versus organization-focused grants (p = .01). Sensitivity analyses suggested these findings were not explained by confounding temporal trends nor by organizational or state characteristics. We did not identify significant quantitative moderators of reach outcomes, but qualitative findings suggested certain facilitating factors were more influential for organization-focused grants (e.g., strategic planning) and certain barrier factors were more impactful for state-focused grants (e.g., states finding it difficult to execute grant activities). DISCUSSION: As the first published comparison of EBP reach outcomes between financing strategies, our findings can help guide state and federal policy related to financing strategies for implementing EBPs that reduce youth substance use. Future work should explore contextual conditions under which different financing strategies can support the widespread implementation of EBPs for substance use disorder treatment.

PurposeThe use of telemedicine (TM) has accelerated in recent years, yet research on the implementation and effectiveness of TM-delivered medication treatment for opioid use disorder (MOUD) has been limited. This study investigated the feasibility of implementing a care coordination model involving MOUD delivered via an external TM provider for the purpose of expanding access to MOUD for patients in rural settings.MethodsThe study tested a care coordination model in 6 rural primary care sites by establishing referral and coordination between the clinic and a TM company for MOUD. The intervention spanned approximately 6 months from July/August 2020 to January 2021, coinciding with the peak of the COVID-19 pandemic. Each clinic tracked patients with OUD in a registry during the intervention period. A pre-/post-intervention design (N = 6) was used to assess the clinic-level outcome as patient-days on MOUD based on patient electronic health records.FindingsAll clinics implemented critical components of the intervention, with an overall TM referral rate of 11.7% among patients in the registry. Five of the 6 sites showed an increase in patient-days on MOUD during the intervention period compared to the 6-month period before the intervention (mean increase per 1,000 patients: 132 days, P = .08, Cohen's d = 0.55). The largest increases occurred in clinics that lacked MOUD capacity or had a greater number of patients initiating MOUD during the intervention period.ConclusionsTo expand access to MOUD in rural settings, the care coordination model is most effective when implemented in clinics that have negligible or limited MOUD capacity.

Human computer interaction (HCI) and implementation science (IS) each have been applied to improve the adoption and delivery of innovative health interventions, and the two fields have complementary goals, foci, and methods. While the IS community increasingly draws on methods from HCI, there are many unrealized opportunities for HCI to draw from IS and to catalyze bidirectional collaborations. This workshop will explore similarities and differences between fields, with a goal of articulating a research agenda at their intersection.

BackgroundAttempting to implement evidence-based practices in contexts for which they are not well suited may compromise their fidelity and effectiveness or burden users (e.g., patients, providers, healthcare organizations) with elaborate strategies intended to force implementation. To improve the fit between evidence-based practices and contexts, implementation science experts have called for methods for adapting evidence-based practices and contexts and tailoring implementation strategies; yet, methods for considering the dynamic interplay among evidence-based practices, contexts, and implementation strategies remain lacking. We argue that harmonizing the three can be facilitated by user-centered design, an iterative and highly stakeholder-engaged set of principles and methods.MethodsThis paper presents a case example in which we used a three-phase user-centered design process to design and plan to implement a care coordination intervention for young adults with cancer. Specifically, we used usability testing to redesign and augment an existing patient-reported outcome measure that served as the basis for our intervention to optimize its usability and usefulness, ethnographic contextual inquiry to prepare the context (i.e., a comprehensive cancer center) to promote receptivity to implementation, and iterative prototyping workshops with a multidisciplinary design team to design the care coordination intervention and anticipate implementation strategies needed to enhance contextual fit.ResultsOur user-centered design process resulted in the Young Adult Needs Assessment and Service Bridge (NA-SB), including a patient-reported outcome measure and a collection of referral pathways that are triggered by the needs young adults report, as well as implementation guidance. By ensuring NA-SB directly responded to features of users and context, we designed NA-SB for implementation, potentially minimizing the strategies needed to address misalignment that may have otherwise existed. Furthermore, we designed NA-SB for scale-up; by engaging users from other cancer programs across the country to identify points of contextual variation which would require flexibility in delivery, we created a tool intended to accommodate diverse contexts.ConclusionsUser-centered design can help maximize usability and usefulness when designing evidence-based practices, preparing contexts, and informing implementation strategies-in effect, harmonizing evidence-based practices, contexts, and implementation strategies to promote implementation and effectiveness.

Telemedicine (TM) enabled by digital health technologies to provide medical services has been considered a key solution to increasing health care access in rural communities. With the immediate need for remote care due to the COVID-19 pandemic, many health care systems have rapidly incorporated digital technologies to support the delivery of remote care options, including medication treatment for individuals with opioid use disorder (OUD). In responding to the opioid crisis and the COVID-19 pandemic, public health officials and scientific communities strongly support and advocate for greater use of TM-based medication treatment for opioid use disorder (MOUD) to improve access to care and have suggested that broad use of TM during the pandemic should be sustained. Nevertheless, research on the implementation and effectiveness of TM-based MOUD has been limited. To address this knowledge gap, the National Drug Abuse Treatment Clinical Trials Network (CTN) funded (via the NIH HEAL Initiative) a study on Rural Expansion of Medication Treatment for Opioid Use Disorder (Rural MOUD; CTN-0102) to investigate the implementation and effectiveness of adding TM-based MOUD to rural primary care for expanding access to MOUD. In preparation for this large-scale, randomized controlled trial incorporating TM in rural primary care, a feasibility study is being conducted to develop and pilot test implementation procedures. In this commentary, we share some of our experiences, which include several challenges, during the initial two-month period of the feasibility study phase. While these challenges could be due, at least in part, to adjusting to the COVID-19 pandemic and new workflows to accommodate the study, they are notable and could have a substantial impact on the larger, planned pragmatic trial and on TM-based MOUD more broadly. Challenges include low rates of identification of risk for OUD from screening, low rates of referral to TM, digital device and internet access issues, workflow and capacity barriers, and insurance coverage. These challenges also highlight the lack of empirical guidance for best TM practice and quality remote care models. With TM expanding rapidly, understanding implementation and demonstrating what TM approaches are effective are critical for ensuring the best care for persons with OUD.

BackgroundUnderstanding the costs and economic benefits of implementation has been identified by policymakers and researchers as critical to increase the uptake and sustainment of evidence-based practices, but this topic remains relatively understudied. Conducting team science with health economists has been proposed as a solution to increase economic evaluation in implementation science; however, these recommendations ignore the differences in goals and perspectives in these two fields. Our recent qualitative research identified that implementation researchers predominantly approach health economists to examine costs, whereas the majority of health economists expressed limited interest in conducting economic evaluations and a desire to be more integrated within implementation science initiatives. These interviews pointed to challenges in establishing fruitful partnerships when health economists are relegated to the "Third Aim" (i.e., lowest-priority research objective) in implementation science projects by their research partners.DiscussionIn this debate paper, we argue that implementation researchers and health economists need to focus on team science research principles to expand capacity to address pressing research questions that cut across the two fields. Specifically, we use the four-phase model of transdisciplinary research to outline the goals and processes needed to build capacity in this area (Hall et al., Transl Behav Med 2:415-30, 2012). The first phase focuses on the development of transdisciplinary research teams, including identifying appropriate partners (e.g., considering policy or public health researchers in addition to health economists) and building trust. The conceptual phase focuses on strategies to consider when developing joint research questions and methodology across fields. In the implementation phase, we outline the effective processes for conducting research projects, such as team learning. Finally, in the translation phase, we highlight how a transdisciplinary approach between health economists and implementation researchers can impact real-world practice and policy. The importance of investigating the economic impact of evidence-based practice implementation is widely recognized, but efforts have been limited due to the challenges in conducting team science across disciplines. Training in team science can help advance transdisciplinary efforts, which has the potential to increase the rigor and impact of economic evaluations in implementation science while expanding the roles taken by health economists.

Numerous evidence-based programs (EBPs) exist for delivery in schools to promote youth mental health outcomes. However, school systems often lack the internal infrastructure to support the effective implementation and sustainment of EBPs when external supports are withdrawn, resulting in notable attenuation in the benefits in youth clinical outcomes that are associated with EBPs. This paper illustrates how to leverage concepts from improvement science and implementation science to develop learning school systems dedicated to enhancing the infrastructure capacity of a school to advance the implementation and sustainment of EBPs. In particular, we discuss how treatment integrity (extent to which an EBP is delivered as designed) and youth mental health outcome data are: (1) collected; (2) analyzed and interpreted; and (3) fed back into the school system to increase organizational supports and promote school practitioners' behavior change to produce improvements in youth mental health outcomes. We also discuss psychological safety among the people within a school system as a key characteristic of a learning school system. We then present a theory-informed approach to learning school systems to demonstrate how data generated by a learning school system can lead to precise and effective plans that continuously improve implementation and result in the eventual sustainment of EBPs. We conclude with a brief research agenda with concrete steps toward realizing the potential of learning school systems to support the implementation and sustainment of EBPs for mental health problems.

BackgroundImplementation research infrequently addresses economic factors, despite the importance of understanding the costs of implementing evidence-based practices (EBPs). Though partnerships with health economists have the potential to increase attention to economic factors within implementation science, barriers to forming these collaborations have been noted. This study investigated the experiences of health economists and implementation researchers who have partnered across disciplines to inform strategies to increase such collaborations.MethodsA purposeful sampling approach was used to identify eight health economists and eight implementation researchers with experience participating in cross-disciplinary research. We used semi-structured interviews to gather information about participants' experiences with collaborative research. Thematic analysis was conducted to identify core themes related to facilitators and barriers to collaborations.ResultsHealth economists and implementation researchers voiced different perspectives on collaborative research, highlighting the importance of increasing cross-disciplinary understanding. Implementation researchers described a need to measure costs in implementation studies, whereas many health economists described that they seek to collaborate on projects that extend beyond conducting cost analyses. Researchers in both disciplines articulated motivations for collaborative research and identified strategies that promote successful collaboration, with varying degrees of convergence across these themes. Shared motivations included improving methodological rigor of research and making a real-world impact. Strategies to improve collaboration included starting partnerships early in the study design period, having a shared interest, and including health economists in the larger scope of the research.ConclusionsHealth economists and implementation researchers both conduct research with significant policy implications and have the potential to inform one another's work in ways that might more rapidly advance the uptake of EBPs. Collaborative research between health economists and implementation science has the potential to advance the field; however, researchers will need to work to bridge disciplinary differences. By beginning to develop strong working relationships; increasing their understanding of one another's disciplinary culture, methodology, and language; and increasing the role economists have within research design and execution, both implementation researchers and health economists can support successful collaborations and robust and informative research.

Background: Intervention adaptation is often necessary to improve the fit between evidence-based practices/programs and implementation contexts. Existing frameworks describe intervention adaptation processes but do not provide detailed steps for prospectively designing adaptations, are designed for researchers, and require substantial time and resources to complete. A pragmatic approach to guide implementers through developing and assessing adaptations in local contexts is needed. The goal of this project was to develop Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI), a method for intervention adaptation that leverages human centered design methods and is tailored to the needs of intervention implementers working in applied settings with limited time and resources. Method: MODIFI was iteratively developed via a mixed-methods modified Delphi process. Feedback was collected from 43 implementation research and practice experts. Two rounds of data collection gathered quantitative ratings of acceptability and inclusion (Round 1) and feasibility (Round 2), as well as qualitative feedback regarding MODIFI revisions analyzed using conventional content analysis. Results: In Round 1, most participants rated all proposed components as essential but identified important avenues for revision which were incorporated into MODIFI prior to Round 2. Round 2 emphasized feasibility, where ratings were generally high and fewer substantive revisions were recommended. Round 2 changes largely surrounded operationalization of terms/processes and sequencing of content. Results include a detailed presentation of the final version of the three-step MODIFI method (Step 1: Learn about the users, local context, and intervention; Step 2: Adapt the intervention; Step 3: Evaluate the adaptation) along with a case example of its application. Discussion: MODIFI is a pragmatic method that was developed to extend the contributions of other research-based adaptation theories, models, and frameworks while integrating methods that are tailored to the needs of intervention implementers. Guiding teams to tailor evidence-based interventions to their local context may extend for whom, where, and under what conditions an intervention can be effective. [ABSTRACT FROM AUTHOR]

Background: The COVID-19 pandemic shed light on stark racial and ethnic inequities in access to care and accurate health information in the U.S. When COVID-19 vaccines became available, communities of color faced multiple barriers that contributed to low vaccine rates. To address this gap, the Equity-First Vaccination Initiative supported community organizations in five demonstration cities to plan and implement hyper-local strategies to increase COVID-19 vaccine access and uptake among communities of color. Purpose: To draw learnings from the experiences of the participating organizations, we applied a framework that integrated implementation science and health equity principles. Design and sample: In this commentary, we describe how we used this framework to guide qualitative interviews with community organizations, focusing on insights across five implementation elements (reach, design, implementation, adaptation, implementation outcomes). Conclusions: Learnings from this evaluation may help guide future implementation of similarly complex initiatives involving multiple organizations and sites to advance health equity during a public health crisis. [ABSTRACT FROM AUTHOR]

Sexual and gender minority (SGM) individuals experience high rates of homelessness and criminal justice system involvement, underscoring the need for supportive housing services. To explore the service needs of this population, we interviewed providers (n = 11) and clients (n = 10) from eight supportive housing organizations working with SGM populations in Los Angeles County, California, USA. We used the Consolidated Framework for Implementation Research to synthesize interview responses into themes (by domain and cross-cutting). Takeaways included the need for investment in systems of care for vulnerable SGM populations; the particular marginalization of Trans individuals and providers that serve them; the roles of supportive housing staff, residents, and leadership in cultivating an affirming environment; the prevalence of discrimination and stigma within supportive housing programs and broader society; and the complex interrelationships among SGM identity, homelessness, and criminal justice system involvement. These findings have important implications for supportive housing services and related policy. [ABSTRACT FROM AUTHOR]