Search

Background: The purpose of this paper is to describe the Juvenile Justice-Translational Research on Interventions for Adolescents in the Legal System (JJ-TRIALS) study, a cooperative implementation science initiative involving the National Institute on Drug Abuse, six research centers, a coordinating center, and Juvenile Justice Partners representing seven US states. While the pooling of resources across centers enables a robust implementation study design involving 36 juvenile justice agencies and their behavioral health partner agencies, co-producing a study protocol that has potential to advance implementation science, meets the needs of all constituencies (funding agency, researchers, partners, study sites), and can be implemented with fidelity across the cooperative can be challenging. This paper describes (a) the study background and rationale, including the juvenile justice context and best practices for substance use disorders, (b) the selection and use of an implementation science framework to guide study design and inform selection of implementation components, and (c) the specific study design elements, including research questions, implementation interventions, measurement, and analytic plan.Methods/design: The JJ-TRIALS primary study uses a head-to-head cluster randomized trial with a phased rollout to evaluate the differential effectiveness of two conditions (Core and Enhanced) in 36 sites located in seven states. A Core strategy for promoting change is compared to an Enhanced strategy that incorporates all core strategies plus active facilitation. Target outcomes include improvements in evidence-based screening, assessment, and linkage to substance use treatment.Discussion: Contributions to implementation science are discussed as well as challenges associated with designing and deploying a complex, collaborative project.Trial Registration: NCT02672150 . [ABSTRACT FROM AUTHOR]

Background: Theories, models, and frameworks used by implementation science, including Diffusion of Innovations, tend to focus on the adoption of one innovation, when often organizations may be facing multiple simultaneous adoption decisions. For instance, despite evidence that care management practices (CMPs) are helpful in managing chronic illness, there is still uneven adoption by physician organizations. This exploratory paper leverages this natural variation in uptake to describe inter-organizational patterns in adoption of CMPs and to better understand how adoption choices may be related to one another.Methods: We assessed a cross section of national survey data from physician organizations reporting on the use of 20 CMPs (5 each for asthma, congestive heart failure, depression, and diabetes). Item response theory was used to explore patterns in adoption, first considering all 20 CMPs together and then by subsets according to disease focus or CMP type (e.g., registries, patient reminders). Mokken scale analysis explored whether adoption choices were linked by disease focus or CMP type and whether a consistent ordering of adoption choices was present.Results: The Mokken scale for all 20 CMPs demonstrated medium scalability (H = 0.43), but no consistent ordering. Scales for subsets of CMPs sharing a disease focus had medium scalability (0.4 < H < 0.5), while subsets sharing a CMP type had strong scalability (H > 0.5). Scales for CMP type consistently ranked diabetes CMPs as most adoptable and depression CMPs as least adoptable. Within disease focus scales, patient reminders were ranked as the most adoptable CMP, while clinician feedback and patient education were ranked the least adoptable.Conclusions: Patterns of adoption indicate that innovation characteristics may influence adoption. CMP dissemination efforts may be strengthened by encouraging traditionally non-adopting organizations to focus on more adoptable practices first and then describing a pathway for the adoption of subsequent CMPs. Clarifying why certain CMPs are "less adoptable" may also provide insights into how to overcome CMP adoption constraints. [ABSTRACT FROM AUTHOR]

Background: While dissemination and implementation (D&I) science has grown rapidly, there is an ongoing need to understand how to build and sustain capacity in individuals and institutions conducting research. There are three inter-related domains for capacity building: people, settings, and activities. Since 2008, Washington University in St. Louis has dedicated significant attention and resources toward building D&I research capacity. This paper describes our process, challenges, and lessons with the goal of informing others who may have similar aims at their own institution.Activities: An informal collaborative, the Washington University Network for Dissemination and Implementation Research (WUNDIR), began with a small group and now has 49 regular members. Attendees represent a wide variety of settings and content areas and meet every 6 weeks for half-day sessions. A logic model organizes WUNDIR inputs, activities, and outcomes. A mixed-methods evaluation showed that the network has led to new professional connections and enhanced skills (e.g., grant and publication development). As one of four, ongoing, formal programs, the Dissemination and Implementation Research Core (DIRC) was our first major component of D&I infrastructure. DIRC's mission is to accelerate the public health impact of clinical and health services research by increasing the engagement of investigators in later stages of translational research. The aims of DIRC are to advance D&I science and to develop and equip researchers with tools for D&I research. As a second formal component, the Washington University Institute for Public Health has provided significant support for D&I research through pilot projects and a small grants program. In a third set of formal programs, two R25 training grants (one in mental health and one in cancer) support post-doctoral scholars for intensive training and mentoring in D&I science. Finally, our team coordinates closely with D&I functions within research centers across the university. We share a series of challenges and potential solutions.Conclusion: Our experience in developing D&I research at Washington University in St. Louis shows how significant capacity can be built in a relatively short period of time. Many of our ideas and ingredients for success can be replicated, tailored, and improved upon by others. [ABSTRACT FROM AUTHOR]

Background: Incorporating evidence-based integrated treatment for dual disorders into typical care settings has been challenging, especially among those serving Veterans who are homeless. This paper presents an evaluation of an effort to incorporate an evidence-based, dual disorder treatment called Maintaining Independence and Sobriety Through Systems Integration, Outreach, and Networking-Veterans Edition (MISSION-Vet) into case management teams serving Veterans who are homeless, using an implementation strategy called Getting To Outcomes (GTO).Methods: This Hybrid Type III, cluster-randomized controlled trial assessed the impact of GTO over and above MISSION-Vet Implementation as Usual (IU). Both conditions received standard MISSION-Vet training and manuals. The GTO group received an implementation manual, training, technical assistance, and data feedback. The study occurred in teams at three large VA Medical Centers over 2 years. Within each team, existing sub-teams (case managers and Veterans they serve) were the clusters randomly assigned. The trial assessed MISSION-Vet services delivered and collected via administrative data and implementation barriers and facilitators, via semi-structured interview.Results: No case managers in the IU group initiated MISSION-Vet while 68% in the GTO group did. Seven percent of Veterans with case managers in the GTO group received at least one MISSION-Vet session. Most case managers appreciated the MISSION-Vet materials and felt the GTO planning meetings supported using MISSION-Vet. Case manager interviews also showed that MISSION-Vet could be confusing; there was little involvement from leadership after their initial agreement to participate; the data feedback system had a number of difficulties; and case managers did not have the resources to implement all aspects of MISSION-Vet.Conclusions: This project shows that GTO-like support can help launch new practices but that multiple implementation facilitators are needed for successful execution of a complex evidence-based program like MISSION-Vet.Trial Registration: ClinicalTrials.gov NCT01430741. [ABSTRACT FROM AUTHOR]

Background: A 2-day consensus working meeting, hosted by the United States National Institutes of Health and the Veterans Administration, focused on issues related to dissemination and implementation (D&I) research in measurement and reporting. Meeting participants included 23 researchers, practitioners, and decision makers from the USA and Canada who concluded that the field would greatly benefit from measurement resources to enhance the ease, harmonization, and rigor of D&I evaluation efforts. This paper describes the findings from an environmental scan and literature review of resources for D&I measures.Findings: We identified a total of 17 resources, including four web-based repositories and 12 static reviews or tools that attempted to synthesize and evaluate existing measures for D&I research. Thirteen resources came from the health discipline, and 11 were populated from database reviews. Ten focused on quantitative measures, and all were generated as a resource for researchers. Fourteen were organized according to an established D&I theory or framework, with the number of constructs and measures ranging from 1 to more than 450. Measure metadata was quite variable with only six providing information on the psychometric properties of measures.Conclusions: Additional guidance on the development and use of measures are needed. A number of approaches, resources, and critical areas for future work are discussed. Researchers and stakeholders are encouraged to take advantage of a number of funding mechanisms supporting this type of work. [ABSTRACT FROM AUTHOR]

Background: Audit and feedback has been shown to be instrumental in improving quality of care, particularly in outpatient settings. The mental model individuals and organizations hold regarding audit and feedback can moderate its effectiveness, yet this has received limited study in the quality improvement literature. In this study we sought to uncover patterns in mental models of current feedback practices within high- and low-performing healthcare facilities.Methods: We purposively sampled 16 geographically dispersed VA hospitals based on high and low performance on a set of chronic and preventive care measures. We interviewed up to 4 personnel from each location (n = 48) to determine the facility's receptivity to audit and feedback practices. Interview transcripts were analyzed via content and framework analysis to identify emergent themes.Results: We found high variability in the mental models of audit and feedback, which we organized into positive and negative themes. We were unable to associate mental models of audit and feedback with clinical performance due to high variance in facility performance over time. Positive mental models exhibit perceived utility of audit and feedback practices in improving performance; whereas, negative mental models did not.Conclusions: Results speak to the variability of mental models of feedback, highlighting how facilities perceive current audit and feedback practices. Findings are consistent with prior research  in that variability in feedback mental models is associated with lower performance.; Future research should seek to empirically link mental models revealed in this paper to high and low levels of clinical performance. [ABSTRACT FROM AUTHOR]

Background: Training investigators for the rapidly developing field of implementation science requires both mentoring and scientific collaboration. Using social network descriptive analyses, visualization, and modeling, this paper presents results of an evaluation of the mentoring and collaborations fostered over time through the National Institute of Mental Health (NIMH) supported by Implementation Research Institute (IRI).Methods: Data were comprised of IRI participant self-reported collaborations and mentoring relationships, measured in three annual surveys from 2012 to 2014. Network descriptive statistics, visualizations, and network statistical modeling were conducted to examine patterns of mentoring and collaboration among IRI participants and to model the relationship between mentoring and subsequent collaboration.Results: Findings suggest that IRI is successful in forming mentoring relationships among its participants, and that these mentoring relationships are related to future scientific collaborations. Exponential random graph network models demonstrated that mentoring received in 2012 was positively and significantly related to the likelihood of having a scientific collaboration 2 years later in 2014 (p = 0.001). More specifically, mentoring was significantly related to future collaborations focusing on new research (p = 0.009), grant submissions (p = 0.003), and publications (p = 0.017). Predictions based on the network model suggest that for every additional mentoring relationships established in 2012, the likelihood of a scientific collaboration 2 years later is increased by almost 7 %.Conclusions: These results support the importance of mentoring in implementation science specifically and team science more generally. Mentoring relationships were established quickly and early by the IRI core faculty. IRI fellows reported increasing scientific collaboration of all types over time, including starting new research, submitting new grants, presenting research results, and publishing peer-reviewed papers. Statistical network models demonstrated that mentoring was strongly and significantly related to subsequent scientific collaboration, which supported a core design principle of the IRI. Future work should establish the link between mentoring and scientific productivity. These results may be of interest to team science, as they suggest the importance of mentoring for future team collaborations, as well as illustrate the utility of network analysis for studying team characteristics and activities. [ABSTRACT FROM AUTHOR]

Background: Hepatitis C virus (HCV) is a common and highly morbid illness. New medications that have much higher cure rates have become the new evidence-based practice in the field. Understanding the implementation of these new medications nationally provides an opportunity to advance the understanding of the role of implementation strategies in clinical outcomes on a large scale. The Expert Recommendations for Implementing Change (ERIC) study defined discrete implementation strategies and clustered these strategies into groups. The present evaluation assessed the use of these strategies and clusters in the context of HCV treatment across the US Department of Veterans Affairs (VA), Veterans Health Administration, the largest provider of HCV care nationally.Methods: A 73-item survey was developed and sent to all VA sites treating HCV via electronic survey, to assess whether or not a site used each ERIC-defined implementation strategy related to employing the new HCV medication in 2014. VA national data regarding the number of Veterans starting on the new HCV medications at each site were collected. The associations between treatment starts and number and type of implementation strategies were assessed.Results: A total of 80 (62%) sites responded. Respondents endorsed an average of 25 ± 14 strategies. The number of treatment starts was positively correlated with the total number of strategies endorsed (r = 0.43, p < 0.001). Quartile of treatment starts was significantly associated with the number of strategies endorsed (p < 0.01), with the top quartile endorsing a median of 33 strategies, compared to 15 strategies in the lowest quartile. There were significant differences in the types of strategies endorsed by sites in the highest and lowest quartiles of treatment starts. Four of the 10 top strategies for sites in the top quartile had significant correlations with treatment starts compared to only 1 of the 10 top strategies in the bottom quartile sites. Overall, only 3 of the top 15 most frequently used strategies were associated with treatment.Conclusions: These results suggest that sites that used a greater number of implementation strategies were able to deliver more evidence-based treatment in HCV. The current assessment also demonstrates the feasibility of electronic self-reporting to evaluate ERIC strategies on a large scale. These results provide initial evidence for the clinical relevance of the ERIC strategies in a real-world implementation setting on a large scale. This is an initial step in identifying which strategies are associated with the uptake of evidence-based practices in nationwide healthcare systems. [ABSTRACT FROM AUTHOR]

Background: Large-scale implementation of evidence-based psychotherapies (EBPs) such as cognitive processing therapy (CPT) for posttraumatic stress disorder can have a tremendous impact on mental and physical health, healthcare utilization, and quality of life. While many mental health systems (MHS) have invested heavily in programs to implement EBPs, few eligible patients receive EBPs in routine care settings, and clinicians do not appear to deliver the full treatment protocol to many of their patients. Emerging evidence suggests that when CPT and other EBPs are delivered at low levels of fidelity, clinical outcomes are negatively impacted. Thus, identifying strategies to improve and sustain the delivery of CPT and other EBPs is critical. Existing literature has suggested two competing strategies to promote sustainability. One emphasizes fidelity to the treatment protocol through ongoing consultation and fidelity monitoring. The other focuses on improving the fit and effectiveness of these treatments through appropriate adaptations to the treatment or the clinical setting through a process of data-driven, continuous quality improvement. Neither has been evaluated in terms of impact on sustained implementation.Methods: To compare these approaches on the key sustainability outcomes and provide initial guidance on sustainability strategies, we propose a cluster randomized trial with mental health clinics (n = 32) in three diverse MHSs that have implemented CPT. Cohorts of clinicians and clinical managers will participate in 1 year of a fidelity oriented learning collaborative or 1 year of a continuous quality improvement-oriented learning collaborative. Patient-level PTSD symptom change, CPT fidelity and adaptation, penetration, and clinics' capacity to deliver EBP will be examined. Survey and interview data will also be collected to investigate multilevel influences on the success of the two learning collaborative strategies. This research will be conducted by a team of investigators with expertise in CPT implementation, mixed method research strategies, quality improvement, and implementation science, with input from stakeholders in each participating MHS.Discussion: It will have broad implications for supporting ongoing delivery of EBPs in mental health and healthcare systems and settings. The resulting products have the potential to significantly improve efforts to ensure ongoing high quality implementation and consumer access to EBPs.Trial Registration: NCT02449421 . Registered 02/09/2015. [ABSTRACT FROM AUTHOR]

Background: Under the Affordable Care Act, States have obtained Medicaid waivers to overhaul their behavioral health service systems to improve quality and reduce costs. Critical to implementation of broad service delivery reforms has been the preparation of organizations responsible for service delivery. This study focused on one large-scale initiative to overhaul its service system with the goal of improving service quality and reducing costs. The study examined the participation of behavioral health organizations in technical assistance efforts and the extent to which organizational factors related to their participation.Methods: This study matched two datasets to examine the organizational characteristics and training participation for 196 behavioral health organizations. Organizational characteristics were drawn from the Substance Abuse and Mental Health Services Administration National Mental Health Services Survey (N-MHSS). Training variables were drawn from the Clinical Technical Assistance Center's master training database. Chi-square analyses and multivariate logistic regression models were used to examine the proportion of organizations that participated in training, the organizational characteristics (size, population served, service quality, infrastructure) that predicted participation in training, and for those who participated, the type (clinical or business) and intensity of training (webinar, learning collaborative, in-person) they received.Results: Overall 142 (72. 4%) of the sample participated in training. Organizations who pursued training were more likely to be large in size (p = .02), serve children in addition to adults (p < .01), provide child evidence-based practices (p = .01), and use computerized scheduling (p = .01). Of those trained, 95% participated in webinars, 64% participated in learning collaboratives and 35% participated in in-person trainings. More organizations participated in business trainings than clinical (63.8 vs. 59.2%). Organizations serving children had higher odds of participating in both clinical training (OR = 5.91, p < .01) and business training (OR = 4.24, p < .01) than those that did not serve children.Conclusions: The majority of organizations participated in trainings indicating desire for technical assistance to prepare for health care reform. Larger organizations and organizations serving children were more likely to participate potentially indicating increased interest in preparation. Over half participated in business trainings highlighting interest in learning to improve efficiency. Further understanding is needed to support organizational readiness for health care reform initiatives among behavioral health organizations. [ABSTRACT FROM AUTHOR]

Background: The lens of complexity theory is widely advocated to improve health care delivery. However, empirical evidence that this lens has been useful in designing health care remains elusive. This review assesses whether it is possible to reliably capture evidence for efficacy in results or process within interventions that were informed by complexity science and closely related conceptual frameworks.Methods: Systematic searches of scientific and grey literature were undertaken in late 2015/early 2016. Titles and abstracts were screened for interventions (A) delivered by the health services, (B) that explicitly stated that complexity science provided theoretical underpinning, and (C) also reported specific outcomes. Outcomes had to relate to changes in actual practice, service delivery or patient clinical indicators. Data extraction and detailed analysis was undertaken for studies in three developed countries: Canada, UK and USA. Data were extracted for intervention format, barriers encountered and quality aspects (thoroughness or possible biases) of evaluation and reporting.Results: From 5067 initial finds in scientific literature and 171 items in grey literature, 22 interventions described in 29 articles were selected. Most interventions relied on facilitating collaboration to find solutions to specific or general problems. Many outcomes were very positive. However, some outcomes were measured only subjectively, one intervention was designed with complexity theory in mind but did not reiterate this in subsequent evaluation and other interventions were credited as compatible with complexity science but reported no relevant theoretical underpinning. Articles often omitted discussion on implementation barriers or unintended consequences, which suggests that complexity theory was not widely used in evaluation.Conclusions: It is hard to establish cause and effect when attempting to leverage complex adaptive systems and perhaps even harder to reliably find evidence that confirms whether complexity-informed interventions are usually effective. While it is possible to show that interventions that are compatible with complexity science seem efficacious, it remains difficult to show that explicit planning with complexity in mind was particularly valuable. Recommendations are made to improve future evaluation reports, to establish a better evidence base about whether this conceptual framework is useful in intervention design and implementation. [ABSTRACT FROM AUTHOR]

Background: Organizational culture and climate are considered key factors in implementation efforts but have not been examined as moderators of implementation strategy comparative effectiveness. We investigated organizational culture and climate as moderators of comparative effectiveness of two sequences of implementation strategies (Immediate vs. Delayed Enhanced Replicating Effective Programs [REP]) combining Standard REP and REP enhanced with facilitation on implementation of an outreach program for Veterans with serious mental illness lost to care at Veterans Health Administration (VA) facilities nationwide.Methods: This study is a secondary analysis of the cluster-randomized Re-Engage implementation trial that assigned 3075 patients at 89 VA facilities to either the Immediate or Delayed Enhanced REP sequences. We hypothesized that sites with stronger entrepreneurial culture, task, or relational climate would benefit more from Enhanced REP than Standard REP. Veteran- and site-level data from the Re-Engage trial were combined with site-aggregated measures of entrepreneurial culture and task and relational climate from the 2012 VA All Employee Survey. Longitudinal mixed-effects logistic models examined whether the comparative effectiveness of the Immediate vs. Delayed Enhanced REP sequences were moderated by culture or climate measures at 6 and 12 months post-randomization. Three Veteran-level outcomes related to the engagement with the VA system were assessed: updated documentation, attempted contact by coordinator, and completed contact.Results: For updated documentation and attempted contact, Veterans at sites with higher entrepreneurial culture and task climate scores benefitted more from Enhanced REP compared to Standard REP than Veterans at sites with lower scores. Few culture or climate moderation effects were detected for the comparative effectiveness of the full sequences of implementation strategies.Conclusions: Implementation strategy effectiveness is highly intertwined with contextual factors, and implementation practitioners may use knowledge of contextual moderation to tailor strategy deployment. We found that facilitation strategies provided with Enhanced REP were more effective at improving uptake of a mental health outreach program at sites with stronger entrepreneurial culture and task climate; Veterans at sites with lower levels of these measures saw more similar improvement under Standard and Enhanced REP. Within resource-constrained systems, practitioners may choose to target more intensive implementation strategies to sites that will most benefit from them.Trial Registration: ISRCTN: ISRCTN21059161 . Date registered: April 11, 2013. [ABSTRACT FROM AUTHOR]