Search

BackgroundOpinion leadership, educational outreach visiting, and innovation championing are commonly used strategies to address barriers to implementing innovations and evidence-based practices in healthcare settings. Despite voluminous research, ambiguities persist in how these strategies work and under what conditions they work well, work poorly, or work at all. The current paper develops middle-range theories to address this gap.MethodsConceptual articles, systematic reviews, and empirical studies informed the development of causal pathway diagrams (CPDs). CPDs are visualization tools for depicting and theorizing about the causal process through which strategies operate, including the mechanisms they activate, the barriers they address, and the proximal and distal outcomes they produce. CPDs also clarify the contextual conditions (i.e., preconditions and moderators) that influence whether, and to what extent, the strategy's causal process unfolds successfully. Expert panels of implementation scientists and health professionals rated the plausibility of these preliminary CPDs and offered comments and suggestions on them.FindingsTheoretically, opinion leadership addresses potential adopters' uncertainty about likely consequences of innovation use (determinant) by promoting positive attitude formation about the innovation (mechanism), which results in an adoption decision (proximal outcome), which leads to innovation use (intermediate outcome). As this causal process repeats, penetration, or spread of innovation use, occurs (distal outcome). Educational outreach visiting addresses knowledge barriers, attitudinal barriers, and behavioral barriers (determinants) by promoting critical thinking and reflection about evidence and practice (mechanism), which results in behavioral intention (proximal outcome), behavior change (intermediate outcome), and fidelity, or guideline adherence (distal outcome). Innovation championing addresses organizational inertia, indifference, and resistance (determinants) by promoting buy-in to the vision, fostering a positive implementation climate, and increasing collective efficacy (mechanisms), which leads to participation in implementation activities (proximal outcome), initial use of the innovation with increasing skill (intermediate outcome) and, ultimately, greater penetration and fidelity (distal outcomes). Experts found the preliminary CPDs plausible or highly plausible and suggested additional mechanisms, moderators, and preconditions, which were used to amend the initial CPD.DiscussionThe middle-range theories depicted in the CPDs furnish testable propositions for implementation research and offer guidance for selecting, designing, and evaluating these social influence implementation strategies in both research studies and practice settings.

Abstract Background Although healthcare is delivered in inherently multilevel contexts, implementation science has no widely endorsed methodological standards defining the characteristics of rigorous, multilevel implementation research. We identify and describe eight characteristics of high-quality, multilevel implementation research to encourage discussion, spur debate, and guide decision-making around study design and methodological issues. Recommendations Implementation researchers who conduct rigorous multilevel implementation research demonstrate the following eight characteristics. First, they map and operationalize the specific multilevel context for defined populations and settings. Second, they define and state the level of each construct under study. Third, they describe how constructs relate to each other within and across levels. Fourth, they specify the temporal scope of each phenomenon at each relevant level. Fifth, they align measurement choices and construction of analytic variables with the levels of theories selected (and hypotheses generated, if applicable). Sixth, they use a sampling strategy consistent with the selected theories or research objectives and sufficiently large and variable to examine relationships at requisite levels. Seventh, they align analytic approaches with the chosen theories (and hypotheses, if applicable), ensuring that they account for measurement dependencies and nested data structures. Eighth, they ensure inferences are made at the appropriate level. To guide implementation researchers and encourage debate, we present the rationale for each characteristic, actionable recommendations for operationalizing the characteristics in implementation research, a range of examples, and references to make the characteristics more usable. Our recommendations apply to all types of multilevel implementation study designs and approaches, including randomized trials, quantitative and qualitative observational studies, and mixed methods. Conclusion These eight characteristics provide benchmarks for evaluating the quality and replicability of multilevel implementation research and promote a common language and reference points. This, in turn, facilitates knowledge generation across diverse multilevel settings and ensures that implementation research is consistent with (and appropriately leverages) what has already been learned in allied multilevel sciences. When a shared and integrated description of what constitutes rigor is defined and broadly communicated, implementation science is better positioned to innovate both methodologically and theoretically.

Abstract Background Engaging policy actors in research design and execution is critical to increasing the practical relevance and real-world impact of policy-focused dissemination and implementation science. Identifying and selecting which policy actors to engage, particularly actors involved in “Big P” public policies such as laws, is distinct from traditional engaged research methods. This current study aimed to develop a transparent, structured method for iteratively identifying policy actors involved in key policy decisions—such as adopting evidence-based interventions at systems-scale—and to guide implementation study sampling and engagement approaches. A flexible policy actor taxonomy was developed to supplement existing methods and help identify policy developers, disseminators, implementers, enforcers, and influencers. Methods A five-step methodology for identifying policy actors to potentially engage in policy dissemination and implementation research was developed. Leveraging a recent federal policy as a case study—The Family First Prevention Services Act (FFPSA)—publicly available documentation (e.g., websites, reports) were searched, retrieved, and coded using content analysis to characterize the organizations and individual policy actors in the “room” during policy decisions. Results The five steps are as follows: (1) clarify the policy implementation phase(s) of interest, (2) identify relevant proverbial or actual policymaking “rooms,” (3) identify and characterize organizations in the room, (4) identify and characterize policy actors in the “room,” and (5) quantify (e.g., count actors across groups), summarize, and compare “rooms” to develop or select engagement approaches aligned with the “room” and actors. The use and outcomes of each step are exemplified through the FFPSA case study. Conclusions The pragmatic and transparent policy actor identification steps presented here can guide researchers’ methods for continuous sampling and successful policy actor engagement. Future work should explore the utility of the proposed methods for guiding selection and tailoring of engagement and implementation strategies (e.g., research-policy actor partnerships) to improve both “Big P” and “little p” (administrative guidelines, procedures) policymaking and implementation in global contexts.

Abstract Background Proctor and colleagues’ 2011 paper proposed a taxonomy of eight implementation outcomes and challenged the field to address a research agenda focused on conceptualization, measurement, and theory building. Ten years later, this paper maps the field’s progress in implementation outcomes research. This scoping review describes how each implementation outcome has been studied, research designs and methods used, and the contexts and settings represented in the current literature. We also describe the role of implementation outcomes in relation to implementation strategies and other outcomes. Methods Arksey and O’Malley’s framework for conducting scoping reviews guided our methods. Using forward citation tracing, we identified all literature citing the 2011 paper. We conducted our search in the Web of Science (WOS) database and added citation alerts sent to the first author from the publisher for a 6-month period coinciding with the WOS citation search. This produced 1346 titles and abstracts. Initial abstract screening yielded 480 manuscripts, and full-text review yielded 400 manuscripts that met inclusion criteria (empirical assessment of at least one implementation outcome). Results Slightly more than half (52.1%) of included manuscripts examined acceptability. Fidelity (39.3%), feasibility (38.6%), adoption (26.5%), and appropriateness (21.8%) were also commonly examined. Penetration (16.0%), sustainability (15.8%), and cost (7.8%) were less frequently examined. Thirty-two manuscripts examined implementation outcomes not included in the original taxonomy. Most studies took place in healthcare (45.8%) or behavioral health (22.5%) organizations. Two-thirds used observational designs. We found little evidence of progress in testing the relationships between implementation strategies and implementation outcomes, leaving us ill-prepared to know how to achieve implementation success. Moreover, few studies tested the impact of implementation outcomes on other important outcome types, such as service systems and improved individual or population health. Conclusions Our review presents a comprehensive snapshot of the research questions being addressed by existing implementation outcomes literature and reveals the need for rigorous, analytic research and tests of strategies for attaining implementation outcomes in the next 10 years of outcomes research.

Abstract Background Implementation science aims to accelerate the public health impact of evidence-based interventions. However, implementation science has had too little focus on the role of health policy — and its inseparable politics, polity structures, and policymakers — in the implementation and sustainment of evidence-based healthcare. Policies can serve as determinants, implementation strategies, the evidence-based “thing” to be implemented, or another variable in the causal pathway to healthcare access, quality, and patient outcomes. Research describing the roles of policy in dissemination and implementation (D&I) efforts is needed to resolve persistent knowledge gaps about policymakers’ evidence use, how evidence-based policies are implemented and sustained, and methods to de-implement policies that are ineffective or cause harm. Few D&I theories, models, or frameworks (TMF) explicitly guide researchers in conceptualizing where, how, and when policy should be empirically investigated. We conducted and reflected on the results of a scoping review to identify gaps of existing Exploration, Preparation, Implementation, and Sustainment (EPIS) framework-guided policy D&I studies. We argue that rather than creating new TMF, researchers should optimize existing TMF to examine policy’s role in D&I. We describe six recommendations to help researchers optimize existing D&I TMF. Recommendations are applied to EPIS, as one example for advancing TMF for policy D&I. Recommendations (1) Specify dimensions of a policy’s function (policy goals, type, contexts, capital exchanged). (2) Specify dimensions of a policy’s form (origin, structure, dynamism, outcomes). (3) Identify and define the nonlinear phases of policy D&I across outer and inner contexts. (4) Describe the temporal roles that stakeholders play in policy D&I over time. (5) Consider policy-relevant outer and inner context adaptations. (6) Identify and describe bridging factors necessary for policy D&I success. Conclusion Researchers should use TMF to meaningfully conceptualize policy’s role in D&I efforts to accelerate the public health impact of evidence-based policies or practices and de-implement ineffective and harmful policies. Applying these six recommendations to existing D&I TMF advances existing theoretical knowledge, especially EPIS application, rather than introducing new models. Using these recommendations will sensitize researchers to help them investigate the multifaceted roles policy can play within a causal pathway leading to D&I success.

Abstract Background There is a fundamental gap in understanding the causal mechanisms by which strategies for implementing evidence-based practices address local barriers to effective, appropriate service delivery. Until this gap is addressed, scientific knowledge and practical guidance about which implementation strategies to use in which contexts will remain elusive. This research project aims to identify plausible strategy-mechanism linkages, develop causal models for mechanism evaluation, produce measures needed to evaluate such linkages, and make these models, methods, and measures available in a user-friendly website. The specific aims are as follows: (1) build a database of strategy-mechanism linkages and associated causal pathway diagrams, (2) develop psychometrically strong, pragmatic measures of mechanisms, and (3) develop and disseminate a website of implementation mechanisms knowledge for use by diverse stakeholders. Methods For the first aim, a combination of qualitative inquiry, expert panel methods, and causal pathway diagramming will be used to identify and confirm plausible strategy-mechanism linkages and articulate moderators, preconditions, and proximal and distal outcomes associated with those linkages. For the second aim, rapid-cycle measure development and testing methods will be employed to create reliable, valid, pragmatic measures of six mechanisms of common strategies for which no high-quality measures exist. For the third aim, we will develop a user-friendly website and searchable database that incorporates user-centered design, disseminating the final product using social marketing principles. Discussion Once strategy-mechanism linkages are identified using this multi-method approach, implementation scientists can use the searchable database to develop tailored implementation strategies and generate more robust evidence about which strategies work best in which contexts. Moreover, practitioners will be better able to select implementation strategies to address their specific implementation problems. New horizons in implementation strategy development, optimization, evaluation, and deployment are expected to be more attainable as a result of this research, which will lead to enhanced implementation of evidence-based interventions for cancer control, and ultimately improvements in patient outcomes.

Abstract Background This study will explore implementation mechanisms through which a single implementation strategy and a multifaceted implementation strategy operate to affect the implementation outcome, which is fidelity to the Guideline For The Prevention of Mental Ill Health within schools. The guideline gives recommendations on how workplaces can prevent mental ill health among their personnel by managing social and organizational risks factors in the work environment. Schools are chosen as the setting for the study due to the high prevalence of mental ill health among teachers and other personnel working in schools. The study builds on our previous research, in which we compared the effectiveness of the two strategies on fidelity to the guideline. Small improvements in guideline adherence were observed for the majority of the indicators in the multifaceted strategy group. This study will focus on exploring the underlying mechanisms of change through which the implementation strategies may operate to affect the implementation outcome. Methods We will conduct a cluster-randomized-controlled trial among public schools (n=55 schools) in Sweden. Schools are randomized (1:1 ratio) to receive a multifaceted strategy (implementation teams, educational meeting, ongoing training, Plan-Do-Study-Act cycles) or a single strategy (implementation teams, educational meeting). The implementation outcome is fidelity to the guideline. Hypothesized mediators originate from the COM-B model. A mixed-method design will be employed, entailing a qualitative study of implementation process embedded within the cluster-randomized controlled trail examining implementation mechanisms. The methods will be used in a complementary manner to get a full understanding of the implementation mechanisms. Discussion This implementation study will provide valuable knowledge on how implementation strategies work (or fail) to affect implementation outcomes. The knowledge gained will aid the selection of effective implementation strategies that fit specific determinants, which is a priority for the field. Despite recent initiatives to advance the understanding of implementation mechanisms, studies testing these mechanisms are still uncommon. Trial registration ClinicalTrials.org dr.nr 2020-01214.

Abstract Background Implementation outcomes research spans an exciting mix of fields, disciplines, and geographical space. Although the number of studies that cite the 2011 taxonomy has expanded considerably, the problem of harmony in describing outcomes persists. This paper revisits that problem by focusing on the clarity of reporting outcomes in studies that examine them. Published recommendations for improved reporting and specification have proven to be an important step in enhancing the rigor of implementation research. We articulate reporting problems in the current implementation outcomes literature and describe six practical recommendations that address them. Recommendations Our first recommendation is to clearly state each implementation outcome and provide a definition that the study will consistently use. This includes providing an explanation if using the taxonomy in a new way or merging terms. Our second recommendation is to specify how each implementation outcome will be analyzed relative to other constructs. Our third recommendation is to specify “the thing” that each implementation outcome will be measured in relation to. This is especially important if you are concurrently studying interventions and strategies, or if you are studying interventions and strategies that have multiple components. Our fourth recommendation is to report who will provide data and the level at which data will be collected for each implementation outcome, and to report what kind of data will be collected and used to assess each implementation outcome. Our fifth recommendation is to state the number of time points and frequency at which each outcome will be measured. Our sixth recommendation is to state the unit of observation and the level of analysis for each implementation outcome. Conclusion This paper advances implementation outcomes research in two ways. First, we illustrate elements of the 2011 research agenda with concrete examples drawn from a wide swath of current literature. Second, we provide six pragmatic recommendations for improved reporting. These recommendations are accompanied by an audit worksheet and a list of exemplar articles that researchers can use when designing, conducting, and assessing implementation outcomes studies.

Abstract Background The COVID-19 pandemic has created a crisis in access to addiction treatment. Programs with residential components have been particularly impacted as they try to keep infection from spreading in facilities and contributing to further community spread of the virus. This crisis highlights the ongoing daily trade-offs that organizations must weigh as they balance the risks and benefits of individual patients with those of the group of patients, staff and the community they serve. Main body The COVID-19 pandemic has forced provider organizations to make individual facility level decisions about how to manage patients who are COVID-19 positive while protecting other patients, staff and the community. While guidance documents from federal, state, and trade groups aimed to support such decision making, they often lagged pandemic dynamics, and provided too little detail to translate into front line decision making. In the context of incomplete knowledge to make informed decisions, we present a way to integrate guidelines and local data into the decision process and discuss the ethical dilemmas faced by provider organizations in preventing infections and responding to COVID positive patients or staff. Conclusion and commentary Provider organizations need decision support on managing the risk of COVID-19 positive patients in their milieu. While useful, guidance documents may not be capable of providing support with the nuance that local data and simulation modeling may be able to provide.

Abstract Background Bridging factors are relational ties, formal arrangements, and processes that connect outer system and inner organizational contexts. They may be critical drivers of evidence-based practice (EBP) implementation and sustainment. Yet, the complex interplay between outer and inner contexts is often not considered. Bridging factors were recently defined in the updated Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Further identification and specification of this construct will advance implementation models, measures, and methods. Our goal is to advance bridging factor research by identifying relevant dimensions and exemplifying these dimensions through illustrative case studies. Methods We used a multiple case study design. Each case (n = 10) represented different contexts, EBPs, and bridging factor types. Inclusion criteria were the presence of clearly distinguishable outer and inner contexts, identifiable bridging factor, sufficient information to describe how the bridging factor affected implementation, and variation from other cases. We used an iterative qualitative inquiry process to develop and refine a list of dimensions. Case data were entered into a matrix. Dimensions comprised the rows and case details comprised the columns. After a review of all cases, we collectively considered and independently coded each dimension as function or form. Results We drew upon the concepts of functions and forms, a distinction originally proposed in the complex health intervention literature. Function dimensions help define the bridging factor and illustrate its purpose as it relates to EBP implementation. Form dimensions describe the specific structures and activities that illustrate why and how the bridging factor has been customized to a local implementation experience. Function dimensions can help researchers and practitioners identify the presence and purpose of bridging factors, whereas form dimensions can help us understand how the bridging factor may be designed or modified to support EBP implementation in a specific context. We propose five function and three form bridging factor dimensions. Conclusions Bridging factors are described in many implementation models and studies, but without explicit reference or investigation. Bridging factors are an understudied and critical construct that requires further attention to facilitate implementation research and practice. We present specific recommendations for a bridging factors research agenda.

Background: Adaptation is an accepted part of implementing evidence-based practices. COVID-19 presented a unique opportunity to examine adaptation in evolving contexts. Delivering service to people with opioid use disorder during the pandemic required significant adaptation due to revised regulations and limited service access. This report evaluated changes to addiction medication services caused by the pandemic, challenges encountered in rapidly adapting service delivery, and initial impressions of which changes might be sustainable over time. Methods: Qualitatively-evaluated structured interviews (N = 20) were conducted in late 2020 with key informants in Pinellas County (FL) to assess the pandemic’s impact. Interviewees represented a cross-section of the professional groups including direct SUD/HIV service providers, and sheriff’s office, Department of Health, and regional clinical program administrative staff. The interview questions examined significant changes necessitated by the pandemic, challenges encountered in adapting to this evolving context, and considerations for sustained change. Results: The most significant changes to service delivery identified were rapid adaptation to a telehealth format, and modifying service consistent with SAMHSA guidance, to allow for ‘take-home’ doses of methadone. Limitations imposed by access to technology, and the retraining of staff and patients to give and receive service differently were the most common themes identified as challenging adaptation efforts. Respondents saw shifts towards telehealth as most likely to being sustained. Conclusions: COVID-19 provided an unprecedented opportunity to examine adaptation in a fast-paced, dynamic, and evolving context. Adaptations identified will only be sustained through multisystem collaboration and validation. Results suggest that additional components could be added to implementation frameworks to assess rapid adaptation during unplanned events, such as access to additional resources or local decision-making that impacts service delivery. Findings will also be integrated with quantitative data to help inform local policy decisions. Plain Language Summary: Adaptation is an accepted part of implementing evidencebased practices. COVID-19 presented a unique opportunity to examine rapid adaptation necessitated within evolving contexts. Delivering services to people with opioid use disorder required significant adaptation due to changing regulations and limited access to lifesaving services. This study examined changes in service delivery due to the pandemic, challenges encountered in rapid adaptation, and initial impressions of which changes might be sustainable over time. Qualitatively-evaluated structured interviews were conducted with a cross-section of professional groups (direct substance use disorder (SUD) and human immunodeficiency virus (HIV) service providers, and sheriff's office, Department of Health, and clinical program administrative staff) in Pinellas County (FL). The most significant changes to service delivery were rapid adaptation to a telehealth format and increased allowance for 'takehome' doses of methadone medication. Limitations imposed by access to technology, as well as the education of and staff and patients were the most common themes identified as challenges. Respondents saw shifts towards telehealth as most likely to be sustained. COVID-19 provided an unprecedented opportunity to examine adaptation in a fast-paced, dynamic, and evolving context. Adaptations will only be sustained through multisystem collaboration and validation. Findings suggest that additional components could be added to implementation frameworks to assess rapid adaptation during unplanned events, such as access to additional resources or local decision-making that impacts service delivery.

Abstract Background The majority of literature on evidence-based practice (EBP) adaptation focuses on changes to clinical practices without explicitly addressing how organizations must adapt to accommodate a new EBP. This study explores the process of organizational—rather than EBP—adaptation during implementation and sustainment. To the authors’ knowledge, there are no previous implementation studies that focus on organizational adaptation in this way. Methods This analysis utilizes a case study approach to examine longitudinal qualitative data from 17 community-based organizations (CBOs) in one state and seven county-based child welfare systems. The CBOs had sustained a child-neglect intervention EBP (SafeCare®) for 2 to 10 years. The unit of analysis was the organization, and each CBO represented a case. Organizational-level profiles were created to describe the organizational adaptation process. Results Three organizational-level adaptation profiles were identified as follows: incorporators (n = 7), early investors (n = 6), and learners (n = 4). Incorporators adapted by integrating SafeCare into existing operations to meet contractual or EBP fidelity requirements. Early Investors made substantial organizational adaptations during the early implementation period, then operated relatively consistently as the EBP became embedded in the organization and service system. Learners were characterized by steady and continuous attention to new ways that the organization could adapt to support the EBP. Conclusion The profiles demonstrated that there can be multiple effective paths to EBP sustainment. Organizational adaptation was calibrated to fit a CBO’s operations (e.g., size of the program) and immediate environmental constraints (e.g., funding levels). Additionally, organizations fulfilled different functional roles in the network of entities involved in EBP implementation. Knowing organizational roles and adaptation profiles can guide implementation planning and help to structure contract designs that bridge the outer (system) and inner (organizational) contexts. Adaptation profiles can also inform the intensity of the implementation strategy tailoring process and the way that strategies are marketed to organizations.

Abstract Background Bridging factors are relational ties (e.g. partnerships), formal arrangements (e.g. contracts or polices) and processes (e.g. data sharing agreements) linking outer and inner contexts and are a recent evolution of the Exploration-Preparation-Implementation-Sustainment (EPIS) framework. Bridging factor research can elucidate ways that service systems may influence and/or be influenced by organizations providing health services. This study used the EPIS framework and open systems and resource dependence theoretical approaches to examine contracting arrangements in U.S. public sector systems. Contracting arrangements function as bridging factors through which systems communicate, interact, and exchange resources with the organizations operating within them. Methods The sample included 17 community-based organizations in eight service systems. Longitudinal data is derived from 113 contract documents and 88 qualitative interviews and focus groups involving system and organizational stakeholders. Analyses consisted of a document review using content analysis and focused coding of transcripts from the interviews and focus groups. A multiple case study analysis was conducted to identify patterns across service systems and organizations. The dataset represented service systems that had sustained the same EBP for between 2 and 10 years, which allowed for observation of bridging factors and outer-inner context interactions over time. Results Service systems and organizations influenced each other in a number of ways through contracting arrangements. Service systems influenced organizations when contracting arrangements resulted in changes to organizational functioning, required organizational responses to insufficient funding, and altered interorganizational network relationships. Organizations influenced service systems when contract arrangements prompted organization-driven contract negotiation/tailoring, changes to system-level processes, and interorganizational collaboration. Service systems and organizations were dependent on each other as implementation progressed. Resources beyond funding emerged, including adequate numbers of eligible clients, expertise in the evidence-based practice, and training and coaching capacity. Conclusion This study advances implementation science by expanding the range and definition of bridging factors and illustrating specific bi-directional influences between outer context service systems and inner context organizations. This study also identifies bi-directional dependencies over the course of implementation and sustainment. An analysis of influence, dependencies, and resources exchanged through bridging factors has direct implications for selecting and tailoring implementation strategies, especially those that require system-level coordination and change.

Abstract Background Implementing and sustaining evidence-based programs with fidelity may require multiple implementation strategies tailored to address multi-level, context-specific barriers and facilitators. Ideally, selecting and tailoring implementation strategies should be guided by theory, evidence, and input from relevant stakeholders; however, methods to guide the selection and tailoring of strategies are not well-developed. There is a need for more rigorous methods for assessing and prioritizing implementation determinants (barriers and facilitators) and linking implementation strategies to determinants. The Collaborative Organizational Approach to Selecting and Tailoring Implementation Strategies (COAST-IS) is an intervention designed to increase the effectiveness of evidence-based practice implementation and sustainment. COAST-IS will enable organizational leaders and clinicians to use Intervention Mapping to select and tailor implementation strategies to address their site-specific needs. Intervention Mapping is a multi-step process that incorporates theory, evidence, and stakeholder perspectives to ensure that implementation strategies effectively address key determinants of change. Methods COAST-IS will be piloted with community mental health organizations that are working to address the needs of children and youth who experience trauma-related emotional or behavioral difficulties by engaging in a learning collaborative to implement an evidence-based psychosocial intervention (trauma-focused cognitive behavioral therapy). Organizations will be matched and then randomized to participate in the learning collaborative only (control) or to receive additional support through COAST-IS. The primary aims of this study are to (1) assess the acceptability, appropriateness, feasibility, and perceived utility of COAST-IS; (2) evaluate the organizational stakeholders’ fidelity to the core elements of COAST-IS; and (3) demonstrate the feasibility of testing COAST-IS in a larger effectiveness trial. Discussion COAST-IS is a systematic method that integrates theory, evidence, and stakeholder perspectives to improve the effectiveness and precision of implementation strategies. If effective, COAST-IS has the potential to improve the implementation and sustainment of a wide range of evidence-based practices in mental health and other sectors. Trial registration This study was registered in ClinicalTrials.gov ( NCT03799432 ) on January 10, 2019 (last updated August 5, 2019).

Technology is one medium to increase youth engagement, especially among underserved and minority groups, in suicide preventive interventions. Technology can be used to supplement or adjunct an in-person intervention, guide an in-person intervention, or be the stand-alone (automated) component of the intervention. This range in technological use is now called the continuum of behavioral intervention technologies (BITs). Overall, suicide intervention researchers do not use this terminology to categorize how the role of technology differs across technology-enhanced youth interventions. There is growing recognition that technology-enhanced interventions will not create substantial public health impact without an understanding of the individual (youth, families, and providers), mezzo (clinics and health systems of care), and contextual factors (society, culture, community) that are associated with their implementation. Implementation science is the study of methods to promote uptake of evidence-based practices and policies into the broader health care system. In this review, we incorporate work from implementation science and BIT implementation to illustrate how the study of technology-enhanced interventions for youth suicide can be advanced by specifying the role of technology and measuring implementation outcomes.

Background: Organizational culture, organizational climate, and implementation climate are key organizational constructs that influence the implementation of evidence-based practices. However, there has been little systematic investigation of the availability of psychometrically strong measures that can be used to assess these constructs in behavioral health. This systematic review identified and assessed the psychometric properties of measures of organizational culture, organizational climate, implementation climate, and related subconstructs as defined by the Consolidated Framework for Implementation Research (CFIR) and Ehrhart and colleagues. Methods: Data collection involved search string generation, title and abstract screening, full-text review, construct assignment, and citation searches for all known empirical uses. Data relevant to nine psychometric criteria from the Psychometric and Pragmatic Evidence Rating Scale (PAPERS) were extracted: internal consistency, convergent validity, discriminant validity, known-groups validity, predictive validity, concurrent validity, structural validity, responsiveness, and norms. Extracted data for each criterion were rated on a scale from −1 (“poor”) to 4 (“excellent”), and each measure was assigned a total score (highest possible score = 36) that formed the basis for head-to-head comparisons of measures for each focal construct. Results: We identified full measures or relevant subscales of broader measures for organizational culture ( n = 21), organizational climate ( n = 36), implementation climate ( n = 2), tension for change ( n = 2), compatibility ( n = 6), relative priority ( n = 2), organizational incentives and rewards ( n = 3), goals and feedback ( n = 3), and learning climate ( n = 2). Psychometric evidence was most frequently available for internal consistency and norms. Information about other psychometric properties was less available. Median ratings for psychometric properties across categories of measures ranged from “poor” to “good.” There was limited evidence of responsiveness or predictive validity. Conclusion: While several promising measures were identified, the overall state of measurement related to these constructs is poor. To enhance understanding of how these constructs influence implementation research and practice, measures that are sensitive to change and predictive of key implementation and clinical outcomes are required. There is a need for further testing of the most promising measures, and ample opportunity to develop additional psychometrically strong measures of these important constructs. Plain Language Summary Organizational culture, organizational climate, and implementation climate can play a critical role in facilitating or impeding the successful implementation and sustainment of evidence-based practices. Advancing our understanding of how these contextual factors independently or collectively influence implementation and clinical outcomes requires measures that are reliable and valid. Previous systematic reviews identified measures of organizational factors that influence implementation, but none focused explicitly on behavioral health; focused solely on organizational culture, organizational climate, and implementation climate; or assessed the evidence base of all known uses of a measure within a given area, such as behavioral health–focused implementation efforts. The purpose of this study was to identify and assess the psychometric properties of measures of organizational culture, organizational climate, implementation climate, and related subconstructs that have been used in behavioral health-focused implementation research. We identified 21 measures of organizational culture, 36 measures of organizational climate, 2 measures of implementation climate, 2 measures of tension for change, 6 measures of compatibility, 2 measures of relative priority, 3 measures of organizational incentives and rewards, 3 measures of goals and feedback, and 2 measures of learning climate. Some promising measures were identified; however, the overall state of measurement across these constructs is poor. This review highlights specific areas for improvement and suggests the need to rigorously evaluate existing measures and develop new measures.

Introduction A 2011 paper proposed a working taxonomy of implementation outcomes, their conceptual distinctions and a two-pronged research agenda on their role in implementation success. Since then, over 1100 papers citing the manuscript have been published. Our goal is to compare the field’s progress to the originally proposed research agenda, and outline recommendations for the next 10 years. To accomplish this, we are conducting the proposed scoping review.Methods and analysis Our approach is informed by Arksey and O’Malley’s methodological framework for conducting scoping reviews. We will adhere to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews. We first aim to assess the degree to which each implementation outcome has been investigated in the literature, including healthcare settings, clinical populations and innovations represented. We next aim to describe the relationship between implementation strategies and outcomes. Our last aim is to identify studies that empirically assess relationships among implementation and/or service and client outcomes. We will use a forward citation tracing approach to identify all literature that cited the 2011 paper in the Web of Science (WOS) and will supplement this with citation alerts sent to the second author for a 6-month period coinciding with the WOS citation search. Our review will focus on empirical studies that are designed to assess at least one of the identified implementation outcomes in the 2011 taxonomy and are published in peer-reviewed journals. We will generate descriptive statistics from extracted data and organise results by these research aims.Ethics and dissemination No human research participants will be involved in this review. We plan to share findings through a variety of means including peer-reviewed journal publications, national conference presentations, invited workshops and webinars, email listservs affiliated with our institutions and professional associations, and academic social media.

Abstract Background: Understanding how to translate research discoveries into solutions for healthcare improvement is a priority of NIH-funded Clinical and Translational Science Awards (CTSA). This study, supported by one CTSA, aims to capture one process of shaping and implementing innovations to advance the timeliness and patient-centeredness of cardiovascular care. Specifically, we sought to understand a partnership between a private digital health startup company, a university innovation lab, and an academic health system’s cardiology program pursuing this goal. Findings: The collaboration proceeded through clear phases to address the questions and challenges: problem definition, exploration and formalization of the partnership, innovation co-creation and pilot test, and scale-up planning. Phases were punctuated by key decisions, such as forming the partnership, negotiating terms of the partnership, iterating form and features of the innovation, and exploring sufficiency of its value-add for scale-up and sustainment. Key implementation concepts were apparent, including implementation strategies (e.g., champions and iterative trialing) and the implementation outcomes of acceptability, sustainment, and scale-up. Participants identified potential risks of collaboration, reflected on their co-creation process, and the value of engaging stakeholders in innovation design. Findings may inform subsequent collaborations between innovators and translational researchers. Methods: We conducted a case study to understand the partnership; characterize the questions they pursued, their decision points, information and data sources; and identify the challenges and risks. Data were collected through a series of four focus groups with members of each partnering organization. A transdisciplinary research team iteratively worked to condense and synthesize data from audio recorded transcripts into a case narrative.