Search

Background: Latinos in the United States face multiple barriers to engaging in physical activity (PA). We implemented a faith-based multilevel intervention to promote PA in parks for Latino adults, which was partially adapted to a virtual platform during the COVID-19 pandemic, and evaluated it using the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework. Methods: We conducted in-depth semistructured interviews (83% in Spanish) with 24 intervention participants (75% women) participating in a cluster randomized controlled trial in 2019–2022 that linked 6 churches (3 intervention, 3 control) with parks in East Los Angeles, CA. The intervention included in-person, park-based fitness classes, which were adapted to Facebook during the pandemic; PA motivational text messages; and other activities. Interviews assessed Reach (participation), Effectiveness (perceived impacts), Implementation (participation barriers/facilitators), and Maintenance (plans for sustaining PA), as well as perceived pandemic impacts. Results: About 80% of interviewees participated in ≥1 park class and 67% in ≥1 virtual class (Reach). Interviewees perceived positive intervention impacts across multiple health and well-being domains (Effectiveness) despite perceived negative pandemic impacts; several facilitators to participation (personal, social, program) and few barriers (personal, virtual, environmental; Implementation); and plans for maintaining PA (eg, revisiting intervention text messages and video recordings; Maintenance). Conclusions: Findings support the utility of Reach, Effectiveness, Adoption, Implementation, and Maintenance to understand the broad impacts of a faith-based PA intervention. Findings point to the adaptability and robustness of the intervention during a public health crisis. Overall, findings may help inform the translation of the intervention to other communities to advance health equity. [ABSTRACT FROM AUTHOR]

BACKGROUND: Pediatric depression is a global concern that has fueled efforts for enhanced detection and treatment engagement. As one example, the US Preventive Services Task Force recommends depression screening for adolescents ages 12-18 years. While many health systems have implemented components of depression screening protocols, there is limited evidence of effective follow-up for pediatric depression. A key barrier is timely team communication and coordination across clinicians and staff within and across service areas for prompt service linkage. However, team effectiveness interventions have been shown to improve team processes and outcomes and can be applied in healthcare settings. METHODS: This project aims to refine and test a team communication training implementation strategy to improve implementation of an existing pediatric depression screening protocol in a large pediatric healthcare system. The team will be defined as part of the study but is expected to include medical assistants, nurses, physicians, and behavioral health clinicians within and across departments. The implementation strategy will target team mechanisms at the team-level (i.e., intra-organizational alignment and implementation climate) and team member-level (i.e., communication, coordination, psychological safety, and shared cognition). First, the project will use mixed methods to refine the team training strategy to fit the organizational context and workflows. Next, a hybrid type 3 implementation-effectiveness pilot trial will assess the initial effectiveness of the team communication training (implementation strategy) paired with the current universal depression screening protocol (clinical intervention) on implementation outcomes (i.e., feasibility, acceptability, appropriateness, workflow efficiency) and clinical/services outcomes (increased frequency of needed screening and reduced time to service linkage). Finally, the study will assess mechanisms at the team and team member levels that may affect implementation outcomes. DISCUSSION: Team communication training is hypothesized to lead to improved, efficient, and effective decision-making to increase the compliance with depression screening and timely service linkage. Findings are expected to yield better understanding and examples of how to optimize team communication to improve efficiency and effectiveness in the pediatric depression screening-to-treatment cascade. This should also culminate in improved implementation outcomes including patient engagement critical to address the youth mental health crisis. TRIAL REGISTRATION: NCT06527196. Trial Sponsor: University of California San Diego.

BACKGROUND: Goals of care conversations explore seriously ill patients values to guide medical decision making and often inform decisions about life sustaining treatments. Ideally, conversations occur before a health crisis between patients and clinicians in the outpatient setting. In the United States Veterans Affairs (VA) healthcare system, most conversations still occur in the inpatient setting. Strategies are needed to improve implementation of outpatient, primary care goals of care conversations. METHODS: We plan a cluster randomized (clinician-level) sequential, multiple assignment randomized trial to evaluate the effectiveness of patient implementation strategies on the outcome of goals of care conversation documentation when delivered in combination with clinician implementation strategies. Across three VA healthcare system sites, we will enroll primary care clinicians with low rates of goals of care conversations and their patients with serious medical illness in the top 10th percentile of risk of hospitalization or death. We will compare the effectiveness of sequences of implementation strategies and explore how patient and site factors modify implementation strategy effects. Finally, we will conduct a mixed-methods evaluation to understand implementation strategy success or failure. The design includes two key innovations: (1) strategies that target both clinicians and patients and (2) sequential strategies with increased intensity for non-responders. CONCLUSION: This study aims to determine the effect of different sequences and combinations of implementation strategies on primary care documentation of goals of care conversations. Study partners, including the VA National Center for Ethics in Health Care and Office of Primary Care, can consider policies based on study findings.

Evidence is sparse on how community-based health-promoting programs can be culturally adapted for racially minoritized, immigrant older adult populations. Choose to Move (CTM) is an evidence-based health-promoting program that enhances physical activity and mobility and diminished social isolation and loneliness in older adults in British Columbia, Canada. However, racially minoritized older adults were not reached in initial offerings. We purposively sampled CTM delivery staff (n = 8) from three not-for-profit organizations, in Metro Vancouver, British Columbia, that serve Chinese older adults. We used semistructured interviews, ethnographic observations, and meeting minutes to understand delivery staff's perspectives on factors that influence CTM adaptations for Chinese older adults. Deductive framework analysis guided by an adaptation framework, Framework for Reporting Adaptations and Modifications-Enhanced, found three dominant cultural- and immigration-related factors influenced CTM adaptations for Chinese older adults: (a) prioritizations, (b) familiarity, and (c) literacy. Findings may influence future program development and delivery to meet the needs of racially minoritized older adult populations. [ABSTRACT FROM AUTHOR]

BACKGROUND: Venous thromboembolism (VTE) is a preventable medical condition which has substantial impact on patient morbidity, mortality, and disability. Unfortunately, adherence to the published best practices for VTE prevention, based on patient centered outcomes research (PCOR), is highly variable across U.S. hospitals, which represents a gap between current evidence and clinical practice leading to adverse patient outcomes. This gap is especially large in the case of traumatic brain injury (TBI), where reluctance to initiate VTE prevention due to concerns for potentially increasing the rates of intracranial bleeding drives poor rates of VTE prophylaxis. This is despite research which has shown early initiation of VTE prophylaxis to be safe in TBI without increased risk of delayed neurosurgical intervention or death. Clinical decision support (CDS) is an indispensable solution to close this practice gap; however, design and implementation barriers hinder CDS adoption and successful scaling across health systems. Clinical practice guidelines (CPGs) informed by PCOR evidence can be deployed using CDS systems to improve the evidence to practice gap. In the Scaling AcceptabLE cDs (SCALED) study, we will implement a VTE prevention CPG within an interoperable CDS system and evaluate both CPG effectiveness (improved clinical outcomes) and CDS implementation. METHODS: The SCALED trial is a hybrid type 2 randomized stepped wedge effectiveness-implementation trial to scale the CDS across 4 heterogeneous healthcare systems. Trial outcomes will be assessed using the RE2-AIM planning and evaluation framework. Efforts will be made to ensure implementation consistency. Nonetheless, it is expected that CDS adoption will vary across each site. To assess these differences, we will evaluate implementation processes across trial sites using the Exploration, Preparation, Implementation, and Sustainment (EPIS) implementation framework (a determinant framework) using mixed-methods. Finally, it is critical that PCOR CPGs are maintained as evidence evolves. To date, an accepted process for evidence maintenance does not exist. We will pilot a Living Guideline process model for the VTE prevention CDS system. DISCUSSION: The stepped wedge hybrid type 2 trial will provide evidence regarding the effectiveness of CDS based on the Berne-Norwood criteria for VTE prevention in patients with TBI. Additionally, it will provide evidence regarding a successful strategy to scale interoperable CDS systems across U.S. healthcare systems, advancing both the fields of implementation science and health informatics. TRIAL REGISTRATION: Clinicaltrials.gov - NCT05628207. Prospectively registered 11/28/2022, https://classic. CLINICALTRIALS: gov/ct2/show/NCT05628207 .

Beyond Bias was an intervention introduced in Burkina Faso, Pakistan and Tanzania, with the aim of reducing health worker bias toward young, unmarried and nulliparous women seeking family planning services. This study used qualitative methods - based on interviews with health workers who participated in the intervention, managers at health facilities that participated in the intervention, and policy and program stakeholders at the national level - to understand implementation experiences with the intervention. The results offer insights for organizations or countries seeking to implement Beyond Bias or similar programs, and point to some other key implementation challenges for multi-component interventions in lower-resource settings. The intervention, developed using a human-centered design approach, was seen as key for successful implementation but there were logistical challenges. The digital intervention was disruptive and distracting to many. In addition, the non-financial rewards intervention was perceived as complex, and some participants expressed feeling discouraged when they did not receive a reward. Beyond Bias did not sufficiently attend to the outer setting, and this was perceived as a major implementation barrier as it limited individuals capacity to fully achieve the desired behavior change; for example, space constraints meant that some health facilities could not ensure private services for all clients. There were scalability concerns related to cost, and there is uncertainty whether diversity of contexts (within and across countries) might constrain implementation of Beyond Bias at scale.

Understanding the acceptability of long-acting injectable antiretroviral therapy (LAI-ART) among people with HIV (PWH), especially priority populations, is essential for effective implementation. We conducted semi-structured interviews with patients in three Ryan White-funded HIV clinics in San Francisco, Chicago, and Atlanta. We employed maximal variation sampling across age, gender, race, ethnicity, and time living with HIV and oversampled for individuals with suboptimal clinical engagement. An 8-step hybrid deductive and inductive thematic analysis approach guided data analysis. Between August 2020 and July 2021, we conducted 72 interviews. Median age was 46 years; 28% were ciswomen, 7% transwomen, 44% Black/African-American and 35% Latinx, 43% endorsed a psychiatric diagnosis, 35% were experiencing homelessness/unstable housing, and 10% had recent substance use. Approximately 24% were sub-optimally engaged in care. We observed a spectrum of LAI-ART acceptability, ranging from enthusiasm to hesitancy to rejection. We also characterized four emergent orientations towards LAI-ART: innovator, pragmatist, deliberator, and skeptic. Overall, the majority of participants expressed favorable initial reactions towards LAI-ART. Most approached LAI-ART pragmatically, but acceptability was not static, often increasing over the course of the interview. Participants considered their HIV providers as essential for affirming personal relevance. HIV stigma, privacy concerns, and medical mistrust had varied impacts, sometimes facilitating and other times hindering personal relevance. These findings held across priority populations, specifically young adults, cis/trans women, racial/ethnic minorities, and individuals with suboptimal clinical engagement. Further research is needed to explore the transition from hypothetical acceptance to uptake and to confirm the actual benefits and drawbacks of this treatment.

BACKGROUND: Patient participation in treatment decision making is a pillar of recovery-oriented care and is associated with improvements in empowerment and well-being. Although demand for increased involvement in treatment decision-making is high among veterans with serious mental illness, rates of involvement are low. Collaborative decision skills training (CDST) is a recovery-oriented, skills-based intervention designed to support meaningful patient participation in treatment decision making. An open trial among veterans with psychosis supported CDSTs feasibility and demonstrated preliminary indications of effectiveness. A randomized control trial (RCT) is needed to test CDSTs effectiveness in comparison with an active control and further evaluate implementation feasibility. METHODS: The planned RCT is a hybrid type 1 trial, which will use mixed methods to systematically evaluate the effectiveness and implementation feasibility of CDST among veterans participating in a VA Psychosocial Rehabilitation and Recovery Center (PRRC) in Southern California. The first aim is to assess the effectiveness of CDST in comparison with the active control via the primary outcome, collaborative decision-making behavior during usual care appointments between veterans and their VA mental health clinicians, and secondary outcomes (i.e., treatment engagement, satisfaction, and outcome). The second aim is to characterize the implementation feasibility of CDST within the VA PRRC using the Practical Robust Implementation and Sustainability Model framework, including barriers and facilitators within the PRRC context to support future implementation. DISCUSSION: If CDST is found to be effective and feasible, implementation determinants gathered throughout the study can be used to ensure sustained and successful implementation at this PRRC and other PRRCs and similar settings nationally. TRIAL REGISTRATION: ClinicalTrials.gov NCT04324944. Registered on March 27, 2020. Trial registration data can be found in Appendix 1.

BACKGROUND: Across sub-Saharan Africa, mid-level healthcare managers oversee implementation of national guidelines. It remains unclear whether leadership and management training can improve population health outcomes. METHODS: We sought to evaluate leadership/management skills among district-level health managers in Uganda participating in the SEARCH-IPT randomised trial to promote isoniazid preventive therapy (IPT) for persons with HIV (PWH). The intervention, which led to higher IPT rates, included annual leadership/management training of managers. We conducted a cross-sectional survey assessing leadership/management skills among managers at trial completion. The survey evaluated self-reported use of leadership/management tools and general leadership/management. We conducted a survey among a sample of providers to understand the interventions impact. Targeted minimum loss-based estimation (TMLE) was used to compare responses between trial arms. RESULTS: Of 163 managers participating in the SEARCH-IPT trial, 119 (73%) completed the survey. Intervention managers reported more frequent use of leadership/management tools taught in the intervention curriculum than control managers (+3.64, 95% CI 1.98-5.30, P < 0.001). There were no significant differences in self-reported leadership skills in the intervention as compared to the control group. Among providers, the average reported quality of guidance and supervision was significantly higher in intervention vs control districts (+1.08, 95% CI 0.63-1.53, P = 0.001). CONCLUSIONS: A leadership and management training intervention increased the use of leadership/management tools among mid-level managers and resulted in higher perceived quality of supervision among providers in intervention vs control districts in Uganda. These findings suggest improved leadership/management among managers contributed to increased IPT use among PWH in the intervention districts of the SEARCH-IPT trial.

To support older adults during the first wave of COVID-19, we rapidly adapted our effective health-promoting intervention (Choose to Move [CTM]) for virtual delivery in British Columbia, Canada. The intervention was delivered (April–October 2020) to 33 groups of older adults ("programs") who were a convenience sample (had previously completed CTM in person; n = 153; 86% female; 73 [6] years). We compared implementation outcomes (recruitment, dose received, retention, and completion of virtual data collection) to predetermined feasibility targets. We assessed mobility, physical activity, and social health outcomes pre- and postintervention (3 months) with validated surveys. We met most (dose received, retention, and virtual data collection), but not all (recruitment), feasibility targets. Approximately two thirds of older adults maintained or improved mobility, physical activity, and social health outcomes at 3 months. It was feasible to implement and evaluate CTM virtually. In future, virtual CTM could help us reach homebound older adults and/or serve as support during public health emergencies. [ABSTRACT FROM AUTHOR]

Background: Trans women in Vietnam are among the most vulnerable groups with high HIV risk and limited access to care. TransAction is an evidence-based intervention to reduce trans women's HIV risks and increase social support and access to care.Aims: The aim of this study was to adapt TransAction to the specific needs of trans women in Vietnam. Methods: This study was conducted in Ho Chi Minh City from November 2020 through June 2021 Using the ADAPT-ITT framework, interviews, focus groups, and community advisory board meetings were conducted with trans women, service providers, and community members to better understand Vietnamese contexts of gender transition, HIV risks, and service gaps. Feedback was solicited on TransAction content and format adaptation. Results: Trans women in Vietnam faced unique challenges related to family norms, policy and regulatory constraints, and limited transgender-specific or gender-inclusive services. TransAction was modified to accommodate identified challenges and needs, and intervention components to enhance family support were added. Strategies to cope with stigma and seek support and services were adapted to Vietnamese culture and policies. Discussion: Post-adaptation interviews and focus groups demonstrated strong feasibility and acceptability for the adapted intervention, which can potentially be used to reduce Vietnamese trans women's HIV risks and increase their social support.

OBJECTIVE: We applied a user experience (UX) design approach to clinical decision support (CDS) tool development for the specific use case of pediatric asthma. Our objective was to understand physicians workflows, decision-making processes, barriers (ie, pain points), and facilitators to increase usability of the tool. MATERIALS AND METHODS: We used a mixed-methods approach with semi-structured interviews and surveys. The coded interviews were synthesized into physician-user journey maps (ie, visualization of a process to accomplish goals) and personas (ie, user types). Interviews were conducted via video. We developed physician journey maps and user personas informed by their goals, systems interactions, and experiences with pediatric asthma management. RESULTS: The physician end-user personas identified were: efficiency, relationship, and learning. Features of a potential asthma CDS tool sought varied by physician practice type and persona. It was important to the physician end-user that the asthma CDS tool demonstrate value by lowering workflow friction (ie, difficulty or obstacles), improving the environment surrounding physicians and patients, and using it as a teaching tool. Customizability versus standardization were important considerations for uptake. DISCUSSION: Different values and motivations of physicians influence their use and interaction with the EHR and CDS tools. These different perspectives can be captured by applying a UX design approach to the development process. For example, with the importance of customizability, one approach may be to build a core module with variations depending on end-user preference. CONCLUSION: A UX approach can drive design to help understand physician-users and meet their needs; ultimately with the goal of increased uptake.

Interventional researchers face many design challenges when assessing intervention implementation in real-world settings. Intervention implementation requires holding fast on internal validity needs while incorporating external validity considerations (such as uptake by diverse subpopulations, acceptability, cost, and sustainability). Quasi-experimental designs (QEDs) are increasingly employed to achieve a balance between internal and external validity. Although these designs are often referred to and summarized in terms of logistical benefits, there is still uncertainty about (a) selecting from among various QEDs and (b) developing strategies to strengthen the internal and external validity of QEDs. We focus here on commonly used QEDs (prepost designs with nonequivalent control groups, interrupted time series, and stepped-wedge designs) and discuss several variants that maximize internal and external validity at the design, execution and implementation, and analysis stages.

BACKGROUND: Despite the increased risk of cervical cancer (CC) among women living with HIV (WLHIV), CC screening and treatment (CCST) rates remain low in Africa. The integration of CCST services into established HIV programs in Africa can improve CC prevention and control. However, the paucity of evidence on effective implementation strategies (IS) has limited the success of integration in many countries. In this study, we seek to identify effective IS to enhance the integration of CCST services into existing HIV programs in Nigeria. METHODS: Our proposed study has formative and experimental activities across the four phases of the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework. Through an implementation mapping conducted with stakeholders in the exploration phase, we identified a core package of IS (Core) and an enhanced package of IS (Core+) mostly selected from the Expert Recommendations for Implementing Change. In the preparation phase, we refined and tailored the Core and Core+ IS with the implementation resource teams for local appropriateness. In the implementation phase, we will conduct a cluster-randomized hybrid type III trial to assess the comparative effectiveness of Core versus Core+. HIV comprehensive treatment sites (k = 12) will be matched by region and randomized to Core or Core+ in the ratio of 1:1 stratified by region. In the sustainment phase, we will assess the sustainment of CCST at each site. The study outcomes will be assessed using RE-AIM: reach (screening rate), adoption (uptake of IS by study sites), IS fidelity (degree to which the IS occurred according to protocol), clinical intervention fidelity (delivery of CC screening, onsite treatment, and referral according to protocol), clinical effectiveness (posttreatment screen negative), and sustainment (continued integrated CCST service delivery). Additionally, we will descriptively explore potential mechanisms, including organizational readiness, implementation climate, CCST self-efficacy, and implementation intentions. DISCUSSION: The assessment of IS to increase CCST rates is consistent with the global plan of eliminating CC as a public health threat by 2030. Our study will identify a set of evidence-based IS for low-income settings to integrate evidence-based CCST interventions into routine HIV care in order to improve the health and life expectancy of WLHIV. TRIAL REGISTRATION: Prospectively registered on November 7, 2023, at ClinicalTrials.gov no. NCT06128304. https://classic. CLINICALTRIALS: gov/ct2/show/study/NCT06128304.

INTRODUCTION: The Allocation System for changes in Equity in Kidney Transplantation (ASCENT) study was a hybrid type 1 trial of a multicomponent intervention among 655 US dialysis facilities with low kidney transplant waitlisting to educate staff and patients about kidney allocation system (KAS) changes and increase access to and reduce racial disparities in waitlisting. Intervention components included a staff webinar, patient and staff educational videos, and facility-specific feedback reports. METHODS: Implementation outcomes were assessed using the Reach, Effectiveness, Adoption, Implementation, and Maintenance Framework. Postimplementation surveys were administered among intervention group facilities (n = 334); interviews were conducted with facility staff (n = 6). High implementation was defined as using 3 to 4 intervention components, low implementation as using 1 to 2 components, and nonimplementation as using no components. RESULTS: A total of 331 (99%) facilities completed the survey; 57% were high implementers, 31% were low implementers, and 12% were nonimplementers. Waitlisting events were higher or similar among high versus low implementer facilities for incident and prevalent populations; for Black incident patients, the mean proportion waitlisted in low implementer facilities was 0.80% (95% confidence interval [CI]: 0.73-0.87) at baseline and 0.55% at 1-year (95% CI: 0.48-0.62) versus 0.83% (95% CI: 0.78-0.88) at baseline and 1.40% at 1-year (95% CI: 1.35-1.45) in high implementer facilities. Interviews revealed that the intervention helped facilities prioritize transplant education, but that intervention components were not uniformly shared. CONCLUSION: The findings provide important context to interpret ASCENT effectiveness results and identified key barriers and facilitators to consider for future modification and scale-up of multilevel, multicomponent interventions in dialysis settings.

BACKGROUND: Palliative care has the potential to address significant unmet needs in people with Parkinsons disease and related disorders, but models that rely on in-person specialty palliative care teams have limited scalability. AIM: To describe patient and care partner experiences with a novel, community-based palliative care intervention for Parkinsons disease. DESIGN: Qualitative study embedded in a randomized clinical trial to document participant experiences with a novel palliative care intervention (community neurologist training and remote team-based specialist palliative care). Transcripts were coded and thematically analyzed through a combination of team-based inductive and deductive coding. SETTING/PARTICIPANTS: Twenty-eight patients and 33 care partners purposively sampled from participants in a randomized clinical trial of a palliative care intervention for Parkinsons disease and related disorders conducted at nine sites. RESULTS: Benefits of the intervention included management of a wider range of non-motor symptoms, facilitation of conversations about the future, greater engagement with the health care team, and increased referrals to resources. Participants identified areas of improvement, including uptake of palliative care training by community neurologists, additional prognostic counseling, and clarity and timeliness of communication with the multidisciplinary team. CONCLUSIONS: Clinicians caring for people with Parkinsons disease and related disorders should screen for non-motor symptoms, provide regular prognostic counseling, and refer to specialty palliative care services earlier in the course of illness. Future interventions should be designed to promote uptake of palliative care training by community neurologists and further optimize referral to and coordination with in-person or remote specialty palliative teams.

Many patients who are at high risk of HIV transmission do not receive pre-exposure prophylaxis (PrEP). HIV risk counselling and PrEP initiation have historically been limited to outpatient settings. Here we describe a novel quality improvement project at San Franciscos main safety-net hospital designed to incorporate universal screening for active HIV risk factors and PrEP initiation into standard inpatient care. Interventions included education sessions and dissemination of clinical materials to increase providers knowledge and comfort with HIV risk screening, prevention counselling and prescribing PrEP. We implemented new workflows on the inpatient medicine service to encourage providers to universally screen all patients at the time of admission and initiate PrEP for appropriate patients during their hospitalisation. Over the first 9 months of the initiative, 14 inpatients were started on PrEP during their admission. As PrEP was initiated in particularly vulnerable patients, using inpatient admissions to engage at-risk patients in HIV prevention may help to reduce disparities in HIV outcomes.

BackgroundThe use of rapid qualitative methods has increased substantially over the past decade in quality improvement and health services research. These methods have gained traction in implementation research and practice, wherein real-time adjustments are often made to optimize processes and outcomes. This brisk increase begs the questions: what does rigor entail in projects that use rapid qualitative analysis (RQA)? How do we define a pragmatic framework to help research teams design and conduct rigorous and valid rapid qualitative projects? How can authors articulate rigor in their methods descriptions? Lastly, how can reviewers evaluate the rigor of rapid qualitative projects?.MethodsA team of seven interdisciplinary qualitative methods experts developed a framework for ensuring rigor and validity in RQA and methods suitable for this analytic approach. We conducted a qualitative evidence synthesis to identify gaps in the literature and then drew upon literature, standard procedures within our teams, and a repository of rapid qualitative training materials to create a planning and reporting framework. We iteratively refined this framework through 11 group working meetings (60-90 minutes each) over the course of one year and invited feedback on items to ensure their completeness, clarity, and comprehensibility.ResultsThe Planning for and Assessing Rigor in Rapid Qualitative Analysis (PARRQA) framework is organized progressively across phases from design to dissemination, as follows: 1) rigorous design (rationale and staffing), 2) semi-structured data collection (pilot and planning), 3) RQA: summary template development (accuracy and calibration), 4) RQA: matrix analysis (matrices), and 5) rapid qualitative data synthesis. Eighteen recommendations across these sections specify best practices for rigor and validity.ConclusionsRapid qualitative methods play a central role in implementation evaluations, with the potential to yield prompt information and insights about context, processes, and relationships. However, guidance on how to assess rigor is nascent. The PARRQA framework enhances the literature by offering criteria to ensure appropriate planning for and assessment of rigor in projects that involve RQA. This framework provides a consensus-based resource to support high-level qualitative methodological rigor in implementation science.

BackgroundImplementation science scholars have made significant progress identifying factors that enable or obstruct the implementation of evidence-based interventions, and testing strategies that may modify those factors. However, little research sheds light on how or why strategies work, in what contexts, and for whom. Studying implementation mechanisms-the processes responsible for change-is crucial for advancing the field of implementation science and enhancing its value in facilitating equitable policy and practice change. The Agency for Healthcare Research and Quality funded a conference series to achieve two aims: (1) develop a research agenda on implementation mechanisms, and (2) actively disseminate the research agenda to research, policy, and practice audiences. This article presents the resulting research agenda, including priorities and actions to encourage its execution.MethodBuilding on prior concept mapping work, in a semi-structured, 3-day, in-person working meeting, 23 US-based researchers used a modified nominal group process to generate priorities and actions for addressing challenges to studying implementation mechanisms. During each of the three 120-min sessions, small groups responded to the prompt: "What actions need to be taken to move this research forward?" The groups brainstormed actions, which were then shared with the full group and discussed with the support of facilitators trained in structured group processes. Facilitators grouped critical and novel ideas into themes. Attendees voted on six themes they prioritized to discuss in a fourth, 120-min session, during which small groups operationalized prioritized actions. Subsequently, all ideas were collated, combined, and revised for clarity by a subset of the authorship team.ResultsFrom this multistep process, 150 actions emerged across 10 priority areas, which together constitute the research agenda. Actions included discrete activities, projects, or products, and ways to shift how research is conducted to strengthen the study of implementation mechanisms.ConclusionsThis research agenda elevates actions to guide the selection, design, and evaluation of implementation mechanisms. By delineating recommended actions to address the challenges of studying implementation mechanisms, this research agenda facilitates expanding the field of implementation science, beyond studying what works to how and why strategies work, in what contexts, for whom, and with which interventions.

BackgroundSmartphone apps may provide an opportunity to deliver mental health resources and interventions in a scalable and cost-effective manner. However, young people from marginalized and underserved groups face numerous and unique challenges to accessing, engaging with, and benefiting from these apps.ObjectiveThis study aims to better understand the acceptability (ie, perceived usefulness and satisfaction with an app) and feasibility (ie, the extent to which an app was successfully used) of mental health apps for underserved young people. A secondary aim was to establish whether adaptations can be made to increase the accessibility and inclusivity of apps for these groups.MethodsWe conducted 2 sequential studies, consisting of a systematic literature review of mental health apps for underserved populations followed by a qualitative study with underserved young male participants (n=20; age: mean 19). Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, an electronic search of 5 databases was conducted in 2021. The search yielded 18,687 results, of which 14 articles met the eligibility criteria.ResultsThe included studies comprised a range of groups, including those affected by homelessness, having physical health conditions, living in low- and middle-income countries, and those with sexual and gender minority identities. Establishing and maintaining user engagement was a pervasive challenge across mental health apps and populations, and dropout was a reported problem among nearly all the included studies. Positive subjective reports of usability, satisfaction, and acceptability were insufficient to determine users' objective engagement.ConclusionsDespite the significant amount of funding directed to the development of mental health apps, juxtaposed with only limited empirical evidence to support their effectiveness, few apps have been deliberately developed or adapted to meet the heterogeneous needs of marginalized and underserved young people. Before mental health apps are scaled up, a greater understanding is needed of the types of services that more at-risk young people and those in limited-resource settings prefer (eg, standard vs digital) followed by more rigorous and consistent demonstrations of acceptability, effectiveness, and cost-effectiveness. Adopting an iterative participatory approach by involving young people in the development and evaluation process is an essential step in enhancing the adoption of any intervention, including apps, in "real-world" settings and will support future implementation and sustainability efforts to ensure that marginalized and underserved groups are reached.Trial registrationPROSPERO CRD42021254241; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=254241.

BackgroundThe Inpatient Telehealth Program permits family to remotely communicate with the patient and care team through secure, live video. We aimed to assess the implementation of this program for family engagement from the perspectives of patients, families, and providers.MethodsWe used a convergent mixed methods design. The quantitative component was a cross-sectional analysis of surveys assessing patient, family, and provider experience. The qualitative component used thematic analysis of patient, family, and provider interviews plus survey free text responses. We performed memo-writing and coding. We developed hypotheses about relationships among categories and identified analytic themes. We used data transformation and narrative discussion to report the integrated findings.ResultsSurveys from 214 individuals (33 patients, 145 family, 36 providers) were evaluated. Mean (standard deviation) experience ratings (1-poor, 5-best) were 4.0 (1.5) for patients, 4.6 (0.8) for family, and 4.0 (1.4) for providers. We received 134 free text responses and conducted 21 interviews. Three themes emerged: (1) inpatient telehealth enhanced patient and family experience through strengthened relationships and increased support; (2) inpatient telehealth enhanced patient care through improved information sharing and engagement; (3) low awareness of the program limited adoption. Quantitative and qualitative data aligned in that participants perceived inpatient telehealth to be valuable; however, surveys revealed that patients and providers have relatively lower satisfaction with the program.ConclusionInpatient telehealth for family engagement was perceived to improve family-centeredness of care. Future work is needed to overcome implementation challenges and to increase awareness of this resource among patients and families.

Typical quantitative evaluations of public policies treat policies as a binary condition, without further attention to how policies are implemented. However, policy implementation plays an important role in how the policy impacts behavioral and health outcomes. The field of policy-focused implementation science is beginning to consider how policy implementation may be conceptualized in quantitative analyses (e.g., as a mediator or moderator), but less work has considered how to measure policy implementation for inclusion in quantitative work. To help address this gap, we discuss four design considerations for researchers interested in developing or identifying measures of policy implementation using three independent NIH-funded research projects studying e-cigarette, food, and mental health policies. Mini case studies of these considerations were developed via group discussions; we used the implementation research logic model to structure our discussions. Design considerations include (1) clearly specifying the implementation logic of the policy under study, (2) developing an interdisciplinary team consisting of policy practitioners and researchers with expertise in quantitative methods, public policy and law, implementation science, and subject matter knowledge, (3) using mixed methods to identify, measure, and analyze relevant policy implementation determinants and processes, and (4) building flexibility into project timelines to manage delays and challenges due to the real-world nature of policy. By applying these considerations in their own work, researchers can better identify or develop measures of policy implementation that fit their needs. The experiences of the three projects highlighted in this paper reinforce the need for high-quality and transferrable measures of policy implementation, an area where collaboration between implementation scientists and policy experts could be particularly fruitful. These measurement practices provide a foundation for the field to build on as attention to incorporating measures of policy implementation into quantitative evaluations grows and will help ensure that researchers are developing a more complete understanding of how policies impact health outcomes.

BackgroundImplementation science frameworks situate intervention implementation and sustainment within the context of the implementing organization and system. Aspects of organizational context such as leadership have been defined and measured largely within US health care settings characterized by decentralization and individual autonomy. The relevance of these constructs in other settings may be limited by differences like collectivist orientation, resource constraints, and hierarchical power structures. We aimed to adapt measures of organizational context in South African primary care clinics.MethodsWe convened a panel of South African experts in social science and HIV care delivery and presented implementation domains informed by existing frameworks and prior work in South Africa. Based on panel input, we selected contextual domains and adapted candidate items. We conducted cognitive interviews with 25 providers in KwaZulu-Natal Province to refine measures. We then conducted a cross-sectional survey of 16 clinics with 5-20 providers per clinic (N = 186). We assessed reliability using Cronbach's alpha and calculated interrater agreement (awg) and intraclass correlation coefficient (ICC) at the clinic level. Within clinics with moderate agreement, we calculated correlation of clinic-level measures with each other and with hypothesized predictors - staff continuity and infrastructure - and a clinical outcome, patient retention on antiretroviral therapy.ResultsPanelists emphasized contextual factors; we therefore focused on elements of clinic leadership, stress, cohesion, and collective problem solving (critical consciousness). Cognitive interviews confirmed salience of the domains and improved item clarity. After excluding items related to leaders' coordination abilities due to missingness and low agreement, all other scales demonstrated individual-level reliability and at least moderate interrater agreement in most facilities. ICC was low for most leadership measures and moderate for others. Measures tended to correlate within facility, and higher stress was significantly correlated with lower staff continuity. Organizational context was generally more positively rated in facilities that showed consistent agreement.ConclusionsAs theorized, organizational context is important in understanding program implementation within the South African health system. Most adapted measures show good reliability at individual and clinic levels. Additional revision of existing frameworks to suit this context and further testing in high and low performing clinics is warranted.

BACKGROUND: Multimodal lifestyle interventions, employing food as medicine, stand as the recommended first-line treatment for obesity. The Shared Medical Appointment (SMA) model, where a physician conducts educational sessions with a group of patients sharing a common diagnosis, offers an avenue for delivery of comprehensive obesity care within clinical settings. SMAs, however, are not without implementation challenges. We aim to detail our experience with three implementation models in launching a virtual integrative health SMA for weight management. METHODS: Eligible patients included individuals 18 years of age or older, having a body mass index (BMI) of 30 kg/m2 or 27 kg/m2 or greater with at least one weight related comorbidity. The Practical, Robust Implementation and Sustainability Model (PRISM), Plan, Do, Study, Act (PDSA), and the Framework for Reporting Adaptations and Modifications-Enhanced (FRAME) models were applied to guide the implementation of the Supervised Lifestyle Integrative Medicine (SLIM) program, a virtually delivered, lifestyle medicine focused SMA program, in a weight management clinic within a major health system. We describe how these models, along with attendance for the initial cohorts, were used for decision-making in the process of optimizing the program. RESULTS: 172 patients completed the SLIM program over two years. Attendance was lowest for sessions held at 8:00 AM and 4:00 PM compared to sessions at 10:00 AM, 1:00 PM, and 3:00 PM, leading to only offering midday sessions (p = 0.032). Attendance data along with feedback from patients, facilitators, and administrative partners led to changes in the curriculum, session number and frequency, session reminder format, and intake visit number. CONCLUSION: The use of implementation and quality improvement models provided crucial insight for deployment and optimization of a virtual, lifestyle medicine focused SMA program for weight management within a large healthcare system.

BACKGROUND: This studys aims are to: (1) Compare the acceptability and feasibility of five types of implementation strategies that could be deployed to increase the reach of evidence-based practices (EBPs) with revenue from policies that earmark taxes for behavioral health services, and (2) Illustrate how definitions of implementation strategies and measures of acceptability and feasibility can be used in policy-focused implementation science research. METHODS: Web-based surveys of public agency and community organization professionals involved with earmarked tax policy implementation were completed in 2022-2023 (N = 211, response rate = 24.9%). Respondents rated the acceptability and feasibility of five types of implementation strategies (dissemination, implementation process, integration, capacity-building, and scale-up). Aggregate acceptability and feasibility scores were calculated for each type of strategy (scoring range 4-20). Analyses of variance compared scores across strategies and between organizational actor types. FINDINGS: For acceptability, capacity-building strategies had the highest rating (M = 16.3, SD = 3.0), significantly higher than each of the four other strategies, p ≤ . 004), and scale-up strategies had the lowest rating (M = 15.6). For feasibility, dissemination strategies had the highest rating (M = 15.3, significantly higher than three of the other strategies, p ≤ .002) and scale-up strategies had the lowest rating (M = 14.4). CONCLUSIONS: Capacity-building and dissemination strategies may be well-received and readily deployed by policy implementers to support EBPs implementation with revenue from taxes earmarked for behavioral health services. Adapting definitions of implementation strategies for policy-focused topics, and applying established measures of acceptability and feasibility to these strategies, demonstrates utility as an approach to advance research on policy-focused implementation strategies.

Cardiovascular disease (CVD) is largely preventable, and the leading cause of death for men and women. Though women have increased life expectancy compared to men, there are marked sex disparities in prevalence and risk of CVD-associated mortality and dementia. Yet, the basis for these and female-male differences is not completely understood. It is increasingly recognized that heart and brain health represent a lifetime of exposures to shared risk factors (including obesity, hyperlipidemia, diabetes, and hypertension) that compromise cerebrovascular health. We describe the process and resources for establishing a new research Center for Women's Cardiovascular and Brain Health at the University of California, Davis as a model for: (1) use of the cy pres principle for funding science to improve health; (2) transdisciplinary collaboration to leapfrog progress in a convergence science approach that acknowledges and addresses social determinants of health; and (3) training the next generation of diverse researchers. This may serve as a blueprint for future Centers in academic health institutions, as the cy pres mechanism for funding research is a unique mechanism to leverage residual legal settlement funds to catalyze the pace of scientific discovery, maximize innovation, and promote health equity in addressing society's most vexing health problems.

Precision prevention embraces personalized prevention but includes broader factors such as social determinants of health to improve cardiovascular health. The quality, quantity, precision, and diversity of data relatable to individuals and communities continue to expand. New analytical methods can be applied to these data to create tools to attribute risk, which may allow a better understanding of cardiovascular health disparities. Interventions using these analytic tools should be evaluated to establish feasibility and efficacy for addressing cardiovascular disease disparities in diverse individuals and communities. Training in these approaches is important to create the next generation of scientists and practitioners in precision prevention. This state-of-the-art review is based on a workshop convened to identify current gaps in knowledge and methods used in precision prevention intervention research, discuss opportunities to expand trials of implementation science to close the health equity gaps, and expand the education and training of a diverse precision prevention workforce.

IntroductionMeaningful engagement of partners in co-creating and refining health-related programs can increase the initial uptake, sustained implementation, broad reach, and effectiveness of these programs. This is especially important for underserved communities where resources are limited and need to be prioritized. Brainwriting premortem is a novel qualitative approach to partner engagement that combines the strengths of individual idea generation with the concept of premortem exercise that addresses failure points prior to the implementation of new programs.MethodsAn adapted form of brainwriting premortem was used to inform iterative refinements to a COVID-19 testing program at a Federally Qualified Health Center (FQHC) in San Diego. Patients and providers from the FQHC participated in interviews at two time points (early- and mid-implementation of the program). Interview data were transcribed, translated, and analyzed using a rapid qualitative approach. Key themes and sub-themes were identified and used to inform refinements to the program.ResultsA total of 11 patients (7 Spanish- and 4 English-speaking) and 8 providers participated in the brainwriting premortem interviews. Key themes related to possible reasons for COVID-19 testing program failure: advertising/sharing information; access to testing; handling of test results; staff and patient safety; patient beliefs and views regarding the SARS-CoV-2 virus; and COVID-19 testing options offered. Proposed solutions were offered for the key failures except for patient beliefs and views regarding the SARS-CoV-2 virus. Additional solutions offered were related to education, physical operations, and recruitment strategies. Real-time changes to the program flow and components were made in response to 7 suggestions from patients and 11 from providers. Changes related to the process of returning results were the most common, and included sending results via email with distinct workflows based on the test result.ConclusionThe implementation of the adapted brainwriting premortem technique allowed us to incorporate the perspective of key partners in the delivery and iterative refinement of the COVID-19 testing program. This was an effective tool in the context of an FQHC and can be a promising and approach to incorporate iterative input from patients and providers to ensure successful program implementation. Future studies, particularly those requiring rapid response to public health emergencies, should consider the use of this technique.

ABSTRACT Aims Design Methods Results Conclusion Implications for the Profession Impact Reporting Method Patient/Public Contribution To co‐identify adaptations with key stakeholders needed to optimise elements of a video‐based intervention (i.e., PREEMIE PROGRESS [PP]), which trains parents in evidence‐based family management skills to care for their very preterm infant in the neonatal intensive care unit (NICU).Descriptive qualitative study oriented with a pragmatic philosophy, informed by the reach, effectiveness, adoption, implementation, maintenance (RE‐AIM) framework and the framework for reporting adaptations and modifications‐expanded (FRAME).Semistructured interviews to identify potential adaptations with key stakeholders: family management researchers (n = 5), clinicians (n = 9), technology experts (n = 5) and parents of preterm infants (n = 17). Weekly design team meetings to select and implement high‐priority adaptations necessary for the next research phase. Monthly NICU parent partnership meetings to review adaptations and make recommendations for potential adaptations with conflicting data.Stakeholders (N = 36) suggested 98 potential adaptations: 32 (33.0%) were completed, 8 (8.2%) were abandoned, 5 (5.2%) have work that is ongoing and 52 (53.6%) were tabled for future research phases. Content adaptations (70, 71.4%) were the most frequently suggested adaptation type. Potential adaptations mostly addressed RE‐AIM dimensions of effectiveness (43, 43.9%), and implementation (46, 46.9%) and were directed at the parent (i.e., intervention recipient) level (79, 81.4%).Use of the RE‐AIM framework ensured we systematically identified needed adaptations with key stakeholders across a range of dimensions that would improve PP for parents now and in future phases of this research.Co‐identifying potential adaptations with key stakeholders, paired with FRAME documentation, can help nurses prioritise adaptations most appropriate for each phase of implementation.Our paper highlights for nurse clinicians and researchers how FRAME documentation of potential adaptations can support stakeholder engagement and a systematic approach to incorporating adaptations throughout all phases of the research process, thereby shortening the evidence to practice gap.COREQ guidelines for qualitative reporting.The research team was supported by members of the NICU's Parent Partnership Council (PPC), whose mission is to promote family‐centred care improvement projects and research within the NICU. This committee is comprised of nursing, physician, allied health leadership and parents of infants previously hospitalised in the NICU. The NICU PPC met monthly to review conflicting data on potential adaptations and provide recommendations on adaptation decisions. [ABSTRACT FROM AUTHOR]

Food is Medicine (FIM) programs to improve the accessibility of fruits and vegetables (FVs) or other healthy foods among patients with low income and diet-related chronic diseases are promising to improve food and nutrition security in the United States (US). However, FIM programs are relatively new and implementation guidance for healthcare settings using an implementation science lens is lacking. We used a narrative review to describe the evidence base on barriers and facilitators to FIM program integration in US healthcare settings following the Exploration, Preparation, Implementation, and Sustainment (EPIS) Framework. Evidence surrounding the EPIS Inner Context was a focus, including constructs Leadership, Organizational Characteristics, Quality and Fidelity Monitoring and Support, Organizational Staffing Processes, and Individual Characteristics. Peer-reviewed and grey literature about barriers and facilitators to FIM programs were of interest, defined as programs that screen and refer eligible patients with diet-related chronic disease experiencing food insecurity to healthy, unprepared foods. Thirty-one sources were included in the narrative review, including 22 peer-reviewed articles, four reports, four toolkits, and one thesis. Twenty-eight sources (90%) described EPIS Inner Context facilitators and 26 sources (84%) described FIM program barriers. The most common barriers and facilitators to FIM programs were regarding Quality and Fidelity Monitoring and Support (e.g., use of electronic medical records for tracking and evaluation, strategies to support implementation) and Organizational Staffing Processes (e.g., clear delineation of staff roles and capacity); although, barriers and facilitators to FIM programs were identified among all EPIS Inner Context constructs. We synthesized barriers and facilitators to create an EPIS-informed implementation checklist for healthcare settings for use among healthcare organizations/providers, partner organizations, and technical assistance personnel. We discuss future directions to align FIM efforts with implementation science terminology and theories, models, and frameworks to improve the implementation evidence base and support FIM researchers and practitioners. [ABSTRACT FROM AUTHOR]

Background: Digital solutions in healthcare can facilitate and improve care. However, the experiences and the usefulness of using either digital foot examinations or traditional foot examinations need to be evaluated. The aims of the study were to evaluate: 1) The differences in patient experiences, having their foot examined supported by the Clinical Decision Support System as compared with having their foot examined in traditional practice, 2) How healthcare professionals, by using the digital tool, experienced the routine compared with performing the foot examination as in traditional practice. Methods: Of a total of 141 patients, 100 patients with diabetes were single-blind digitally randomised to one of two parallel arms: having their foot examined by a healthcare professional using a digital tool (n = 47) or having their foot examined as in traditional practice (n = 53) at the Department of Prosthetics and Orthotics at Sahlgrenska University Hospital, Gothenburg, Sweden. Patients filled in a modified version of the National Patient Survey and the Orthotics and Prosthetics Users' Survey at study end. Two healthcare professionals, working at a Department of Prosthetics and Orthotics, answered surveys regarding the interaction between the patient and the certified prosthetist and orthotist. Results: Patients, aged 66 ± 13 years, perceived a high level of satisfaction with the service at the department, regardless of the method used. No significant differences between groups were found when evaluated by 27 questions in the National Patient Survey or by the Orthotics and Prosthetics Users' Survey, with scores of 67.17 ± 12.18 vs. 66.35 ± 16.52 (p = 0.78) for the intervention and control group respectively. For the same patient that healthcare professionals foot examined, the risk class was fully obtained when the risk to develop foot ulcers was assessed by using the digital tool, whereas only 2% of the patients were classified when foot assessed in traditional practice. Conclusions: Regardless of the method used for the foot examination, patients perceived a high level of satisfaction with the services at the Department of Prosthetics and Orthotics. All the patients were risk classified in the intervention group. The healthcare professionals found that, by using the Clinical Decision Support System, the foot examination was structured and followed clinical guidelines. Furthermore, the documentation in the electronic health record was thorough, even though further improvements, such as integration with co-existing health record systems, were requested. Trial registration: Clinical Trials NCT03088566, Registered 23 March 2017. [ABSTRACT FROM AUTHOR]

Background: Electronic patient-reported outcome (ePRO)-based symptom management improves cancer patients' outcomes. However, implementation of ePROs is challenging, requiring technical resources for integration into clinical systems, substantial buy-in from clinicians and patients, novel workflows to support between-visit symptom management, and institutional investment. Methods: The SIMPRO Research Consortium developed eSyM, an electronic health record-integrated, ePRO-based symptom management program for medical oncology and surgery patients and deployed it at six cancer centers between August 2019 and April 2022 in a type II hybrid effectiveness-implementation cluster randomized stepped-wedge study. Sites documented implementation strategies monthly using REDCap, itemized them using the Expert Recommendations for Implementation Change (ERIC) list and mapped their target barriers using the Consolidated Framework for Implementation Research (CFIR) to inform eSyM program enhancement, facilitate inter-consortium knowledge sharing and guide future deployment efforts. Results: We documented 226 implementation strategies: 35 'foundational' strategies were applied consortium-wide by the coordinating center and 191 other strategies were developed by individual sites. We consolidated these 191 site-developed strategies into 64 unique strategies (i.e., removed duplicates) and classified the remainder as either 'universal', consistently used by multiple sites (N = 29), or 'adaptive', used only by individual sites (N = 35). Universal strategies were perceived as having the highest impact; they addressed eSyM clinical preparation, training, engagement of patients/clinicians, and program evaluation. Across all documented SIMPRO strategies, 44 of the 73 ERIC strategies were addressed and all 5 CFIR barriers were addressed. Conclusion: Methodical collection of theory-based implementation strategies fostered the identification of universal, high-impact strategies that facilitated adoption of a novel care-delivery intervention by patients, clinicians, and institutions. Attention to the high-impact strategies identified in this project could support implementation of ePROs as a component of routine cancer care at other institutions. Trial registration: ClinicalTrials.gov. NCT03850912. February 22, 2019. https://clinicaltrials.gov/ct2/show/NCT03850912?term=hassett&draw=2&rank=1 [ABSTRACT FROM AUTHOR]

Integrated behavioral healthcare interventions have increased access to care for people with behavioral health conditions in primary care settings. However, they have not been widely implemented in low-barrier HIV care settings where undertreated behavioral health needs remain high. We conducted a formative qualitative evaluation, using in-depth interviews with purposively selected stakeholders (n = 13) and patients (n = 16), to identify anticipated barriers and facilitators to integrating care for depression and opioid use disorder for people with HIV via the Collaborative Care Model at a low-barrier HIV clinic. Patients and stakeholders expressed their enthusiasm for the Collaborative Care Model based on its perceived relative advantage over the standard of care referral system. Availability of resources, practical concerns about perceived fit with low-barrier HIV care, and anticipated suitability given other behavioral health comorbidities and patients’ complex socioeconomic needs partially tempered stakeholder perceptions of appropriateness for the Collaborative Care Model. Patients and service delivery stakeholders were receptive to the Collaborative Care Model, but felt it was moderately appropriate in the context of low-barrier HIV care, which necessitated key adaptations to core model components to improve its contextual fit. [ABSTRACT FROM AUTHOR]

The delayed and modest uptake of evidence-based treatments following cardiovascular clinical trials highlights the need for greater attention to implementation early in the development and testing of treatments. However, implementation science is not well understood and is often an afterthought following phase 3 trials. In this review, we describe the goals, frameworks, and methods of implementation science, along with common multilevel barriers and facilitators of implementation. We propose that some of the approaches used for implementation well after a trial has ended can be incorporated into the design of phase 3 trials to foster early post-trial implementation. Approaches include, but are not limited to, engaging broad stakeholders including patients, clinicians, and decision-makers in trial advisory boards; using less restrictive eligibility criteria that ensure both internal validity and generalizability; having trial protocols reviewed by regulators; integrating trial execution with the health care system; evaluating and addressing barriers and facilitators to deployment of the intervention; and undertaking cost-effectiveness and cost utility analyses across jurisdictions. We provide case examples to highlight concepts and to guide end-of-trial implementation. • Cardiovascular disease remains inadequately treated, highlighting the need for greater attention to implementation science. • Trialists and clinicians could harness key principles of implementation science to foster better end-of-trial uptake. • Phase 3 trials could use less restrictive eligibility criteria, engage broad stakeholders in trial design, use existing health care systems to execute the trial, evaluate barriers and facilitators to implementation, and improve workflow processes within the trial. • When trial results are positive, trial sites could be engaged in early implementation research and deployment of the intervention. [ABSTRACT FROM AUTHOR]

Introduction: The translational gap from the discovery of evidence-based solutions to their implementation in healthcare delivery organizations derives from an incorrect assumption that the need for change among executive, administrative, or clinical personnel is the same as the demand for change. For sickle cell disease (SCD), implementation of evidence-based guidelines is often delayed or obstructed due to lack of demand. This challenge allows for the persistence of resource limitations and care delivery models that do not meet the community's unique needs. Agile Storytelling is a process built on the scientific foundations of behavioral economics, complexity science, and network science to create local demand for the implementation of evidence-based solutions. Methods: Agile Storytelling includes a design phase and a testing phase. The design phase converts the evidence-based solution into a minimally viable story of a hero, a villain, struggle, drama, and a resolution. The testing phase evaluates the effectiveness of the story via a series of storytelling sprints in the target local healthcare delivery organization. The efficacy of Agile Storytelling was tested in an iterative n -of-1 case study design. Results: Agile Storytelling was used in a large, urban, healthcare system within the United States to facilitate implementation of national SCD best-practice guidelines. After repeated failures attempting to use national and local data regarding the high societal need to hire a SCD-specific social worker, an Agile change conductor using Agile Storytelling was able to create demand for the new position within a week. This decision has ultimately improved patient outcomes and led to the adoption of a specialized collaborative care team for SCD within the health network. Discussion: Agile Storytelling can lead to structured, effective, and informed storytelling to create local demand within healthcare delivery organizations. [ABSTRACT FROM AUTHOR]

Background: Delays and missed opportunities for timely treatment contribute significantly to stark inequities in cervical cancer mortality in low- and middle-income countries (LMICs) compared to high-income countries. The vast majority (approximately 90%) of new cases and deaths occur in LMICs, particularly those with high rates of HIV such as Botswana. To date, most of the implementation and cancer control research in Botswana and other LMICs has focused on cancer prevention and screening, with limited focus on cancer treatment. As such, there is a critical need to identify effective strategies to ensure timely care, and to understand contextual factors that shape the response to strategies. Without this fundamental knowledge, cervical cancer will remain a public health crisis in Botswana and other LMICs. Methods: To help fill this known gap, this study tests the effectiveness of adaptive strategies on timely treatment adoption using a hybrid (type III) Sequential Multiple Assignment Randomized Trial (SMART) design and evaluate contextual mechanisms contributing to the success or failure of each adaptive strategy. The adaptive strategies are designed to target contextual determinants identified in our prior work, including delayed communication of results to patients, individual and structural barriers to accessing treatment, and suboptimal care coordination between referring and cancer treatment clinics, and are supported by systematic evidence of the effectiveness of nudge strategies in clinical care. The primary implementation outcome is adoption, defined as the initiation of treatment within 90 days. Secondary outcomes include fidelity, reach, acceptability, implementation costs, and cancer and HIV-related clinical outcomes. The rationale for the study is that enhancing coordination, communication, and navigation through centralized outreach will both increase timely treatment adoption and be scalable and sustainable after the project is completed. Discussion: This innovative study seeks to decrease cervical cancer mortality in LMICs by developing and implementing effective and sustainable strategies that can be sustained and adapted to other contexts. Additionally, this study seeks to advance the long-term impact of global implementation science through strong and sustained partnerships in Botswana and other LMICs. Trial registration: ClinicalTrials.gov NCT05952141. Registered on July 11, 2023. https://clinicaltrials.gov/study/NCT05952141 Protocol version and date: Version 1 (September 28, 2024). [ABSTRACT FROM AUTHOR]

ABSTRACT Aim Design Methods Results Conclusion Impact Implications for the Profession This study explores contextual factors affecting evidence‐based practice (EBP) adoption among Spanish nurses, examining their interaction with sociodemographic and practice variables to enhance implementation strategies.A national multicenter study employing a cross‐sectional online survey design was conducted with an intentional sample of 934 voluntary Spanish practicing nurses.Participants completed the 12‐item Barriers/facilitators dimension of the HS‐EBP instrument, along with sociodemographic, training and practice information relevant to EBP implementation. Psychometric networks have been utilised as a novel analytical approach.The HS‐EBP scores confirm validity and reliability for assessing EBP barriers and facilitators among Spanish nurses. Primary barriers include inadequate EBP discussion spaces, lack of guidance, resistance to change, patient apathy, time constraints and support gaps. Key facilitators are higher academic qualifications, EBP training, professional development participation, educational institution employment and student supervision. Focus is needed on rural nurses and those with heavy clinical duties, facing major EBP barriers.Spanish nurses encounter organisational barriers to EBP implementation, including insufficient incentives and support. Prioritising hands‐on training, professional development in teaching and research and robust programs can enhance nurses' ability to advance EBP implementation.The HS‐EBP questionnaire is reliable in measuring organisational and contextual factors affecting EBP implementation.Successful implementation of evidence‐based practice (EBP) in the Spanish healthcare system depends on developing an appropriate organisational culture and securing support from formal leaders. Additionally, fostering an understanding of the importance of patients and their families among Spanish nurses is crucial for promoting EBP adoption. [ABSTRACT FROM AUTHOR]

Objective: Artificial intelligence (AI) is revolutionizing healthcare, but less is known about how it may facilitate methodological innovations in research settings. In this manuscript, we describe a novel use of AI in summarizing and reporting qualitative data generated from an expert panel discussion about the role of electronic health records (EHRs) in implementation science. Materials and methods: 15 implementation scientists participated in an hour-long expert panel discussion addressing how EHRs can support implementation strategies, measure implementation outcomes, and influence implementation science. Notes from the discussion were synthesized by ChatGPT (a large language model—LLM) to generate a manuscript summarizing the discussion, which was later revised by participants. We also surveyed participants on their experience with the process. Results: Panelists identified implementation strategies and outcome measures that can be readily supported by EHRs and noted that implementation science will need to evolve to assess future EHR advancements. The ChatGPT-generated summary of the panel discussion was generally regarded as an efficient means to offer a high-level overview of the discussion, although participants felt it lacked nuance and context. Extensive editing was required to contextualize the LLM-generated text and situate it in relevant literature. Discussion and conclusions: Our qualitative findings highlight the central role EHRs can play in supporting implementation science, which may require additional informatics and implementation expertise and a different way to think about the combined fields. Our experience using ChatGPT as a research methods innovation was mixed and underscores the need for close supervision and attentive human involvement. [ABSTRACT FROM AUTHOR]

Introduction: The Implementation Research Logic Model (IRLM) aids users in combining, organizing, and specifying the relationships between important constructs in implementation research. The goal of the IRLM is to improve the rigor, reproducibility, and transparency of implementation research projects. The article describing the IRLM was published September 25, 2020 (Implement Sci , Vol 15); it has since been highly cited and included as a required element in multiple funding opportunity announcements from federal agencies. The proliferation of IRLM use across dissemination and implementation research projects and practice provides an excellent opportunity to examine applications across a variety of different contexts. This protocol will result in a description of the impact of the IRLM on the field of dissemination and implementation science and guidance on refinements to the IRLM to increase its utility and impact through (1) a citation analysis, (2) a scoping review, and (3) user surveys and interviews. Methods and analysis: This protocol follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping review reporting guidelines (PRISMA-ScR). We plan to conduct a citation search and analysis of the Smith et al. 2020 article and a scoping review. The review search will be conducted in Medline, Embase, CINAHL Complete, Cochrane Library, APA PsycINFO4, ProQuest Dissertations & Theses Global, Scopus and Web of Science Core Collection., and grey literature will be searched to identify studies that use alternative logic models for implementation research. A survey will be developed from the findings of the scoping review and administered to individuals who used the IRLM. Semi-structured interviews will then be conducted with a sample of survey respondents to provide an opportunity for sequential mixed-methods analysis to achieve a deeper understanding of needed IRLM refinements and recommendations. Ethics and dissemination: Ethics approval for the scoping review and citation analysis is not applicable as only data from published literature will be used and no original data will be collected. For the survey, IRB will be completed once items are developed from the results of the scoping review and citation analysis. Results will be disseminated through peer-reviewed publications, conference presentations, and via online tools. Registration details: This protocol was registered with OSF, https://osf.io/y94bj (1). [ABSTRACT FROM AUTHOR]

Objective: To evaluate implementation of a patient decision aid for symptomatic uterine fibroid management to improve shared decision-making at five clinical settings across the United States. Methods: We used a type 3 hybrid effectiveness-implementation stepped-wedge design and the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) planning and evaluation framework. We conducted clinician training, monthly reach tracking with feedback to site clinical leads, patient and clinician surveys, and visit audio-recordings. Implementation strategies included assessment of organizational readiness for shared decision-making, synchronous clinician training, audit and feedback of decision aid reach, and access to multiple decision aid formats. Outcomes and analyses included patient-level reach, clinician-level adoption, and associations of patient-reported decision aid exposure (as treated) and setting-level implementation (intention-to-treat) with patient-reported (collaboRATE measure) and observed (OPTION-5 measure) shared decision-making. We also designed and assessed setting-level plans for sustainability and other factors impacting sustained decision aid use. Results: The decision aid was adopted by 72 of the 74 eligible gynecologists (97%) and reached 2553 patients across five settings. CollaboRATE scores improved among patients who reported receiving the decision aid (as-treated analysis, 69% vs. 59%; OR 1.6, 95% CI 1.16–2.27). CollaboRATE scores remained consistent before and after setting-level decision aid implementation (intention-to-treat analysis, 64% vs. 63%; OR 0.86, 95% CI 0.61–1.22). Participants would prefer to receive a decision aid at multiple time points (91.9% before the visit, 90.7% during the visit, 86.5% after the visit). Shared decision-making experiences did not improve when comparing pre vs. post-implementation collaboRATE scores across included settings (intention-to-treat, 64% vs. 63%; OR 0.86, 95% CI 0.61–1.22). Conclusion: When patients with symptomatic uterine fibroids are given decision aids, they report higher shared decision-making scores. However, the differences we observed between the as-treated and intention-to-treat results suggest that unaddressed implementation challenges continue to limit the extent to which patients receive decision aids and likely hinder their overall impact. Future efforts to implement decision aids should explore enhancing their integration into clinical workflows and standard operating procedures, supported by organizational incentives that prioritize shared decision-making. Trial registration: ClinicalTrials.gov NCT03985449; registered 6 June 2019. [ABSTRACT FROM AUTHOR]

Background: Implementation strategies are key to enhancing the translation of new innovations but there is a need to systematically design and tailor strategies to match the targeted implementation context and address determinants. There are increasing methods to inform the development and tailoring of implementation strategies to maximize their usability, feasibility, and appropriateness in new settings such as the Cognitive Walkthrough for Implementation Strategies (CWIS) approach. The aim of the current project is to apply the CWIS approach to inform the redesign of a multifaceted selection-quality implementation toolkit entitled Adoption of Curricular supports Toolkit: Systematic Measurement of Appropriateness and Readiness for Translation in Schools (ACT SMARTS) for use in middle and high schools. Methods: We systematically applied CWIS as the second part of a community-partnered iterative redesign of ACT SMARTS for schools to evaluate the usability and inform further toolkit redesign areas. We conducted three CWIS user testing sessions with key end users of school district administrators (n = 3), school principals (n = 6), and educators (n = 6). Results: Our CWIS application revealed that end users found ACT SMARTS acceptable and relevant but anticipate usability issues engaging in the ACT SMARTS process. Results informed the identification of eleven usability issues and corresponding redesign solutions to enhance the usability of ACT SMARTS for use in middle and high schools. Conclusions: Results indicated the utility of CWIS in assessing implementation strategy usability in service of informing strategy modification as part of our broader redesign to improve alignment with end user, end recipient, and setting needs. Recommendations regarding the use of this participatory approach are discussed. [ABSTRACT FROM AUTHOR]

Background: Dissemination and implementation science is an evolving field that focuses on the strategies and mechanisms by which scientific evidence is adopted, used, and sustained in clinical and community practice. Main body: Implementation scientists are confronted by the challenge to balance rigor and generalizability in their work while also attempting to speed the translation of evidence into clinical and community practice. Hybrid Effectiveness-Implementation studies and the RE-AIM framework were conceptualized to address these challenges. Hybrid Effectiveness-Implementation (HEI) studies provide methods of examining the effectiveness of health promoting interventions while concurrently assessing the utility of dissemination and implementation strategies designed to enhance the application of evidence-based principles in practice. RE-AIM provides a set of planning and evaluation dimensions that can be assessed with a goal to balance internal and external validity. The purpose of this commentary is to provide clarity on definitions of each approach and how to effectively use them together to answer research questions that will advance dissemination and implementation science for health promotion. Conclusions: We provide examples of concerted use of RE-AIM within HEI studies from the literature and focus on language to provide a clarity and consistency across research questions, designs, and settings. We share how to operationalize RE-AIM dimensions in HEI studies for both dissemination and implementation strategies. Future directions include refining, defining, and evaluating each RE-AIM dimension within hybrid studies. [ABSTRACT FROM AUTHOR]

Sleep is a crucial aspect of geriatric assessment for hospitalized older adults, and implementing AI-driven technology for sleep monitoring can significantly enhance the rehabilitation process. Sleepsense, an AI-driven sleep-tracking device, provides real-time data and insights, enabling healthcare professionals to tailor interventions and improve sleep quality. This study explores the perspectives of an interdisciplinary hospital team on implementing Sleepsense in geriatric hospital care. Using the interpretive description approach, we conducted focus groups with physicians, nurses, care aides, and an activity worker. The Consolidated Framework for Implementation Research (CFIR) informed our thematic analysis to identify barriers and facilitators to implementation. Among 27 healthcare staff, predominantly female (88.89%) and Asian (74.1%) and mostly aged 30–50 years, themes emerged that Sleepsense is perceived as a timesaving and data-driven tool that enhances patient monitoring and assessment. However, barriers such as resistance to change and concerns about trusting the device for patient comfort and safety were noted, while facilitators included training and staff engagement. The CFIR framework proved useful for analyzing implementation barriers and facilitators, suggesting future research should prioritize effective strategies for interdisciplinary team support to enhance innovation adoption and patient outcomes in rehabilitation settings. [ABSTRACT FROM AUTHOR]

Purpose: The importance and effectiveness of palliative care (PC) in intensive care units (ICU) are known. Less is known about the implementation and integration of ICU-based PC interventions. This systematic review aims to use a modified implementation research logic model (IRLM) to identify, map, and synthesise evidence on implementation of ICU–PC (primary and/or specialist) interventions. Methods: This systematic review used an adapted Smith's IRLM to understand relationships between implementation factors—determinants (barriers and facilitators), strategies, and mechanisms—and report intervention characteristics and outcomes. Searches up to 2nd December 2023, of MEDLINE, Embase, Cochrane, CINAHL, and PsycINFO, combined PC, intensive care, and implementation terms. Results: 84 studies (8 process evaluations, 76 effectiveness studies) were included. Published evidence on ICU–PC interventions is substantial, but reporting on implementation factors is variable and often lacking, especially for patient and family-related determinants and for all aspects of mechanisms. Main facilitators for implementation are adequate resources and collaboration between PC and ICU teams. Main barriers to implementation are lack of resources, negative perceptions of PC, and high ICU acuity. Implementation strategies include auditing resources, building stakeholder collaboratives, creating adaptable interventions, utilising champions, and supporting education. Mechanisms most commonly worked by facilitating collaborative working. Conclusion: This review provides recommendations for ICUs when designing (stakeholder involvement, ICU–PC collaboration, assessment of culture and resources); implementing (targeted and adapted strategies, champions, and education); and evaluating/reporting (collect effectiveness and implementation data, including mechanisms) ICU–PC interventions. Use of implementation structures and patient/family involvement are both needed and important to be included. [ABSTRACT FROM AUTHOR]

Background Neonatal hypothermia significantly contributes to infant morbidity and mortality in low-resource settings like Malawi. Kangaroo mother care (KMC) is essential but faces challenges in providing continuous thermal support. The Dream Warmer is a neonatal warming device that was developed to complement KMC. We studied its implementation outside a research environment. Methods Using an implementation science approach, we conducted a prospective interventional cohort study in two hospitals and four health centres in Malawi. Through audits and surveys, we assessed the effect of the Dream Warmer on neonatal hypothermia as well as healthcare provider (HCP) and parent attitudes regarding thermoregulation and related issues. Results The Dream Warmer raised no safety concerns and effectively treated hypothermia in 90% of uses. It was positively received by HCPs and parents, who reported it had a favourable effect on the care of small and sick newborns. Challenges identified included a scarcity of water and electricity, lack of availability of the device and HCPs forgetting to prepare it in advance of need or to use it when indicated. Feedback for future training was obtained. The Dream Warmer's strong safety and effectiveness performance is consistent with results from strict research studies. Training materials can be adapted to optimize integration into daily practice and provide educational content for parents. Conclusions The Dream Warmer is a safe and effective device to treat neonatal hypothermia, particularly when KMC is insufficient. We gained an understanding of how to optimize implementation through robust HCP and family education to help combat hypothermia. [ABSTRACT FROM AUTHOR]

Background: Implementing evidence that changes practice in emergency departments (EDs) is notoriously difficult due to well-established barriers including high levels of uncertainty arising from undifferentiated nature of ED patients, resource shortages, workload unpredictability, high staff turnover, and a constantly changing environment. We developed and implemented a behaviour-change informed strategy to mitigate these barriers for a clinical trial to implement the evidence-based emergency nursing framework HIRAID® (History including Infection risk, Red flags, Assessment, Interventions, Diagnostics, communication, and reassessment) to reduce clinical variation, and increase safety and quality of emergency nursing care. Aim: To evaluate the behaviour-change-informed HIRAID® implementation strategy on reach, effectiveness, adoption, quality (dose, fidelity) and maintenance (sustainability). Methods: An effectiveness-implementation hybrid design including a step–wedge cluster randomised control trial (SW-cRCT) was used to implement HIRAID® with 1300 + emergency nurses across 29 Australian rural, regional, and metropolitan EDs. Evaluation of our behaviour-change informed strategy was informed by the RE-AIM Scoring Instrument and measured using data from (i) a post HIRAID® implementation emergency nurse survey, (ii) HIRAID® Instructor surveys, and (iii) twelve-week and 6-month documentation audits. Quantitative data were analysed using descriptive statistics to determine the level of each component of RE-AIM achieved. Qualitative data were analysed using content analysis and used to understand the 'how' and 'why' of quantitative results. Results: HIRAID® was implemented in all 29 EDs, with 145 nurses undertaking instructor training and 1123 (82%) completing all four components of provider training at 12 weeks post-implementation. Modifications to the behaviour-change informed strategy were minimal. The strategy was largely used as intended with 100% dose and very high fidelity. We achieved extremely high individual sustainability (95% use of HIRAID® documentation templates) at 6 months and 100% setting sustainability at 3 years. Conclusion: The behaviour-change informed strategy for the emergency nursing framework HIRAID® in rural, regional, and metropolitan Australia was highly successful with extremely high reach and adoption, dose, fidelity, individual and setting sustainability across substantially variable clinical contexts. Trial registration: ANZCTR, ACTRN12621001456842. Registered 25 October 2021. [ABSTRACT FROM AUTHOR]

Background: Female sex workers (FSWs) face a confluence of multilevel barriers to HIV care. In South Africa, 63% of FSWs are living with HIV and, <40% are virally suppressed. The objective of this analysis was to identify implementation determinants of 2 HIV treatment support strategies. Methods: The Siyaphambili trial tested a decentralized treatment provision and an individualized case management strategy aimed to support FSWs living with unsuppressed HIV viral loads. We identified a nested sample of trial participants using maximum variation sampling (n = 36) as well as a purposively selected sample of implementors (n = 12). We used semistructured interview guides, developed using the Consolidated Framework for Implementation Research (CFIR) and deductively coded the transcripts using CFIR, systematically assessing the strength and valence of implementation. We compared construct ratings to determine whether any constructs distinguished implementation across strategies. Results: Across 3 CFIR domains (innovation characteristics, inner setting, and outer setting), 12 constructs emerged as facilitating, hindering, or having mixed effects on strategy implementation. The relative advantage, design, adaptability, and complexity constructs of the innovation characteristics and the work infrastructure construct of the inner setting were strongly influential (± 2 or +2). While the majority of construct valence and strength rating (9–12) were not distinguishing across strategies, we observed 3 weakly distinguishing CFIR constructs (relative advantage, complexity, and available resources). Conclusions: Given the potential benefits of differentiated service delivery strategies, identifying the relative importance of implementation determinants facilitates transparency and evaluation, supporting future strategy design and implementation. Optimizing implementation will support addressing inequities in HIV care and treatment services. [ABSTRACT FROM AUTHOR]

Purpose: This study consisted of a psychometric analysis of the brief and expanded versions of the Organizational Readiness for Implementing Change (ORIC) to assess the scale's validity and reliability in a child welfare setting. Method: Based on responses from a sample of 313 child welfare professionals, factorial structure was assessed through exploratory and confirmatory factor analysis. Construct validity was assessed through correlations with theoretically linked concepts, and internal consistency was assessed through Cronbach's alpha. Results: Factor analysis yielded a unidimensional structure for the brief version and a two-factor structure for the expanded version. However, the models did not meet the standards for sufficient fit, as evidenced by the fit indices. Both the brief and expanded versions were significantly and positively correlated with theoretically linked concepts. Internal consistency of the brief (α =.96) and expanded versions (α =.96) was excellent. Discussion: Though the measure holds promise with respect to construct validity and internal consistency, further testing of the factorial structure in a variety of child welfare samples is warranted. Conclusion: This study supports the field of child welfare with information regarding one possible measure of organizational readiness, an important concept in implementation science. [ABSTRACT FROM AUTHOR]

Pragmatic measures of evidence-based practice (EBP) implementation can support and evaluate implementation efforts. We examined the predictive validity of therapist reports of EBP strategy delivery for children's mental health outcomes. Data were obtained from 1,380 sessions with 248 children delivered by 76 therapists in two county systems. Children (Mage=11.8 years, SD = 3.7) presented with internalizing (52%), externalizing (27%), trauma (16%), and other (5%) concerns. Therapists reported their delivery of EBP strategies on a revised version of the EBP Concordant Care Assessment (ECCA; Brookman-Frazee, et al., Administration and Policy in Mental Health and Mental Health Services Research, 48, 155–170, 2021) that included 25 content (e.g., parenting, cognitive behavioral) and 12 technique strategies (e.g., modeling, practice/role-play). On average, 5.6 ECCA session reports (SD = 2.3) were obtained for each client, and caregivers reported symptoms on the Brief Problem Checklist (Chorpita, et al., Journal of Consulting and Clinical Psychology, 78(4), 526–536, 2010) at baseline, weekly over two months, and again at four months. Multilevel models examined whether the mean extensiveness of each EBP strategy predicted trajectories of child outcomes. More individual technique (6 of 12) than content strategies (1 of 25) were associated with outcome trajectories. For techniques, more extensive use of Performance Feedback and Live Coaching and less extensive use of Addressing Barriers were associated with greater declines in total symptoms, and more extensive use of Establishing/Reviewing Goals, Tracking/Reviewing Progress, and Assigning/Reviewing Homework was associated with declines in externalizing symptoms. For content, more extensive use of Cognitive Restructuring was associated with declines in total symptoms. In addition, higher average extensiveness ratings of the top content strategy across sessions was associated with greater declines in total and externalizing symptoms. Therapist-reported delivery of some EBP strategies showed evidence of predictive validity and may hold utility in indexing quality of care. [ABSTRACT FROM AUTHOR]