Search

Abstract Background Evidence suggests that prehabilitation interventions, which optimise physical and mental health prior to treatment, can improve outcomes for surgical cancer patients and save costs to the health system through faster recovery and fewer complications. However, robust, theory-based evaluations of these programmes are needed. Using a theory of change (ToC) approach can guide evaluation plans by describing how and why a programme is expected to work. Theories of Change have not been developed for cancer prehabilitation programmes in the literature to date. This paper aims to provide an overview of the methodological steps we used to retrospectively construct a ToC for Prehab2Rehab (P2R), a cancer prehabilitation programme being implemented by the Cardiff and Vale University Health Board. Methods We used an iterative, participatory approach to develop the ToC. Following a literature review and document analysis, we facilitated a workshop with fourteen stakeholders from across the programme using a ‘backwards mapping’ approach. After the workshop, stakeholders had three additional opportunities to refine and validate a final working version of the ToC. Results Our process resulted in the effective and timely development of a ToC. The ToC captures how P2R’s interventions or activities are expected to bring about short, medium and long-term outcomes that, collectively, should result in the overarching desired impacts of the programme, which were improved patient flow and reduced costs to the health system. The process of developing a ToC also enabled us to have a better understanding of the programme and build rapport with key stakeholders. Conclusions The ToC has guided the design of an evaluation that covers the complexity of P2R and will generate lessons for policy and clinical practice on supporting surgical cancer patients in Wales and beyond. We recommend that evaluators apply a ToC development process at the outset of evaluations to bring together stakeholders and enhance the utilisation of the findings. This paper details a pragmatic, efficient and replicable process that evaluators could adopt to develop a ToC. Theory-informed evaluations may provide better evidence to develop and refine cancer prehabilitation interventions and other complex public health interventions.

Abstract Background Co-design with consumers and healthcare professionals is widely used in applied health research. While this approach appears to be ethically the right thing to do, a rigorous evaluation of its process and impact is frequently missing. Evaluation of research co-design is important to identify areas of improvement in the methods and processes, as well as to determine whether research co-design leads to better outcomes. We aimed to build on current literature to develop a framework to assist researchers with the evaluation of co-design processes and impacts. Methods A multifaceted, iterative approach, including three steps, was undertaken to develop a Co-design Evaluation Framework: 1) A systematic overview of reviews; 2) Stakeholder panel meetings to discuss and debate findings from the overview of reviews and 3) Consensus meeting with stakeholder panel. The systematic overview of reviews included relevant papers published between 2000 and 2022. OVID (Medline, Embase, PsycINFO), EBSCOhost (Cinahl) and the Cochrane Database of Systematic reviews were searched for papers that reported co-design evaluation or outcomes in health research. Extracted data was inductively analysed and evaluation themes were identified. Review findings were presented to a stakeholder panel, including consumers, healthcare professionals and researchers, to interpret and critique. A consensus meeting, including a nominal group technique, was applied to agree upon the Co-design Evaluation Framework. Results A total of 51 reviews were included in the systematic overview of reviews. Fifteen evaluation themes were identified and grouped into the following seven clusters: People (within co-design group), group processes, research processes, co-design context, people (outside co-design group), system and sustainment. If evaluation methods were mentioned, they mainly included qualitative data, informal consumer feedback and researchers’ reflections. The Co-Design Evaluation Framework used a tree metaphor to represent the processes and people in the co-design group (below-ground), underpinning system- and people-level outcomes beyond the co-design group (above-ground). To evaluate research co-design, researchers may wish to consider any or all components in the tree. Conclusions The Co-Design Evaluation Framework has been collaboratively developed with various stakeholders to be used prospectively (planning for evaluation), concurrently (making adjustments during the co-design process) and retrospectively (reviewing past co-design efforts to inform future activities).

Abstract Background Medical education is a multifarious endeavour integrating a range of pedagogies and philosophies. Complexity as a science or theory (‘complexity’) signals a move away from a reductionist paradigm to one which appreciates that interactions in multi-component systems, such as healthcare systems, can result in adaptive and emergent outcomes. This examination of the nexus between medical education and complexity theory aims to discover ways that complexity theory can inform medical education and medical education research. Methods A structured literature review was conducted to examine the nexus between medical education and complexity; 5 databases were searched using relevant terms. Papers were included if they engaged fully with complexity as a science or theory and were significantly focused on medical education. All types of papers were included, including conceptual papers (e.g. opinion and theoretical discussions), case studies, program evaluations and empirical research. A narrative and thematic synthesis was undertaken to create a deep understanding of the use of complexity in medical education. Results Eighty-three papers were included; the majority were conceptual papers. The context and theoretical underpinnings of complexity as a relevant theory for medical education were identified. Bibliographic and temporal observations were noted regarding the entry of complexity into medical education. Complexity was relied upon as a theoretical framework for empirical studies covering a variety of elements within medical education including: knowledge and learning theories; curricular, program and faculty development; program evaluation and medical education research; assessment and admissions; professionalism and leadership; and learning for systems, about systems and in systems. Discussion There is a call for greater use of theory by medical educators. Complexity within medical education is established, although not widespread. Individualistic cultures of medicine and comfort with reductionist epistemologies challenges its introduction. However, complexity was found to be a useful theory across a range of areas by a limited number of authors and is increasingly used by medical educators and medical education researchers. This review has further conceptualized how complexity is being used to support medical education and medical education research. Conclusion This literature review can assist in understanding how complexity can be useful in medical educationalists' practice.

Plain English summary Children and young people (CYP) have a right to be involved in things that affect them, including research. There is growing interest in children and young people’s involvement in health research in the United Kingdom (UK), as well as understanding that what works for CYP is often different to what works for adults. This paper presents an evaluation of the Young Person's Advisory Group (YPAG) at Moorfields Eye Hospital in London. Evaluation uses research methods to find out how well a service or project is working and meeting the needs of those who use it, and how to improve things that could be better. A group of young evaluators, supported by adult researchers, designed and ran the evaluation; three of the young evaluators also helped write this paper. In our evaluation we used focus (discussion) groups and online surveys with young group members, their parents and carers, researchers who had worked with the group, the group’s facilitators (adults who help manage the group) and funders. We found that group members valued the social and creative aspects of the group as well as learning about research and developing skills and confidence. Learning was a two-way process, with both researchers and facilitators talking about how much they had learnt from working with the YPAG. All participants talked about the importance of feeling that CYP are making a difference to research, as well as of CYP’s right to be involved. Planning and support were important to the group working well, but we found that having the money and time to do this well was not always easy. And, while lots of researchers were keen to work with the group, and talked about how this had helped their research, we need to do more to engage researchers who have yet to be convinced. We also found that, while we wanted to ‘co-produce’ the group and share power for all big decisions, this was something we had to work towards, especially when group members were young and/or new to research and involvement. Co-producing an evaluation helped us to learn about the benefits and challenges of involving CYP in research, as well as how to involve them in evaluating that involvement.

Abstract Background Artificial intelligence (AI) is seen as one of the major disrupting forces in the future healthcare system. However, the assessment of the value of these new technologies is still unclear, and no agreed international health technology assessment-based guideline exists. This study provides an overview of the available literature in the value assessment of AI in the field of medical imaging. Methods We performed a systematic scoping review of published studies between January 2016 and September 2020 using 10 databases (Medline, Scopus, ProQuest, Google Scholar, and six related databases of grey literature). Information about the context (country, clinical area, and type of study) and mentioned domains with specific outcomes and items were extracted. An existing domain classification, from a European assessment framework, was used as a point of departure, and extracted data were grouped into domains and content analysis of data was performed covering predetermined themes. Results Seventy-nine studies were included out of 5890 identified articles. An additional seven studies were identified by searching reference lists, and the analysis was performed on 86 included studies. Eleven domains were identified: (1) health problem and current use of technology, (2) technology aspects, (3) safety assessment, (4) clinical effectiveness, (5) economics, (6) ethical analysis, (7) organisational aspects, (8) patients and social aspects, (9) legal aspects, (10) development of AI algorithm, performance metrics and validation, and (11) other aspects. The frequency of mentioning a domain varied from 20 to 78% within the included papers. Only 15/86 studies were actual assessments of AI technologies. The majority of data were statements from reviews or papers voicing future needs or challenges of AI research, i.e. not actual outcomes of evaluations. Conclusions This review regarding value assessment of AI in medical imaging yielded 86 studies including 11 identified domains. The domain classification based on European assessment framework proved useful and current analysis added one new domain. Included studies had a broad range of essential domains about addressing AI technologies highlighting the importance of domains related to legal and ethical aspects.

Abstract Background An increasing number of evaluations of social accountability (SA) interventions have been published in the past decade, however, reporting gaps make it difficult to summarize findings. We developed the Social Accountability Reporting for Research (SAR4Research) checklist to support researchers to improve the documentation of SA processes, context, study designs, and outcomes in the peer reviewed literature and to enhance application of findings. Methods We used a multi-step process, starting with an umbrella review of reviews on SA to identify reporting gaps. Next, we reviewed existing guidelines for reporting on behavioral interventions to determine whether one could be used in its current or adapted form. We received feedback from practitioners and researchers and tested the checklist through three worked examples using outcome papers from three SA projects. Results Our umbrella review of SA studies identified reporting gaps in all areas, including gaps in reporting on the context, intervention components, and study methods. Because no existing guidelines called for details on context and the complex processes in SA interventions, we used CONSORT-SPI as the basis for the SAR4Research checklist, and adapted it using other existing checklists to fill gaps. Feedback from practitioners, researchers and the worked examples suggested the need to eliminate redundancies, add explanations for items, and clarify reporting for quantitative and qualitative study components. Conclusions Results of SA evaluations in the peer-reviewed literature will be more useful, facilitating learning and application of findings, when study designs, interventions and their context are described fully in one or a set of papers. This checklist will help authors report better in peer-reviewed journal articles. With sufficient information, readers will better understand whether the results can inform accountability strategies in their own contexts. As a field, we will be better able to identify emerging findings and gaps in our understanding of SA.

Abstract Background The AgResults initiative tests the efficacy of results-based prizes to scale-up smallholder technology adoption. In Nigeria the project awarded a $18.75/ton prize for private sector actors who aggregated maize from smallholders that was treated by Aflasafe—a biocontrol that addresses liver cancer-causing aflatoxin contamination in maize. This paper examines the impact of AgResults initiative on smallholder farmers. Methods This evaluation estimates the causal effect of the AgResults program on farmer outcomes by comparing survey data from AgResults farmers to survey data from a matched comparison group of farmers. To improve balance, we use propensity score weights. In considering inestimable selection bias, we describe several key considerations, including the inclusion of comparison areas for treatment post-evaluation. Results The project increased Aflasafe adoption by 56% points, farmers earned 16% more net maize income on average. However, the majority of farmers in villages engaged by the project did not know about Aflasafe’s health benefits. This suggests that complimentary donor-directed efforts may still be needed to generate general awareness about the technologies whose benefit is not immediately visible to the smallholders. With the prize focused on aggregation, private sector actors may have reduced incentive to raise awareness about Aflasafe’s health benefits in case farmers held back Aflasafe-treated maize for consumption. Conclusions This paper highlights the potential of results-based prizes to engage the private sector in solving development problems. However, it also equally highlights the gaps that such an approach may have, arguing for the need for having complimentary efforts to address those gaps. This is particularly the case when the technology’s benefits are not perceived by the consumer (aflatoxins are not visible to the eye, and the health benefits are not immediate), or when the technology results in positive externalities (final consumers of Aflasafe-treated maize also benefit). Broader consumer awareness needs to be raised to promote continued development of the market for Aflasafe-treated maize, aflatoxin standards need to be enforced, and aflatoxin testing needs to be more easily available.

Abstract Background Maintaining quality of care and improving the quality of life (QOL) of patients and their families are important issues in palliative care. Therefore, there is a need to continuously evaluate the quality and outcomes of the care provided. In Japan, the Japan hospice and palliative evaluation (J-HOPE) study series has been conducted every three to four years since 2010, and we will conduct the fifth J-HOPE study (J-HOPE5). In the present paper, we describe the protocol of the J-HOPE5 study. The main objectives are: (1) to evaluate the processes, structures and outcomes of care at hospices or palliative care units; (2) to examine bereaved family members’ self-reported psychosocial condition, such as grief and depression as bereavement outcomes; (3) to provide data to ensure and improve the quality of care provided by participating institutions via feedback based on the results from each institution; and (4) to provide clinical and academic information regarding the implications of various issues in palliative care by conducting specific research. Methods We will conduct a cross-sectional, anonymous, self-reported questionnaire survey. In total, 153 institutions have agreed to participate in this study, meaning that approximately 12,240 bereaved family members (n = 80/institution) will be sent a questionnaire. Discussion This is one of the largest cross-sectional bereavement surveys to evaluate the quality of specialized palliative care for patients with cancer, both in Japan and worldwide. The large sample size of this study will enable wide analyses of specific targets and topics.

Abstract Background The primary aim of this scoping review was to synthesise key domains and sub-domains described in existing clinical decision support systems (CDSS) implementation frameworks into a novel taxonomy and demonstrate most-studied and least-studied areas. Secondary objectives were to evaluate the frequency and manner of use of each framework, and catalogue frameworks by implementation stage. Methods A scoping review of Pubmed, Scopus, Web of Science, PsychInfo and Embase was conducted on 12/01/2022, limited to English language, including 2000–2021. Each framework was categorised as addressing one or multiple stages of implementation: design and development, evaluation, acceptance and integration, and adoption and maintenance. Key parts of each framework were grouped into domains and sub-domains. Results Of 3550 titles identified, 58 papers were included. The most-studied implementation stage was acceptance and integration, while the least-studied was design and development. The three main framework uses were: for evaluating adoption, for understanding attitudes toward implementation, and for framework validation. The most frequently used framework was the Consolidated Framework for Implementation Research. Conclusions Many frameworks have been published to overcome barriers to CDSS implementation and offer guidance towards successful adoption. However, for co-developers, choosing relevant frameworks may be a challenge. A taxonomy of domains addressed by CDSS implementation frameworks is provided, as well as a description of their use, and a catalogue of frameworks listed by the implementation stages they address. Future work should ensure best practices for CDSS design are adequately described, and existing frameworks are well-validated. An emphasis on collaboration between clinician and non-clinician affected parties may help advance the field.

Abstract Background Due to the global digitalization, implementation of digital elements into daily work can support physiotherapists’ work but may also pose some challenges. Only little is known about physiotherapists’ attitude towards digitalization. This study primarily aimed to analyze physiotherapists’ attitude towards digitalization and to what extend digital tools have been implemented into their daily work. In second analysis, participants’ characteristics such as age, working place, gender and mode of survey participation were assessed. Methods A 12-main-item survey amongst voluntary course participants of one physiotherapeutic training center was conducted via paper-based as well as online questionnaires between July 2018 and June 2019 including questions on participants’ general as well as particular attitude towards digitalization, the use of (mobile) applications and possible advantages and disadvantages of the ongoing digital transformation. Sub-analysis was performed for age (≤40 years versus > 40 years), gender, mode of participation (paper vs. online) and working place (practice vs. hospital). Results Overall, 488 physiotherapists participated in the survey. In comparison of the age groups, younger participants had more concerns about data security (p = 0.042) and insufficient financial remuneration (p

Abstract Background Various questionnaires are used for evaluating satisfaction, usability, acceptance, and quality outcomes of mobile health (mHealth) services. Using the best one to meet the needs of an mHealth study is a challenge for researchers. Therefore, this study aimed to review and determine the frequently used questionnaires for evaluating the mentioned outcomes of mHealth services. Methods The PubMed database was searched for conducting this review in April 2021. Papers that used a referenced questionnaire to evaluate the satisfaction, usability, acceptance, or quality outcomes of mHealth were included. The first author’s name, year of publication, evaluation outcome, and evaluation questionnaire were extracted from relevant papers. Data were analyzed using descriptive statistics. Results In total, 247 papers were included in the study. Questionnaires were used for usability (40%), quality (34.5%), acceptance (8.5%), and satisfaction (4%) outcomes, respectively. System usability scale (36.5%), mobile application rating scale (35.5%), post study system usability questionnaire (6%), user mobile application rating scale (5%), technology acceptance model (4.5%), computer system usability questionnaire (2.5%), net promoter score (2%), health information technology usability evaluation scale (2%), the usefulness, satisfaction, and ease of use (1.5%), client satisfaction questionnaire (1.5%), unified theory of acceptance and use of technology (1.5%), questionnaire for user interaction satisfaction (1%), user experience questionnaire (1%), and after-scenario questionnaire (1%) were the most used questionnaires, respectively. Conclusion Despite the existence of special questionnaires for evaluating several outcomes of mHealth, general questionnaires with fewer items and higher reliability have been used more frequently. Researchers should pay more attention to questionnaires with a goal-based design.

Abstract Background The role of evaluation evidence in guiding health systems strengthening (HSS) investments at the global-level remains contested. A lack of rigorous impact evaluations is viewed by some as an obstacle to scaling resources. However, others suggest that power dynamics and knowledge hierarchies continue to shape perceptions of rigor and acceptability in HSS evaluations. This debate has had major implications on HSS resource allocation in global-level funding decisions. Yet, few studies have examined the relationship between HSS evaluation evidence and prioritization of HSS. In this paper, we explore the perspectives of key global health stakeholders, specifically around the nature of evidence sought regarding HSS and its potential impact on prioritization, the challenges in securing such evidence, and the drivers of intra- and inter-organizational divergences. We conducted a stakeholder analysis, drawing on 25 interviews with senior representatives of major global health organizations, and utilized inductive approaches to data analysis to develop themes. Results Our analysis suggests an intractable challenge at the heart of the relationship between HSS evaluations and prioritization. A lack of evidence was used as a reason for limited investments by some respondents, citing their belief that HSS was an unproven and potentially risky investment which is driven by the philosophy of HSS advocates rather than evidence. The same respondents also noted that the ‘holy grail’ of evaluation evidence that they sought would be rigorous studies that assess the impact of investments on health outcomes and financial accountability, and believed that methodological innovations to deliver this have not occurred. Conversely, others held HSS as a cross-cutting principle across global health investment decisions, and felt that the type of evidence sought by some funders is unachievable and not necessary – an ‘elusive quest’ – given methodological challenges in establishing causality and attribution. In their view, evidence would not change perspectives in favor of HSS investments, and evidence gaps were used as a ‘convenient excuse’. Respondents raised additional concerns regarding the design, dissemination and translation of HSS evaluation evidence. Conclusions Ongoing debates about the need for stronger evidence on HSS are often conducted at cross-purposes. Acknowledging and navigating these differing perspectives on HSS evaluation may help break the gridlock and find a more productive way forward.

Abstract Background On January 31st, 2023, the province of British Columbia (BC), Canada, was granted a federal exemption allowing adults (aged 18 +) to possess up to 2.5 g of select illicit drugs. The exemption will be in place for three years (2023–2026), marking the first formal decriminalization of illicit drug policy reform in Canada. BC’s decriminalization initiative is premised on several goals. This project seeks to evaluate each of these goals and their individual and combined contributions to determine the overall success of this policy. Methods The following protocol paper provides a detailed outline of a five-year (2022-2027) national evaluation of BC’s decriminalization initiative, as well as the specific objectives, methodologies, and planned analyses for eight interrelated sub-studies that comprise the evaluation. These sub-studies fall under the following five topical areas of research: 1) people who use drugs (PWUD), 2) the police and the criminal justice system, 3) the general public, 4) the health services system, and 5) an economic analysis. Additional research activities may also be explored. Results The overall evaluation and specific sub-study designs were informed by intensive stakeholder engagement. The evaluation was developed in collaboration with an international expert committee who came together to undertake a nominal group technique to decide on the final evaluation design and corresponding logic model. The evaluation will also employ an advisory board and individual sub-study working groups comprised of experts and PWUD who will oversee the development and implementation of the overall evaluation as well as each sub-study. Discussion This evaluation will draw on implementation science research practices to evaluate and understand the full impacts of this novel drug policy experiment. Results will be widely disseminated through manuscripts, reports, presentations, and infographics, which will be adapted and tailored for specific audiences. The protocol identifies several anticipated challenges and limitations. This evaluation’s evidence-based findings will be poised to offer pivotal insights that can shape and refine the discourse on drug policy and will serve as a critical resource for understanding the multifaceted impacts of decriminalization.

Abstract Background Addressing the upstream social determinants of health (e.g. built environment, education) can reduce the burden of non-communicable diseases. To do so effectively often requires system-wide collaboration. However, collaborating across multiple sectors, organizations and disciplines within a complex system can be challenging. ActEarly was a public health research consortium that aimed to improve child health by building an interdisciplinary, cross-city partnership to develop and/or evaluate upstream interventions, increase research capacity and improve collaboration between researchers, local authorities and communities. This paper explores ActEarly’s experiences of navigating complexity to identify mechanisms that supported its implementation and proposes recommendations for future intersectoral and interdisciplinary population health research collaborations. Methods We conducted a longitudinal qualitative study of ActEarly, integrating findings from inductive documentary analysis of internal documents (mainly meetings minutes and reports) (n = 114) and interviews (n = 70) with 45 consortium members at three different timepoints (2018, 2021, 2023). Participants worked across different organizations, cities, roles and levels of seniority in the consortium. Findings Clarity, Unity, Flexibility and Feasibility were seen as the key mechanisms required to support ActEarly’s implementation. Clear aims, governance structures and communication were necessary to manage the uncertainty of the complex system. A unified approach, characterized by strong relationships, having a shared vision and communal access to resources supported effective collaboration. Flexibility was required to adjust to different ways of working, respond to wider system events and manage the consortium. Establishing feasible aims that responded to the limitations of the system, the available resources and research infrastructure was required for teams to deliver the work. Conclusions Implementing multi-faceted programmes in a complex system can be challenging. We recommend that future whole-systems consortia seeking to improve population health build Clarity, Unity, Flexibility and Feasibility into their programmes, noting the complex interrelationships between these factors. Iterative reflections from all parties should support delivery amidst the uncertainty that comes with running a population health research collaboration, and strong leadership and governance should play a key role in ensuring that these are built into foundations the programme.

Abstract Background In 2016, a voluntary National Healthy Food and Drink Policy (hereafter, “the Policy”) was released to encourage public hospitals in New Zealand to provide food and drink options in line with national dietary guidelines. Five years later, eight (of 20) organisations had adopted it, with several preferring to retain or update their own institutional-level version. This study assessed staff and visitors’ awareness and support for and against the Policy, and collected feedback on perceived food environment changes since implementation of the Policy. Methods Cross-sectional electronic and paper-based survey conducted from June 2021 to August 2022. Descriptive statistics were used to present quantitative findings. Free-text responses were analysed following a general inductive approach. Qualitative and quantitative findings were compared by level of implementation of the Policy, and by ethnicity and financial security of participants. Results Data were collected from 2,526 staff and 261 visitors in 19 healthcare organisations. 80% of staff and 56% of visitors were aware of the Policy. Both staff and visitors generally supported the Policy, irrespective of whether they were aware of it or not, with most agreeing that “Hospitals should be good role models.” Among staff who opposed the Policy, the most common reason for doing so was freedom of choice. The Policy had a greater impact, positive and negative, on Māori and Pacific staff, due to more frequent purchasing onsite. Most staff noticed differences in the food and drinks available since Policy implementation. There was positive feedback about the variety of options available in some hospitals, but overall 40% of free text comments mentioned limited choice. 74% of staff reported that food and drinks were more expensive. Low-income staff/visitors and shift workers were particularly impacted by reduced choice and higher prices for healthy options. Conclusions The Policy led to notable changes in the healthiness of foods and drinks available in NZ hospitals but this was accompanied by a perception of reduced value and choice. While generally well supported, the findings indicate opportunities to improve implementation of food and drink policies (e.g. providing more healthy food choices, better engagement with staff, and keeping prices of healthy options low) and confirm that the Policy could be expanded to other public workplaces.

Abstract Background Since 1996, an urban community-based organization whose primary mission is to serve diverse94 and emerging community health needs has provided screening, testing, overdose prevention and training, referrals, and access to treatment for substance use disorders (SUD) and communicable diseases such as HIV through its Life Points harm reduction program. Methods As a partner in a State survey in 2021, the community organization recruited a convenience sample of people who use drugs to participate in a survey focused on their substance use, healthcare, and barriers to SUD services. Community health workers conducted outreach and used an encrypted identifier to collect data from a convenience sample of harm reduction participants regarding demographics, legal justice, engagement in harm reduction and access to healthcare. Evaluators entered paper surveys into Qualtrics for reporting and summative analysis. Results A convenience sample of fifty-five people who use drugs were recruited and surveyed. The majority (86%, n = 47) were active participants in the agency Life Points (LP) harm reduction service. Participants’ average age was 42.9 years (SD = 11.5). About half (51%, n = 28) were male, 48% (n = 26) were female, and 2% (n = 1) was transgender. About two-thirds (67%, n = 37) of participants were White/Caucasian, 13% (n = 7) were Black/African-American, 11% (n = 6) were Hispanic and 7% (n = 4) were Multi-Racial. Regarding current substance use, 98% (n = 54) reported use of heroin, 51% (n = 28) reported crack, 47% (n = 26) cocaine, 25% (n = 14) alcohol, 24% (n = 13) opioids, and 15% (n = 8) marijuana. The majority, 87% (n = 48) said they had health care insurance and over two-thirds (69%, n = 37) said they had been arrested for a felony. Almost three quarters (71%, n = 39) reported receiving services from the Department of Health & Human Services. A higher percentage of females compared to males (65% and 29% respectively) reported engagement in community mental health services and 69% of females (n = 18) compared to 15% (n = 4) of males reported needing to participate in sex to meet basic social needs. Participants described social determinants of health as barriers to services, including access to food, legal justice and transportation. About 44% (n = 24) said they would consider enrolling in a drug treatment program in the next 30 days. Conclusion This sample was reflective of increased participation by White participants that began to appear about a decade ago. The majority of participants reported having healthcare insurance, which may be reflective of engagement with community health workers to access appropriate services. Community organizations and healthcare professionals should continue to explore social determinants of health that can impact the health of people who use drugs, including overcoming barriers to health care access such as investing in mobile unit outreach.

Abstract Background ALL_EARS@UoS is a patient and public involvement and engagement (PPIE) group for people with lived experience of hearing loss. The purpose of the group is to share experiences of hearing loss and hearing healthcare, inform research and improve services for patients at University of Southampton Auditory Implant Service. A year after inception, we wanted to critically reflect on the value and challenges of the group. Four members of ALL_EARS@UoS were recruited to an evaluation steering group. This paper reports the evaluation of the group using the UK Standards for Public Involvement. Methods An anonymous, mixed-methods questionnaire was co-designed and shared with members of ALL_EARS@UoS using an online platform. The questionnaire was designed to capture satisfaction, individual feedback through free-text answers, and demographic information. Descriptive statistics have been used to express the satisfaction and demographic data. Reflexive thematic analysis has been used to analyse the free-text responses. Group engagement and activity data over time were monitored and collected. Results The questionnaire response rate was 61% (11/18). Areas identified as strengths were ‘Communication’ and ‘Working together’. Five themes were developed from the thematic analysis; (1) Increased knowledge and awareness around the topic of hearing health for group members and wider society, (2) supporting research, (3) inclusivity within the group, (4) opportunity to make a difference for people in the future and (5) running of the group/group organisation. The data highlighted the value and challenges of PPIE. Members described feeling listened to and appreciation of being able to share experiences. Time of day and meeting format were identified as challenges as they affected who could attend the meetings. The ability to secure and maintain sufficient funding and time to support inclusive and diverse PPIE activities is a challenge for researchers. Conclusions We have identified how PPIE added value to both group members and researchers, emphasising the true benefit of PPIE. We have highlighted challenges we are facing and our plan to tackle these. We aim to continue to develop and sustain a group that reflects the diversity of the Deaf/deaf or hard of hearing community and of our local community.

Abstract Background By including the needs and perspectives of relevant stakeholders, co-creation is seen as a promising approach for tackling complex public health problems. However, recommendations and guidance on how to plan and implement co-creation are lacking. By identifying and analysing existing implementation and evaluation frameworks for public health, this study aims to offer key recommendations for professional stakeholders and researchers wanting to adopt a co-creation approach to public health interventions. Methods Firstly, PubMed and CINAHL databases were screened for articles introducing original implementation and evaluation frameworks for public health interventions. Backwards snowballing techniques were applied to the included papers. Secondly, identified frameworks were classified and relevant data extracted, including steps and constructs present in the frameworks. Lastly, recommendations were derived by conducting thematic analysis on the included frameworks. Results Thirty frameworks were identified and data related to their nature and scope extracted. The frameworks’ prominent steps and constructs were also retrieved. Recommendations related to implementation and evaluation in the context of co-creation were included. Conclusion When engaging in co-creation, we recommend including implementation considerations from an early stage and suggest adopting a systems thinking as a way to explore multiple levels of influence, contextual settings and systems from an early planning stage. We highlight the importance of partnering with stakeholders and suggest applying an evaluation design that is iterative and cyclical, which pays particular attention to the experience of the engaged co-creators.

Abstract Background This project (named Reinvent) aimed to promote Public Involvement (PI) in health research. Academics worked with a community group, the Eloquent Praise & Empowerment Dance Company, to develop a community partnership with young people from Black African, Asian and Caribbean heritage communities. The goal of this paper is to evaluate the Reinvent project for key learnings on how to engage and build partnerships with young people from Black African, Asian and Caribbean heritage communities. Methods Reinvent developed a steering group which consisted of five young people, one academic, a Race Equality Ambassador and the Director of Eloquent. The steering group co-produced an agenda for two workshops and the evaluation tools used. The content of the workshops included drama exercises, discussions on physical and mental health, nutrition and school-life, short introductions to the concepts of research and PI, and group work to critique and improve a video currently used to promote PI in health research to young people. The evaluation tools included using the ‘Cube’ evaluation framework, video-blogging and collecting anonymous feedback. Findings The responses to the ‘Cube’ evaluation framework were positive across all four domains (agenda, voice, contribute change) in both workshops. A few of the young people described having a better understanding of the meaning and practice of PI in a video-blog. The anonymous feedback suggested that the workshops had increased young people’s confidence in sharing their thoughts and opinions about health and PI. Conclusion Reinvent has shown that academic institutions and young people from an under-served community can partner to co-design workshops and apply evaluation tools. Working with young people in an environment in which they were comfortable, and by researchers joining in with the activities that the young people enjoyed (such as dance), enabled more informal and open conversations to develop. More work is needed to build upon this project so that young people can feel confident and supported to get involved in PI activities relating to research.

Abstract Based on the panel data of 31 provinces (municipalities and autonomous regions) in China from 2012 to 2019, this paper constructs the evaluation index system of basic medical and health services in China from seven dimensions: medical and health facilities, health expenditure, medical services, traditional Chinese medicine hospital services, maternal and child health care, people’s health and medical security, disease control and public health. The entropy method was used to measure the level of basic medical and health services in China, and its spatial differences and convergence characteristics were further investigated. In this study, we employ the entropy weight method, σ convergence, and β convergence as our primary methodologies. The entropy weight method is used to evaluate the variability of each indicator, determine the weights of indicators, and quantify the information content of the data. σ convergence illustrates the process by which the variance of a sample decreases over time. β convergence refers to the gradual approach of variables within an economic system towards their long-term equilibrium level over time. The results show that: (1) The scores of basic medical and health services in China’s four major regions (including Northeast, East, Central and West) remain in a relatively stable state, with small fluctuations and great room for improvement; (2) There are significant regional differences in the level of basic medical and health services in China, and the intra-regional differences are much greater than the inter-regional differences; (3) There is no significant σ convergence observed in China and its four major regions; however, there is a notable presence of β convergence.

Abstract Objectives To evaluate the impact a novel education programme - to improve research engagement, awareness, understanding and confidence - had on a diverse health and social care workforce. Barriers and facilitators to engagement were explored together with research capacity-building opportunities and ways to embed a research culture. The programme is entitled ‘Supporting The Advancement of Research Skills’ (STARS programme); the paper reports findings from a health and social care setting in England, UK. Methods A four-level outcome framework guided the approach to evaluation and was further informed by key principles of research capacity development and relevant theory. Quantitative data were collected from learners before and after engagement; these were analysed descriptively. Semi-structured online interviews were conducted with learners and analysed thematically. A purposive sample was achieved to include a diversity in age, gender, health and social care profession, and level of attendance (regular attendees, moderate attendees and non-attenders). Results The evaluation spanned 18 half-day workshops and 11 seminars delivered by expert educators. 165 (2% of total staff at Midlands Partnership University NHS Foundation Trust (MPFT)) staffs booked one or more education sessions; 128 (77%) including Allied Health Professionals (AHPs), psychologists, nursing and midwifery, and social workers attended one or more session. Key themes of engagement with teaching sessions, relevance and impact of training and promoting a research active environment were identified with relevant sub-themes. Positive impacts of training were described in terms of research confidence, intentions, career planning and application of research skills as a direct result of training. Lack of dedicated time for research engagement, work pressures and time commitments required for the programme were key barriers. Facilitators that facilitated engagement are also described. Conclusions Findings demonstrate the impact that a free, virtual and high-quality research education programme had at individual and organisational levels. The programme is the product of a successful collaboration between health and social care and academic organisations; this provides a useful framework for others to adapt and adopt. Key barriers to attendance and engagement spoke to system-wide challenges that an education programme could not address in the short-term. Potential solutions are discussed in relation to protecting staff time, achieving management buy-in, recognising research champions, and having a clear communication strategy.

Abstract Background Over 180,000 people use crack cocaine in England, yet provision of smoking equipment to support safer crack use is prohibited under UK law. Pipes used for crack cocaine smoking are often homemade and/or in short supply, leading to pipe sharing and injuries from use of unsafe materials. This increases risk of viral infection and respiratory harm among a marginalised underserved population. International evaluations suggest crack pipe supply leads to sustained reductions in pipe sharing and use of homemade equipment; increased health risk awareness; improved service access; reduction in injecting and crack-related health problems. In this paper, we introduce the protocol for the NIHR-funded SIPP (Safe inhalation pipe provision) project and discuss implications for impact. Methods The SIPP study will develop, implement and evaluate a crack smoking equipment and training intervention to be distributed through peer networks and specialist drug services in England. Study components comprise: (1) peer-network capacity building and co-production; (2) a pre- and post-intervention survey at intervention and non-equivalent control sites; (3) a mixed-method process evaluation; and (4) an economic evaluation. Participant eligibility criteria are use of crack within the past 28 days, with a survey sample of ~ 740 for each impact evaluation survey point and ~ 40 for qualitative process evaluation interviews. Our primary outcome measure is pipe sharing within the past 28 days, with secondary outcomes pertaining to use of homemade pipes, service engagement, injecting practice and acute health harms. Anticipated impact SIPP aims to reduce crack use risk practices and associated health harms; including through increasing crack harm reduction awareness among service providers and peers. Implementation has only been possible with local police approvals. Our goal is to generate an evidence base to inform review of the legislation prohibiting crack pipe supply in the UK. This holds potential to transform harm reduction service provision and engagement nationally. Conclusion People who smoke crack cocaine in England currently have little reason to engage with harm reduction and drug services. Little is known about this growing population. This study will provide insight into population characteristics, unmet need and the case for legislative reform. Trial registration : ISRCTN12541454 https://doi.org/10.1186/ISRCTN12541454

Abstract Background A robust evidence base is required to assist healthcare commissioners and providers in selecting effective and sustainable approaches to improve cancer diagnosis and treatment. Such evidence can be difficult to build, given the fast-paced and highly pressured nature of healthcare delivery, the absence of incentives, and the presence of barriers in conducting pragmatic yet robust research evaluations. Cancer Research UK (CRUK) has played an active part in building the evidence base through its funding of programmes to identify, evaluate and scale-up innovative approaches across the UK. The aim of this paper is to describe and explain the research design and intended approach and activities for two cancer services improvement projects in Scotland funded by CRUK. Methods A hybrid effectiveness-implementation study design will assess both the efficiency of the new pathways and their implementation strategies, with the aim of generating knowledge for scale-up. A range of implementation, service and clinical outcomes will be assessed as determined by the projects’ Theories of Change (ToCs). A naturalistic case study approach will enable in-depth exploration of context and process, and the collection and synthesis of data from multiple sources including routine datasets, patient and staff surveys, in-depth interviews and observational and other data. The evaluations are informed throughout by a patient/public representatives’ group, and by small group discussions with volunteer cancer patients. Discussion Our approach has been designed to provide a holistic understanding of how (well) the improvement projects work (in relation to their anticipated outcomes), and how they interact with their wider contexts. The evaluations will help identify barriers, facilitators, and unanticipated consequences that can impact scalability, sustainability and spread. By opting for a pragmatic, participatory evaluation research design, we hope to inform strategies for scaling up successful innovations while addressing challenges in a targeted manner.

Abstract The field of transdisciplinary sustainability research has brought forward a number of approaches aimed at fostering sustainability transformations and generating knowledge through collaborative experimentation in real-world settings. These cases are strongly embedded in their local context and thus the transfer of knowledge remains a key challenge. In this paper, we propose a case reporting approach that supports the structured and coherent reporting of such cases. This scheme is aimed at sustainability-oriented labs, where sustainability solutions are collaboratively developed through experimentation. The scheme focuses the reporting on local contexts, lab processes, and experiments. It is accompanied by a logic model and a set of four principles guiding the reporting procedure. The approach is designed to be general, in that it is applicable to diverse contexts and project designs, while its modularity allows the scheme to be adapted to the needs and specifics of each cases. The scheme was jointly developed and tested by a group of seven Urban Living Labs, each in their own unique context. With our approach we aim to contribute to knowledge transfer from and across cases of sustainability-oriented labs as emerging approaches in action-oriented research bridging the divide of case-based research and (meta) comparison.

Abstract Background The growth of data science and artificial intelligence offers novel healthcare applications and research possibilities. Patients should be able to make informed choices about using healthcare. Therefore, they must be provided with lay information about new technology. A team consisting of academic researchers, health professionals, and public contributors collaboratively co-designed and co-developed the new resource offering that information. In this paper, we evaluate this novel approach to co-production. Methods We used participatory evaluation to understand the co-production process. This consisted of creative approaches and reflexivity over three stages. Firstly, everyone had an opportunity to participate in three online training sessions. The first one focused on the aims of evaluation, the second on photovoice (that included practical training on using photos as metaphors), and the third on being reflective (recognising one’s biases and perspectives during analysis). During the second stage, using photovoice, everyone took photos that symbolised their experiences of being involved in the project. This included a session with a professional photographer. At the last stage, we met in person and, using data collected from photovoice, built the mandala as a representation of a joint experience of the project. This stage was supported by professional artists who summarised the mandala in the illustration. Results The mandala is the artistic presentation of the findings from the evaluation. It is a shared journey between everyone involved. We divided it into six related layers. Starting from inside layers present the following experiences (1) public contributors had space to build confidence in a new topic, (2) relationships between individuals and within the project, (3) working remotely during the COVID-19 pandemic, (4) motivation that influenced people to become involved in this particular piece of work, (5) requirements that co-production needs to be inclusive and accessible to everyone, (6) expectations towards data science and artificial intelligence that researchers should follow to establish public support. Conclusions The participatory evaluation suggests that co-production around data science and artificial intelligence can be a meaningful process that is co-owned by everyone involved.

Abstract Background Recent studies mention a need to investigate partnership roles and dynamics within patient and public involvement and engagement (PPIE) in health research, and how impact and outcomes are achieved. Many labels exist to describe involvement processes, but it is unknown whether the label has implications on partnerships and outcomes. This rapid review investigates how roles between patients, relatives and researchers in a broad variety of PPIE activities in health research are described in peer reviewed papers and explores what enables these partnerships. Methods Rapid review of articles published between 2012 and February 2022 describing, evaluating, or reflecting on experiences of PPIE in health research. All research disciplines and research areas were eligible. Four databases (Medline, Embase, PsychInfo and CINAHL) were searched between November 2021 and February 2022. We followed PRISMA guidelines and extracted descriptive factors: year, origin, research area and discipline, study focus, framework used and co-authorship. On a selection of articles, we performed a narrative analysis of partnership roles using Smits et al.’s. Involvement Matrix. Lastly, we performed a meta synthesis of reported enablers and outcomes of the partnerships. Patients and Relatives (PRs) have been involved in the whole rapid review process and are co-authors of this article. Results Seventy articles from various research disciplines and areas were included. Forty articles were selected for a narrative analysis of the role description of PRs and researchers, and a meta synthesis of enablers and outcomes. Most articles described researchers as decision-makers throughout the research cycle. PRs most often were partners when they were included as co-authors; they were mostly partners in the design, analysis, write-up, and dissemination stages. Enablers of partnerships included: PR training, personality of PRs and communication skills, trust, remuneration and time. Conclusions Researchers’ decision-making roles gives them control of where and when to include PRs in their projects. Co-authorship is a way of acknowledging patients’ contributions which may lead to legitimation of their knowledge and the partnership. Authors describe common enablers, which can help future partnership formation.

Abstract Background In 2020, the Health Resources and Services Administration’s HIV/AIDS Bureau funded an initiative to promote implementation of rapid antiretroviral therapy initiation in 14 HIV treatment settings across the U.S. The goal of this initiative is to accelerate uptake of this evidence-based strategy and provide an implementation blueprint for other HIV care settings to reduce the time from HIV diagnosis to entry into care, for re-engagement in care for those out of care, initiation of treatment, and viral suppression. As part of the effort, an evaluation and technical assistance provider (ETAP) was funded to study implementation of the model in the 14 implementation sites. Method The ETAP has used implementation science methods framed by the Dynamic Capabilities Model integrated with the Conceptual Model of Implementation Research to develop a Hybrid Type II, multi-site mixed-methods evaluation, described in this paper. The results of the evaluation will describe strategies associated with uptake, implementation outcomes, and HIV-related health outcomes for patients. Discussion This approach will allow us to understand in detail the processes that sites to implement and integrate rapid initiation of antiretroviral therapy as standard of care as a means of achieving equity in HIV care.

Abstract Background Clinical pathways are one of the main tools to manage the health care’s quality and concerned with the standardization of care processes. They have been used to help frontline healthcare workers by presenting summarized evidence and generating clinical workflows involving a series of tasks performed by various people within and between work environments to deliver care. Integrating clinical pathways into Clinical Decision Support Systems (CDSSs) is a common practice today. However, in a low-resource setting (LRS), this kind of decision support systems is often not readily accessible or even not available. To fill this gap, we developed a computer aided CDSS that swiftly identifies which cases require a referral and which ones may be managed locally. The computer aided CDSS is designed primarily for use in primary care settings for maternal and childcare services, namely for pregnant patients, antenatal and postnatal care. The purpose of this paper is to assess the user acceptance of the computer aided CDSS at the point of care in LRSs. Methods For evaluation, we used a total of 22 parameters structured in to six major categories, namely “ease of use, system quality, information quality, decision changes, process changes, and user acceptance.” Based on these parameters, the caregivers from Jimma Health Center's Maternal and Child Health Service Unit evaluated the acceptability of a computer aided CDSS. The respondents were asked to express their level of agreement using 22 parameters in a think-aloud approach. The evaluation was conducted in the caregiver's spare-time after the clinical decision. It was based on eighteen cases over the course of two days. The respondents were then asked to score their level of agreement with some statements on a five-point scale: strongly disagree, disagree, neutral, agree, and strongly agree. Results The CDSS received a favorable agreement score in all six categories by obtaining primarily strongly agree and agree responses. In contrast, a follow-up interview revealed a variety of reasons for disagreement based on the neutral, disagree, and strongly disagree responses. Conclusions Though the study had a positive outcome, it was limited to the Jimma Health Center Maternal and Childcare Unit, and hence a wider scale evaluation and longitudinal measurements, including computer aided CDSS usage frequency, speed of operation and impact on intervention time are needed.

Abstract Background Young people with a sibling or parent who experiences mental health challenges have their own support needs. Most programs designed for this population lack a strong evidence base, and the involvement of young people in the development and evaluation of programs designed to support them is unclear or lacking. Methods This paper describes a protocol for a mixed methods, longitudinal, collaborative evaluation of a suite of programs delivered by The Satellite Foundation, a not-for-profit organisation for young people (5–25 years) who have a family member with mental health challenges. Young people’s lived experience and knowledge will guide the research approach. Institutional ethics approval has been obtained. Over a three year period, approximately 150 young people will be surveyed online on various wellbeing outcome measures, prior to, six and twelve months following program participation with data analysed using multi-level modelling. Groups of young people will be interviewed after participating in different Satellite programs each year. An additional group of young people will be interviewed individually over time. Transcripts will be analysed using thematic analysis. Young people’s creative artworks on their experiences will be included as part of the evaluation data. Discussion This novel, collaborative evaluation will provide vital evidence on young people’s experiences and outcomes during their time with Satellite. Findings will be used to inform future program development and policy. The approach used here may guide other researchers engaging in collaborative evaluations with community organisations.

Abstract Background The COVID-19 pandemic changed the way we work, spend, live, and learn. The impact was felt in the health sector where hospitals cancelled elective surgery, put on hold outpatient services, and implemented new social distancing procedures and telehealth systems, to enable hospitals to increase bed capacity. For medical students, these factors meant significant disruption to their clinical placements, remote delivery of their education, cessation of international and interstate placements, complicated by significant travel restrictions and border closures. There were concerns that final year students might be unable to graduate that year due to this lack of clinical exposure. Innovation As a result of this disruption in late March 2020 we developed an innovative 6 week ‘COVID-19 e-lective’ rotation, consisting of online modules, virtual clinical tutorials and a COVID project totalling the equivalent of 200 h of work. Results An evaluation was undertaken that found it to be remarkably successful in meeting the students’ learning needs and alleviating concerns about disrupted placements. It was also conducted during 2021 for all Year 4 students to help expand clinical placement opportunities. Outcomes This paper describes the e-lective, its innovations, its challenges, and its evaluation findings, for others to learn from.

Abstract Background Peer-research is steered and conducted by people with lived experience of the issues being researched. This paper explores the value of peer-research in two complex public health intervention evaluations in the UK. Methods Reports from 18 peer research projects, completed by residents from 12 communities in the UK taking part in two community empowerment interventions, were analysed using cross-case analysis. Results Undertaking peer research helped to build the evaluation and research skills within individual projects as well as providing data on other outcomes related to the programmes Theory of Change. Some peer researchers, however, felt unprepared for the activity despite support from the academic team and were unsatisfied with project outcomes. While peer research projects provided more opportunities for local residents to engage with the overall evaluations, there was an overreliance on people closely connected to the programmes to be peer researchers. The peer research projects explored topics that were broader than the aims and objectives of the overall programme evaluations. All provided insight into the context in which projects occurred, while some also informed understanding of programme change mechanisms. Conclusions Including peer research as part of complex public health intervention evaluations can help uncover important contextual and ecological details beyond the reach of more traditional evaluation data collection. Peer research can also empower and build research/evaluation capacity within communities, which is particularly pertinent for community empowerment interventions.

Abstract Background This paper explores the feasibility of delivering a music festival-based drug checking service in Australia, evaluating service design and stakeholder acceptability. Methods Questionnaire and interview data were collected from adult service users and key stakeholders. A mixed methods approach was used to analyse the data on implementation, impact and acceptability. Results The trial service tested 170 substances with more than 230 patrons (including individuals who attended in groups). Adult service users had an average age of 21 years. Voluntary participation in the evaluation resulted in 158 participants completing the pre-service questionnaire, most of whom also completed the post-service (147 participants). Eleven in-depth qualitative interviews were conducted with patrons in the weeks following the drug checking. Concordance between what the patron expected the drug to be and drug checking results occurred in 88 per cent (n = 139) of the sample. Evaluation results show that the experience of testing and the accompanying harm reduction brief interventions positively impacted on patrons’ self-reported drug harm reduction knowledge, trust of health providers and stated drug use intentions. The service was received positively by service users. Conclusion This is the first independent evaluation of a pilot drug checking service in Australia. Consideration of operational feasibility and self-reported behavioural change suggests that the program was successful, although communication about the interpretation of drug checking results could be improved. Future studies should develop strategies for follow-up and consider the applicability of behavioural change theory.

Abstract Psychometrically sound resilience outcome measures are essential to establish how health and care services or interventions can enhance the resilience of people living with dementia (PLWD) and their carers. This paper systematically reviews the literature to identify studies that administered a resilience measurement scale with PLWD and/or their carers and examines the psychometric properties of these measures. Electronic abstract databases and the internet were searched, and an international network contacted to identify peer-reviewed journal articles. Two authors independently extracted data. They critically reviewed the measurement properties from the available psychometric data in the studies, using a standardised checklist adapted for purpose. Fifty-one studies were included in the final review, which applied nine different resilience measures, eight developed in other populations and one developed for dementia carers in Thailand. None of the measures were developed for use with people living with dementia. The majority of studies (N = 47) focussed on dementia carers, three studies focussed on people living with dementia and one study measured both carers and the person with dementia. All the studies had missing information regarding the psychometric properties of the measures as applied in these two populations. Nineteen studies presented internal consistency data, suggesting seven of the nine measures demonstrate acceptable reliability in these new populations. There was some evidence of construct validity, and twenty-eight studies hypothesised effects a priori (associations with other outcome measure/demographic data/differences in scores between relevant groups) which were partially supported. The other studies were either exploratory or did not specify hypotheses. This limited evidence does not necessarily mean the resilience measure is not suitable, and we encourage future users of resilience measures in these populations to report information to advance knowledge and inform further reviews. All the measures require further psychometric evaluation in both these populations. The conceptual adequacy of the measures as applied in these new populations was questionable. Further research to understand the experience of resilience for people living with dementia and carers could establish the extent current measures -which tend to measure personal strengths -are relevant and comprehensive, or whether further work is required to establish a new resilience outcome measure.

Abstract Background and aim Recent UK policy has focussed on improving support for victims of domestic violence and abuse (DVA), in healthcare settings. DVA victims attending hospital are often at highest risk of harm, yet DVA support in hospitals has been inadequate. A targeted service supporting high risk DVA victims, was implemented at a hospital Trust in North West England. The service was provided by Independent Domestic Violence Advisors (IDVAs). This paper assesses the activity in the hospital-based IDVA service during the COVID-19 pandemicand addresses the research questions: What was the demand for the service? How did the service respond? What facilitated this response? Methods A mixed-methods study was undertaken. Quantitative data on referrals to the service were examined using simple descriptive statistics and compared to other DVA services. Semi-structured interviews were undertaken with IDVAs and other hospital staff involved with the service and the data subjected to thematic analysis. Results The quantitative analysis showed that referrals dropped at the start of lockdown, then increased and continued to rise; the qualitative findings reiterated this pattern. Referrals came from a range of departments across the Trust, with the majority from A&E. Pre-pandemic, the population supported by the service included higher proportions of males and people aged 40 and over than at other IDVA services; this continued during the pandemic. The qualitative findings indicated a flexible response during the pandemic, enabled by strong working relationships and by using workarounds. Conclusions The hospital-based IDVAs provided an efficient, flexible serviceduring the COVID-19 pandemic. Referrals increased during the first lockdown and subsequent relaxing of restrictions. Locating the IDVAs within a team working across the organisation, and building good working relationships facilitated an effective disclosure and referral route, which endured through social restrictions. The IDVAs supported high-risk victims who may otherwise not have been identified in traditional community-based DVA settings during the pandemic. Hospital-based IDVA services can broaden access by supporting vulnerable, at risk populations whose needs may not be identified at other services.

Abstract Background Given the magnitude and frequency of conflicts in the MENA region along with their devastating impact on health responses and outcomes, there exists a strong need to invest in contextualized, innovative, and accessible capacity building approaches to enhance leadership and skills in global health. The MENA region suffers from limited (1) continued educational and career progression opportunities, (2) gender balance, and (3) skill-mix among its health workforce, which require significant attention. Main text The Global Health Institute at the American University of Beirut incepted the Academy division to develop and implement various global health capacity building (GHCB) initiatives to address those challenges in fragile settings across low-and middle-income countries in the MENA region. These initiatives play a strategic role in this context, especially given their focus on being accessible through employing innovative learning modalities. However, there exists a dearth of evidence-based knowledge on best practices and recommendations to optimize the design, implementation, and evaluation of GHCB in fragile settings in the MENA region. The present paper describes the development of the evaluation of capacity building program (eCAP), implemented under the Academy division, to assess the effectiveness of its initiatives. eCAP is composed of 3 phases: (1) a situational assessment, followed by (2) production of multiple case studies, and finally (3) a meta-assessment leading to model development. The goal of eCAP is not only to inform the Academy’s operations, but also to synthesize produced knowledge into the formation of an evidence-based, scalable, and replicable model for GHCB in fragile settings. Conclusion eCAP is an important initiative for researchers, educators, and practitioners interested in GHCB in fragile settings. Several lessons can be learned from the outcomes it has yielded so far in its first two phases of implementation, ranging from the situational assessment to the production of evaluation case studies, which are expanded on in the manuscript along with pertinent challenges.

Abstract Background Latin America and the Caribbean (LAC) are among those regions most affected by the COVID-19 pandemic worldwide. The COVID-19 pandemic has strained health systems in the region. In this context of severe healthcare resource constraints, there is a need for systematic priority-setting to support decision-making which ensures the best use of resources while considering the needs of the most vulnerable groups. The aim of this paper was to provide a critical description and analysis of how health systems considered priority-setting in the COVID-19 response and preparedness plans of a sample of 14 LAC countries; and to identify the associated research gaps. Methods A documentary analysis of COVID-19 preparedness and response plans was performed in a sample of 14 countries in the LAC region. We assessed the degree to which the documented priority-setting processes adhered to established quality indicators of effective priority-setting included in the Kapiriri and Martin framework. We conducted a descriptive analysis of the degree to which the reports addressed the quality parameters for each individual country, as well as a cross-country comparison to explore whether parameters varied according to independent variables. Results While all plans were led and supported by the national governments, most included only a limited number of quality indicators for effective priority-setting. There was no systematic pattern between the number of quality indicators and the country’s health system and political contexts; however, the countries that had the least number of quality indicators tended to be economically disadvantaged. Conclusion This study adds to the literature by providing the first descriptive analysis of the inclusion of priority-setting during a pandemic, using the case of COVID-19 response and preparedness plans in the LAC region. The analysis found that despite the strong evidence of political will and stakeholder participation, none of the plans presented a clear priority-setting process, or used a formal priority-setting framework, to define interventions, populations, geographical regions, healthcare setting or resources prioritized. There is need for case studies that analyse how priority-setting actually occurred during the COVID-19 pandemic and the degree to which the implementation reflected the plans and the parameters of effective priority-setting, as well as the impact of the prioritization processes on population health, with a focus on the most vulnerable groups.

Abstract Background There is increasing awareness among researchers and policymakers of the potential for healthcare interventions to have consequences beyond those initially intended. These unintended consequences or “spillover effects” result from the complex features of healthcare organisation and delivery and can either increase or decrease overall effectiveness. Their potential influence has important consequences for the design and evaluation of implementation strategies and for decision-making. However, consideration of spillovers remains partial and unsystematic. We develop a comprehensive framework for the identification and measurement of spillover effects resulting from changes to the way in which healthcare services are organised and delivered. Methods We conducted a scoping review to map the existing literature on spillover effects in health and healthcare interventions and used the findings of this review to develop a comprehensive framework to identify and measure spillover effects. Results The scoping review identified a wide range of different spillover effects, either experienced by agents not intentionally targeted by an intervention or representing unintended effects for targeted agents. Our scoping review revealed that spillover effects tend to be discussed in papers only when they are found to be statistically significant or might account for unexpected findings, rather than as a pre-specified feature of evaluation studies. This hinders the ability to assess all potential implications of a given policy or intervention. We propose a taxonomy of spillover effects, classified based on the outcome and the unit experiencing the effect: within-unit, between-unit, and diagonal spillover effects. We then present the INTENTS framework: Intended Non-intended TargEted Non-Targeted Spillovers. The INTENTS framework considers the units and outcomes which may be affected by an intervention and the mechanisms by which spillover effects are generated. Conclusions The INTENTS framework provides a structured guide for researchers and policymakers when considering the potential effects that implementation strategies may generate, and the steps to take when designing and evaluating such interventions. Application of the INTENTS framework will enable spillover effects to be addressed appropriately in future evaluations and decision-making, ensuring that the full range of costs and benefits of interventions are correctly identified.

Abstract Background Systems approaches are currently being advocated and implemented to address complex challenges in Public Health. These approaches work by bringing multi-sectoral stakeholders together to develop a collective understanding of the system, and then to identify places where they can leverage change across the system. Systems approaches are unpredictable, where cause-and-effect cannot always be disentangled, and unintended consequences – positive and negative – frequently arise. Evaluating such approaches is difficult and new methods are warranted. Methods Ripple Effects Mapping (REM) is a qualitative method which can capture the wider impacts, and adaptive nature, of a systems approach. Using a case study example from the evaluation of a physical activity-orientated systems approach in Gloucestershire, we: a) introduce the adapted REM method; b) describe how REM was applied in the example; c) explain how REM outputs were analysed; d) provide examples of how REM outputs were used; and e) describe the strengths, limitations, and future uses of REM based on our reflections. Results Ripple Effects Mapping is a participatory method that requires the active input of programme stakeholders in data gathering workshops. It produces visual outputs (i.e., maps) of the programme activities and impacts, which are mapped along a timeline to understand the temporal dimension of systems change efforts. The REM outputs from our example were created over several iterations, with data collected every 3–4 months, to build a picture of activities and impacts that have continued or ceased. Workshops took place both in person and online. An inductive content analysis was undertaken to describe and quantify the patterns within the REM outputs. Detailed guidance related to the preparation, delivery, and analysis of REM are included in this paper. Conclusion REM may help to advance our understanding and evaluation of complex systems approaches, especially within the field of Public Health. We therefore invite other researchers, practitioners and policymakers to use REM and continuously evolve the method to enhance its application and practical utility.

Abstract Background Systems thinking embraces the complexity of public health problems, including childhood overweight and obesity. It aids in understanding how factors are interrelated, and it can be targeted to produce favourable changes in a system. There is a growing call for systems approaches in public health research, yet limited practical guidance is available on how to evaluate public health programmes within complex adaptive systems. The aim of this paper is to present an evaluation framework that supports researchers in designing systems evaluations in a comprehensive and practical way. Methods We searched the literature for existing public health systems evaluation studies. Key characteristics on how to conduct a systems evaluation were extracted and compared across studies. Next, we overlaid the identified characteristics to the context of the Lifestyle Innovations Based on Youth Knowledge and Experience (LIKE) programme evaluation and analyzed which characteristics were essential to carry out the LIKE evaluation. This resulted in the Evaluation of Programmes in Complex Adaptive Systems (ENCOMPASS) framework. Results The ENCOMPASS framework includes five iterative stages: (1) adopting a system dynamics perspective on the overall evaluation design; (2) defining the system boundaries; (3) understanding the pre-existing system to inform system changes; (4) monitoring dynamic programme output at different system levels; and (5) measuring programme outcome and impact in terms of system changes. Conclusions The value of ENCOMPASS lies in the integration of key characteristics from existing systems evaluation studies, as well as in its practical, applied focus. It can be employed in evaluating public health programmes in complex adaptive systems. Furthermore, ENCOMPASS provides guidance for the entire evaluation process, all the way from understanding the system to developing actions to change it and to measuring system changes. By the nature of systems thinking, the ENCOMPASS framework will likely evolve further over time, as the field expands with more completed studies.

Abstract Background Emergency obstetric care training, using Advances in Labour and Risk Management (ALARM) International Program (AIP) was implemented in Ukraine, a country with universal access to skilled perinatal and obstetric care but restricted resources. A total of 577 providers (65.5% of total) from 28 maternal clinics attended a 5-day training session focused on the five main causes of maternal mortality, with hands-on skill workshops, pre- and post- tests, and an objective structured clinical examination. The effects of this emergency obstetric care training on maternal outcomes is the subject of this paper. Methods A non-randomized controlled trial was conducted. The pilot areas where the training was implemented consisted of 64 maternity clinics of which 28 were considered as cases and 36 non-participating clinics were the referents. Data on maternal outcomes were collected for a 2-year span (2004-2005) prior to the trainings, which took place 2006-2007 and again after implementation of the trainings, from 2008 to 2009. Information was collected from 189,852 deliveries. Outcomes for the study were incidences of operative delivery and postpartum hemorrhage. Non-parametric statistics, meta-analyses, and difference in difference (DID) estimation were used to assess the effect of the AIP on maternal indices. Results DID analysis showed that after the training, compared to the referents, the cases had significant reduction of blood transfusions (OR: 0.56; 95%CI: 0.48-0.65), plasma transfusions (OR: 0.70; 95%CI: 0.63-0.78), and uterus explorations (OR: 0.64; 95%CI: 0.59-0.69). We observed a non-significant reduction of postpartum hemorrhage ≥1000 ml (OR: 0.92; 95%CI: 0.81-1.04; P = 0.103). Utilization of vacuum extraction for vaginal delivery increased (OR: 2.86; 95%CI: 1.80-4.57), as well as forceps assisted delivery (OR: 1.80; 95%CI: 1.00-3.25) and cesarean section (OR: 1.11; 95%CI: 1.06-1.17). There was no change in the occurrence of postpartum hysterectomy and maternal mortality. Conclusions After one week of Emergency Obstetrics Care training of the obstetric staff in a setting with universal access to perinatal and obstetric care but restricted resources, an association with the reduction of postpartum hemorrhage related interventions was observed. The effects on the use of vacuum extraction and cesarean section were minimal. Trial registration Retrospectively registered 071212007807 from 07/12/2012.