Search

Introduction To help students think critically about health information and decisions, we developed the Informed Health Choices (IHC) secondary school intervention. We are evaluating the intervention in cluster-randomised trials, and linked process evaluations, in Kenya, Rwanda, and Uganda. The study planned herein is a qualitative evidence synthesis (QES), using data about experiences and views of adverse effects from all three process evaluations. The QES overlaps with the process evaluations. The QES will allow us to comprehensively explore, report, and discuss experiences and views of potential adverse effects, and potential mechanisms. The findings are intended to help inform decisions about whether or how to redevelop, re-evaluate, or implement the intervention. The methods and findings might also be helpful to developing, evaluating, or implementing other educational interventions—especially interventions intended to improve critical thinking, within health or other fields. Objective Explore participants’ and investigators’ experiences and views of potential adverse effects of the IHC secondary school intervention, and potential mechanisms of those effects. Methods An independent researcher will assess methodological limitations of the included studies, based on a list of domains used by the Cochrane Effective Practice and Organisation of Care (EPOC) group. We will merge and modify framework analysis as outlined by Ritchie and Spencer, framework synthesis as described by Barnett-Page et al., and “best fit” framework synthesis as outlined by Carrol et al. All three are pragmatic approaches with a deductive analysis using an a priori framework, followed by an inductive thematic analysis. We will note possible differences in how participants generally conceptualise adverse effects of educational interventions, compared to the study team, and possible differences in adverse effects across trial settings, for the purposes of future research. To assess confidence in the synthesis findings, we will apply the Grading of Recommendations Assessment, Development and Evaluation Confidence in Evidence from Reviews of Qualitative research (GRADE-CERQual) approach, using the interactive Summary of Qualitative Findings (iSoQ) tool. We will produce a Qualitative Evidence Profile, and Summary of Qualitative Findings Tables. Discussion The QES and the process evaluations overlapping and being part of the same project has methodological implications that amount to both strengths and limitations. Like in prospective meta-analyses, when planning the process evaluations, we harmonised the objectives, facilitating synthesis, while investigators in each setting still had autonomy to explore phenomena specifically for their study. Like individual patient data (IDP) meta-analyses, this study will be based on data from the process evaluations, facilitating more reliable analysis and synthesis than if it was only based on reported findings. As far as we are aware, this QES will be the first of its kind methodologically, and first empirical study of its size and rigour focusing on potential adverse effects of an educational intervention. The QES and process evaluations overlapping and being part of the same project also introduces risk of bias. Another challenge is that we are including the study team’s experiences and views. In general, to address these challenges, we will be transparent, and apply reflexivity throughout.

Additional file 1.

The intervention Complexity Assessment Tool for Systematic Reviews (iCAT_SR) has been developed to facilitate detailed assessments of intervention complexity in systematic reviews.2 This study aimed to assess, using the iCAT_SR, the complexity of interventions aimed at improving appropriate polypharmacy in older people, as reported in studies (n = 20) included in the update of a Cochrane review. Interventions were assessed independently by two authors using the six core iCAT_SR domains: (1) ‘Active intervention components’; (2) ‘Target behaviour/actions’; (3) ‘Target organisational levels/categories’; (4) ‘Degree of tailoring’; (5) ‘Level of skill required by intervention deliverers’; (6) ‘Level of skill required by intervention recipients’.2 Attempts were made to apply the tool’s four optional domains: ‘Interaction between intervention components’; ‘Context/setting’; ‘Recipient/provider factors’; ‘Nature of causal pathway’. Disagreements were resolved by consensus discussion. The findings demonstrate that interventions in this area often comprise multiple components and target behaviours. Intermediate to high level skills were typically required by those delivering and receiving the interventions. A lack of detailed reporting in study reports precluded consistent application of the iCAT_SR’s optional domains. This study provides the first detailed assessment of complexity for interventions aimed at improving appropriate polypharmacy in older people. Further work is needed to determine if application of the iCAT_SR can help to determine whether varying levels of complexity impact on the effectiveness of interventions aimed at improving appropriate polypharmacy in older people.

Background We designed the “Be smart about your health” digital resources to teach lower secondary school students how to assess the trustworthiness of claims about the effects of treatments. We shall evaluate effects of using these resources in a randomised trial in Uganda. This paper describes the process evaluation that will be conducted alongside this trial. The aim is to identify factors affecting the implementation, fidelity, effects, and scaling up use of the Be smart about your health teaching resources in Uganda and potential adverse and beneficial effects of the intervention. Methods Forty teachers from 40 schools in the intervention arm of the trial will complete a lesson evaluation questionnaire after each of the 10 lessons and at the end of the term. We will conduct structured classroom observations at all 40 schools. We will purposively select eight schools where we will conduct focus group discussions with teachers and students. We also will conduct key informant interviews among education officers (that visit the schools to monitor the implementation), teachers, head teachers, learners, and their parents. We will use a framework analysis approach to analyse the data. Expected results We anticipate that the findings from this evaluation will provide insights into factors that may impact the effectiveness of using the Be smart about your health resources, factors that can inhibit or facilitate scaling up use of the resources, and potential effects that were not measured quantitatively after the intervention, including transfer of what was learned to other contexts and adverse effects.

Background: People frequently make decisions about what can improve their health. They do this based on large amounts of information that they get from different sources, which include claims about what harms or improves health. People therefore need skills to assess the trustworthiness of health claims. The Informed Health Choices (IHC) Key Concepts framework was used to develop the ‘Be Smart about your Health’ secondary school resources to help students in lower secondary to assess health claims about the effects of treatments. We are evaluating these resources in a cluster randomized trial in 80 lower secondary schools in Kenya. This protocol is for a process evaluation that will be conducted alongside the trial. The objective of this evaluation is to explore the extent to which the intervention was delivered as planned; understand factors that facilitated or hindered the delivery and impact of the resources; and antici-pated and unanticipated effects. Study design: We will employ a mixed-methods design using quantitative and qualita-tive data. We will collect quantitative data from all schools (n=40) allocated to the in-tervention arm using lesson evaluation forms. Our qualitative data collection will in-clude: (a) structured classroom observations in all schools (n=40), with at least one lesson observed in each school. In a sample of eight schools, we shall observe more than one lesson. (b) We will conduct focus group discussions (with students (n=4), teachers (n=1) and parents (n=4). (c) Key informant interviews with policymakers in education (n=5), teachers (n=8), and with school principals (n=8)). We will purposively select these schools based on location (urban and rural) and ownership (private and public). Data analysis: We will use framework analysis to analyze qualitative data and descrip-tive analysis to analyze quantitative data. We will summarize and appraise the confi-dence of the key findings from the qualitative data using a modified version of the GRADE-CERQual approach.

As noted in the protocol, we planned to assess potential adverse effects and use of what was learned in daily life (far transfer) in the one-year follow-up studies. This addendum to the protocol describes how we will do this.

Background: Many people find it difficult to make decisions about what to believe or do regarding “health actions” (things that they can do to care for their health or the health of others). To avoid being misled by unreliable information and to make good choices, people must be able to understand and apply some basic principles (concepts). The Informed Health Choices (IHC) secondary school resources are designed to help students learn and use nine of these concepts. This is a protocol for a prospective meta-analysis of three cluster-randomized trials in Kenya, Rwanda, and Uganda. Objective: To estimate the effect of using the IHC secondary school resources in addition to the standard curriculum compared to the standard curriculum in the context where the trial is conducted) on students’ ability to think critically about health actions. Methods: The participants are lower secondary school students (age 14-16) and their teachers. The intervention is a 2-3-day teacher training workshop, a teachers’ guide, and 10 40-minute lessons taught to the students during a single school term. The comparison is the standard curriculum in each country. The primary outcome measure is the Critical Thinking about Health (CTH) Test. The test is a measure of an individual’s ability to understand and apply the nine key critical thinking concepts included in the lessons. It includes two multiple-choice questions for each concept. The primary outcome is the proportion of students with a passing score on the test. Secondary outcomes include the average score on the test, the proportion of students with a score that indicates mastery of the nine concepts, retention of what the students learned after one year, self-efficacy, intended behaviours, performance on standard, end-of-term examinations, and teachers’ ability to understand and apply the nine concepts measured using the CTH Test. Overall effects will be estimated using inverse variance methods. We will use mixed effects models with individual student data to estimate variation in effects for the primary and first two secondary outcomes in relation to three potential effect modifiers: use of a projector version compared to a blackboard version of the lessons, class size, and student performance on end-of-term examinations prior to the intervention. Strengths and limitations: Strengths of this meta-analysis are that it is prospective, allowing for collaboration on the design of the trials and certainty that all eligible trials have been included. A limitation is that the review authors are both developing and evaluating the intervention, although there are efforts to minimise bias.

Background: Young people are exposed to many claims about the effects of things they can do to protect or improve health. To make good health choices, they need the ability to assess the reliability of those claims critically. Critical thinking is a core competence in the Rwandan secondary school curriculum and in many countries. However, critical thinking about health is rarely taught in Rwanda. Digital secondary school resources for critical thinking about health can potentially address this gap and if published be widely disseminated at low cost. The objective of the planned study is to evaluate the effects of using digital secondary school resources to help students learn to think critically about health. Methods: We will conduct a two-arm cluster-randomized trial. We will randomly select 84 lower secondary schools from 10 districts representing all five provinces of Rwanda. Using stratified random allocation, we will assign 84 schools to the intervention or control arm. Schools in the intervention arm will teach 10-lessons after teacher training on the content of the secondary school resources. Schools in the control arm will not receive any training and resources. They will carry on with teaching the national curriculum. The primary outcome will be the proportion of students with a predetermined passing score on a test with multiple-choice questions from the Claim Evaluation Tools item bank. The questions measure the ability to apply key concepts related to assessing health claims and making informed health choices. The test will include two questions addressing each of the nine concepts covered by the secondary school resources. Discussion: This is one of three randomized trials to assess the effects of the Informed Health Choices secondary school resources in Rwanda, Kenya, and Uganda. The findings will inform decisions about how to promote critical thinking about health. Trial registration: Pan African Clinical Trial Registry, trial identifier: PACTR202203880375077, Registered on 15th February 2022.

Background: There is an overabundance of claims about the advantages and disadvantages of health interventions. Many people are unable to assess the reliability of these claims. Acting on unreliable health claims or failing to act on reliable advice can lead to waste of resources and unnecessary suffering. There is a global focus on teaching critical thinking, including in the new Kenyan curriculum, however, critical thinking about health is not taught in Kenyan schools. The study planned for in this protocol is an evaluation of the effects of the Informed Health Choices secondary school intervention on the ability of lower secondary school students in Kenya to think critically about health information and choices. The intervention consists of providing teachers with digital teaching resources and training them to deliver the intervention to students. The intervention covers 10 lessons that address nine key concepts. Methods: This study is a cluster-randomized trial. We will stratify lower secondary schools by ownership and geographical location, and randomly select 80 of them. We will randomly allocate the schools to either the intervention or control group. The intervention schools will teach the 10 lessons in one academic term alongside the Kenya national secondary school curriculum. The control schools will continue teaching the national secondary school curriculum. The primary outcome measure will be a test with multiple-choice questions from the Claim Evaluations Tools item bank. The test will include two items for each of the nine concepts covered by the teaching resources. The primary outcome will be the proportion of students attaining a predetermined passing score. Trial registration: Pan African Clinical Trial Registry, trial identifier: PACTR202204883917313. Registered on 05/04/2022.

Background To make well-informed choices, people must possess skills to assess the trustworthiness of health-related claims. It is important that young people learn to assess the reliability of claims to inform decisions, both when making their own choices and as citizens participating in a democracy. This trial aims to evaluate the effect of the Informed Health Choices (IHC) teaching resources on secondary school students’ ability to assess health-related claims and make informed choices. Methods This will be a two-arm cluster-randomised trial. We will randomise 80 lower secondary schools to evaluate the IHC secondary school digital teaching resources. The resources for teachers include 10 lessons to be delivered in a single school term, using lesson plans developed for classrooms equipped with only a blackboard or with a projector. Teachers in the intervention arm will be invited to a three-day teacher training workshop. Teachers in the control schools will continue teaching the national curriculum. Uganda’s National Curriculum Development Centre introduced a new competence-based curriculum in 2020. This curriculum has critical thinking as one of seven generic skills to be taught across all subjects. The curriculum does not explicitly include critical thinking about health. The IHC lessons address nine prioritised key concepts. We will use multiple choice questions – two for each key concept - to evaluate the student’s ability to assess claims and make informed choices. We will measure the proportion of students with a passing score at the end of the school term, and again after one year to assess retention of what was learned. Expected results Based on previous work done in Ugandan primary schools, we anticipate that the use of the teaching resources will lead to a large improvement in the lower secondary school students’ ability to assess claims and make informed health choices.

Additional file 2. Intervention complexity assessments based on the iCAT_SR tool.

Additional file 1. Interview guides.

Additional file 1. Data extraction form.

Additional file 3. ICT observation checklist.

Additional file 5. COREQ checklist.