Search

Introduction To help students think critically about health information and decisions, we developed the Informed Health Choices (IHC) secondary school intervention. We are evaluating the intervention in cluster-randomised trials, and linked process evaluations, in Kenya, Rwanda, and Uganda. The study planned herein is a qualitative evidence synthesis (QES), using data about experiences and views of adverse effects from all three process evaluations. The QES overlaps with the process evaluations. The QES will allow us to comprehensively explore, report, and discuss experiences and views of potential adverse effects, and potential mechanisms. The findings are intended to help inform decisions about whether or how to redevelop, re-evaluate, or implement the intervention. The methods and findings might also be helpful to developing, evaluating, or implementing other educational interventions—especially interventions intended to improve critical thinking, within health or other fields. Objective Explore participants’ and investigators’ experiences and views of potential adverse effects of the IHC secondary school intervention, and potential mechanisms of those effects. Methods An independent researcher will assess methodological limitations of the included studies, based on a list of domains used by the Cochrane Effective Practice and Organisation of Care (EPOC) group. We will merge and modify framework analysis as outlined by Ritchie and Spencer, framework synthesis as described by Barnett-Page et al., and “best fit” framework synthesis as outlined by Carrol et al. All three are pragmatic approaches with a deductive analysis using an a priori framework, followed by an inductive thematic analysis. We will note possible differences in how participants generally conceptualise adverse effects of educational interventions, compared to the study team, and possible differences in adverse effects across trial settings, for the purposes of future research. To assess confidence in the synthesis findings, we will apply the Grading of Recommendations Assessment, Development and Evaluation Confidence in Evidence from Reviews of Qualitative research (GRADE-CERQual) approach, using the interactive Summary of Qualitative Findings (iSoQ) tool. We will produce a Qualitative Evidence Profile, and Summary of Qualitative Findings Tables. Discussion The QES and the process evaluations overlapping and being part of the same project has methodological implications that amount to both strengths and limitations. Like in prospective meta-analyses, when planning the process evaluations, we harmonised the objectives, facilitating synthesis, while investigators in each setting still had autonomy to explore phenomena specifically for their study. Like individual patient data (IDP) meta-analyses, this study will be based on data from the process evaluations, facilitating more reliable analysis and synthesis than if it was only based on reported findings. As far as we are aware, this QES will be the first of its kind methodologically, and first empirical study of its size and rigour focusing on potential adverse effects of an educational intervention. The QES and process evaluations overlapping and being part of the same project also introduces risk of bias. Another challenge is that we are including the study team’s experiences and views. In general, to address these challenges, we will be transparent, and apply reflexivity throughout.

Background We designed the “Be smart about your health” digital resources to teach lower secondary school students how to assess the trustworthiness of claims about the effects of treatments. We shall evaluate effects of using these resources in a randomised trial in Uganda. This paper describes the process evaluation that will be conducted alongside this trial. The aim is to identify factors affecting the implementation, fidelity, effects, and scaling up use of the Be smart about your health teaching resources in Uganda and potential adverse and beneficial effects of the intervention. Methods Forty teachers from 40 schools in the intervention arm of the trial will complete a lesson evaluation questionnaire after each of the 10 lessons and at the end of the term. We will conduct structured classroom observations at all 40 schools. We will purposively select eight schools where we will conduct focus group discussions with teachers and students. We also will conduct key informant interviews among education officers (that visit the schools to monitor the implementation), teachers, head teachers, learners, and their parents. We will use a framework analysis approach to analyse the data. Expected results We anticipate that the findings from this evaluation will provide insights into factors that may impact the effectiveness of using the Be smart about your health resources, factors that can inhibit or facilitate scaling up use of the resources, and potential effects that were not measured quantitatively after the intervention, including transfer of what was learned to other contexts and adverse effects.

Background Researchers often overlook potential adverse effects of educational and public health interventions.Adverse effects of an intervention are increases in adverse outcomes, or decreases in beneficial outcomes, attributed to the intervention in a randomised trial.In a previous study, we developed a framework of potential adverse effects of an intervention intended to improve critical thinking about health choices: the IHC secondary school intervention. The framework includes potential effects on both students and teachers. In this study, building on the framework, we will develop and evaluate (validate) questionnaire items (questions), for measuring outcomes identified in the development of the framework.In a separate, subsequent study, we will conduct a quantitative evaluation of the items. Objectives Prioritise potential adverse effects of the IHC secondary school intervention, for the 1-year-follow-up assessments of the trials of the intervention Develop and evaluate questionnaire items for measuring potential adverse effects, focusing on those that we have prioritised Methods The overarching steps of this study are 1) prioritising outcomes included in the framework developed previously; 3) brainstorming and drafting questionnaire items; and 3) conducting a qualitative evaluation of the items. Results Results of the study will include: a set of potential adverse effects (increases in adverse outcomes) prioritised for the trials of the IHC secondary school intervention, and measures of potential adverse outcomes that have been qualitatively evaluated.

Background The value of a formal education is limited if people are unable to use or “transfer” what they learn in school to other contexts. However, there is uncertainty about how to achieve and evaluate transfer of learning, especially “far” transfer.Intended transfer of learning from an intervention is an intended increase in transfer outcomes attributed to the intervention in a randomised trial. To help assess potential far transfer of learning (far transfer effects) from an intervention intended to improve critical thinking about health choices—the Informed Health Choices (IHC) secondary school intervention—we are developing and evaluating (validating) questionnaire items (questions or tasks).In a separate, subsequent study, we will conduct a quantitative evaluation of the items. We will use the items developed in this study to measure outcomes in the 1-year-follow-up assessments of randomised trials of the IHC secondary school intervention in Kenya, Rwanda, and Uganda.In a separate, subsequent study, we will conduct a quantitative evaluation of the items. Objectives Identify intended transfer of learning (intended transfer effects, or increases in transfer outcomes) caused by the IHC secondary school intervention, by developing a model Prioritise potential outcomes included in the model for the 1-year-follow-up assessments of the trials of the intervention Develop and evaluate questionnaire items for measuring potential effects included in the model, focusing on the prioritised outcomes Methods The overarching steps of this study are 1) developing a model of potential transfer of learning from the intervention; 2) prioritising outcomes included in the model; 3) brainstorming and drafting questionnaire items; and 4) conducting a qualitative evaluation of the items. Results Results of the study will include: a model of intended transfer of learning (intended transfer effects, or increases in transfer outcomes) from the IHC secondary school intervention, a set of outcomes included in the model prioritised for the trials of the intervention, and measures of potential transfer of learning that have been qualitatively evaluated. Discussion An important limitation of the study is that any measure of skill developed will be intervention-inherent when used in the trials.

Background: People frequently make decisions about what can improve their health. They do this based on large amounts of information that they get from different sources, which include claims about what harms or improves health. People therefore need skills to assess the trustworthiness of health claims. The Informed Health Choices (IHC) Key Concepts framework was used to develop the ‘Be Smart about your Health’ secondary school resources to help students in lower secondary to assess health claims about the effects of treatments. We are evaluating these resources in a cluster randomized trial in 80 lower secondary schools in Kenya. This protocol is for a process evaluation that will be conducted alongside the trial. The objective of this evaluation is to explore the extent to which the intervention was delivered as planned; understand factors that facilitated or hindered the delivery and impact of the resources; and antici-pated and unanticipated effects. Study design: We will employ a mixed-methods design using quantitative and qualita-tive data. We will collect quantitative data from all schools (n=40) allocated to the in-tervention arm using lesson evaluation forms. Our qualitative data collection will in-clude: (a) structured classroom observations in all schools (n=40), with at least one lesson observed in each school. In a sample of eight schools, we shall observe more than one lesson. (b) We will conduct focus group discussions (with students (n=4), teachers (n=1) and parents (n=4). (c) Key informant interviews with policymakers in education (n=5), teachers (n=8), and with school principals (n=8)). We will purposively select these schools based on location (urban and rural) and ownership (private and public). Data analysis: We will use framework analysis to analyze qualitative data and descrip-tive analysis to analyze quantitative data. We will summarize and appraise the confi-dence of the key findings from the qualitative data using a modified version of the GRADE-CERQual approach.

Background: Young people are exposed to many claims about the effects of things they can do to protect or improve health. To make good health choices, they need the ability to assess the reliability of those claims critically. Critical thinking is a core competence in the Rwandan secondary school curriculum and in many countries. However, critical thinking about health is rarely taught in Rwanda. Digital secondary school resources for critical thinking about health can potentially address this gap and if published be widely disseminated at low cost. The objective of the planned study is to evaluate the effects of using digital secondary school resources to help students learn to think critically about health. Methods: We will conduct a two-arm cluster-randomized trial. We will randomly select 84 lower secondary schools from 10 districts representing all five provinces of Rwanda. Using stratified random allocation, we will assign 84 schools to the intervention or control arm. Schools in the intervention arm will teach 10-lessons after teacher training on the content of the secondary school resources. Schools in the control arm will not receive any training and resources. They will carry on with teaching the national curriculum. The primary outcome will be the proportion of students with a predetermined passing score on a test with multiple-choice questions from the Claim Evaluation Tools item bank. The questions measure the ability to apply key concepts related to assessing health claims and making informed health choices. The test will include two questions addressing each of the nine concepts covered by the secondary school resources. Discussion: This is one of three randomized trials to assess the effects of the Informed Health Choices secondary school resources in Rwanda, Kenya, and Uganda. The findings will inform decisions about how to promote critical thinking about health. Trial registration: Pan African Clinical Trial Registry, trial identifier: PACTR202203880375077, Registered on 15th February 2022.

Background: There is an overabundance of claims about the advantages and disadvantages of health interventions. Many people are unable to assess the reliability of these claims. Acting on unreliable health claims or failing to act on reliable advice can lead to waste of resources and unnecessary suffering. There is a global focus on teaching critical thinking, including in the new Kenyan curriculum, however, critical thinking about health is not taught in Kenyan schools. The study planned for in this protocol is an evaluation of the effects of the Informed Health Choices secondary school intervention on the ability of lower secondary school students in Kenya to think critically about health information and choices. The intervention consists of providing teachers with digital teaching resources and training them to deliver the intervention to students. The intervention covers 10 lessons that address nine key concepts. Methods: This study is a cluster-randomized trial. We will stratify lower secondary schools by ownership and geographical location, and randomly select 80 of them. We will randomly allocate the schools to either the intervention or control group. The intervention schools will teach the 10 lessons in one academic term alongside the Kenya national secondary school curriculum. The control schools will continue teaching the national secondary school curriculum. The primary outcome measure will be a test with multiple-choice questions from the Claim Evaluations Tools item bank. The test will include two items for each of the nine concepts covered by the teaching resources. The primary outcome will be the proportion of students attaining a predetermined passing score. Trial registration: Pan African Clinical Trial Registry, trial identifier: PACTR202204883917313. Registered on 05/04/2022.

Background To make well-informed choices, people must possess skills to assess the trustworthiness of health-related claims. It is important that young people learn to assess the reliability of claims to inform decisions, both when making their own choices and as citizens participating in a democracy. This trial aims to evaluate the effect of the Informed Health Choices (IHC) teaching resources on secondary school students’ ability to assess health-related claims and make informed choices. Methods This will be a two-arm cluster-randomised trial. We will randomise 80 lower secondary schools to evaluate the IHC secondary school digital teaching resources. The resources for teachers include 10 lessons to be delivered in a single school term, using lesson plans developed for classrooms equipped with only a blackboard or with a projector. Teachers in the intervention arm will be invited to a three-day teacher training workshop. Teachers in the control schools will continue teaching the national curriculum. Uganda’s National Curriculum Development Centre introduced a new competence-based curriculum in 2020. This curriculum has critical thinking as one of seven generic skills to be taught across all subjects. The curriculum does not explicitly include critical thinking about health. The IHC lessons address nine prioritised key concepts. We will use multiple choice questions – two for each key concept - to evaluate the student’s ability to assess claims and make informed choices. We will measure the proportion of students with a passing score at the end of the school term, and again after one year to assess retention of what was learned. Expected results Based on previous work done in Ugandan primary schools, we anticipate that the use of the teaching resources will lead to a large improvement in the lower secondary school students’ ability to assess claims and make informed health choices.

Additional file 1. Interview guides.

Additional file 3. ICT observation checklist.

Additional file 5. COREQ checklist.

Unreliable information about health care, combined with the inabilityto assess the reliability of such information, can lead to uninformed decisionsand, ultimately, waste and unnecessary suffering. Journalism is a particularlyimportant source of health information. There have been several scientificstudies of the quality (reliability) of news media reports about the effects ofhealth interventions. There does not appear to be a systematic review of thesestudies. The planned review is intended to inform: the production and consumption of health news; as well as further research, including the developmentof interventions to help people assess health care information in the news, sothey can make well-informed decisions.

When developing new, digital resources for secondary school students in Kenya, Rwanda and Uganda, we will explore issues that can impact eventual use and implementation of the new resources, in separate context analysesin each country. We will do this before starting to design the new resources, so we can address the issues in the design process.

Additional file 2: Table S1. Results for each concept one year after listening to the podcast. Table S2. Intended behaviours. Table S3. Self-efficacy. Table S4. Self-reported behaviour - awareness of treatment claims. Table S5. Self-reported behaviour - assessment of the basis of treatment claims. Table S6. Self-reported behaviour - assessment of trustworthiness of treatment claims. Table S7. Self-reported behaviour - assessment of advantages and disadvantages of treatments. Table S8. Subgroup analyses - education. Table S9. Subgroup analyses - child in school that used IHC primary school resources. Table S10. Effect of IHC primary school resources on parents.

Additional file 3. Prioritisation of Key Concepts.

Additional file 3. Distribution of scores and curves.

Additional file 1. The Claim Evaluation Tools.

The Informed Health Choices (IHC) Key Concepts are principles for thinking critically about healthcare claims and deciding what to do. The Key Concepts provide a framework for designing curricula, learning resources, and evaluation tools. The objective of this study is to prioritise which of the 49 IHC Key Concepts to include in learning resources that we are developing for lower secondary schools in East Africa, and to determine the order in which the concepts should be learned, and the competences and dispositions needed to use those Key Concepts.

The objective of this research isto develop a set of accessible digital learning resources that are experiencedas useful, usable, understandable, credible, desirable, and well-suited for use by secondary schoolstudents and teachers in Uganda, Kenya and Rwanda.

Additional file 2. Structured press releases and a news service

The objective of this research was to design mass media resources to enable the public to assess the trustworthiness of claims about the effects (benefits and harms) of treatments, and make appropriate health choices.

The objectives of this research were to develop the Informed Health Choices (IHC) primary school resources, and to ensure that teachers and children experienced these resources as useful, usable, understandable, credible, desirable, and well-suited to them.