Search

Background People of all ages are flooded with health claims about treatment effects (benefits and harms of treatments). Many of these are not reliable, and many people lack skills to assess their reliability. Primary school is the ideal time to begin to teach these skills, to lay a foundation for continued learning and enable children to make well-informed health choices, as they grow older. However, these skills are rarely being taught and yet there are no rigorously developed and evaluated resources for teaching these skills. Objectives To develop the Informed Health Choices (IHC) resources (for learning and teaching people to assess claims about the effects of treatments) for primary school children and teachers. Methods We prototyped, piloted, and user-tested resources in four settings that included Uganda, Kenya, Rwanda, and Norway. We employed a user-centred approach to designing IHC resources which entailed multiple iterative cycles of development (determining content scope, generating ideas, prototyping, testing, analysing and refining) based on continuous close collaboration with teachers and children. Results We identified 24 Key Concepts that are important for children to learn. We developed a comic book and a separate exercise book to introduce and explain the Key Concepts to the children, combining lessons with exercises and classroom activities. We developed a teachers’ guide to supplement the resources for children. Conclusion By employing a user-centred approach to designing resources to teach primary children to think critically about treatment claims and choices, we developed learning resources that end users experienced as useful, easy to use and well-suited to use in diverse classroom settings.

Introduction Earlier, we designed and evaluated an educational mass media intervention for improving people’s ability to think more critically and to assess the trustworthiness of claims (assertions) about the benefits and harms (effects) of treatments. The overall aims of this follow-up study were to evaluate the impact of our intervention 1 year after it was administered, and to assess retention of learning and behaviour regarding claims about treatments. Methods We randomly allocated consenting parents to listen to either the Informed Health Choices podcast (intervention) or typical public service announcements about health issues (control) over 7–10 weeks. Each intervention episode explained how the trustworthiness of treatment claims can be assessed by using relevant key concepts of evidence-informed decision-making. Participants listened to two episodes per week, delivered by research assistants. We evaluated outcomes immediately, and a year after the intervention. Primary outcomes were mean score and the proportion with a score indicating a basic ability to apply the key concepts (> 11 out of 18 correct answers) on a tool measuring people’s ability to critically appraise the trustworthiness of treatment claims. Skills decay/retention was estimated by calculating the relative difference between the follow-up and initial results in the intervention group, adjusting for chance. Statistical analyses were performed using R (R Core Team, Vienna, Austria; version 3.4.3). Results After 1 year, the mean score for parents in the intervention group was 58.9% correct answers, compared to 52.6% in the control (adjusted mean difference of 6.7% (95% CI 3.3% to 10.1%)). In the intervention group, 47.2% of 267 parents had a score indicating a basic ability to assess treatment claims compared to 39.5% of 256 parents in the control (adjusted difference of 9.8% more parents (95% CI 0.9% to 18.9%). These represent relative reductions of 29% in the mean scores and 33% in the proportion of parents with a score indicating a basic ability to assess the trustworthiness of claims about treatment effects. Conclusions Although listening to the Informed Health Choices podcast initially led to a large improvement in the ability of parents to assess claims about the effects of treatments, our findings show that these skills decreased substantially over 1 year. More active practice could address the substantial skills decay observed over 1 year. Trial registration Pan African Clinical Trial Registry (www.pactr.org), PACTR201606001676150. Registered on 12 June 2016.

Background: The James Lind Initiative (JLI) was a work programme inaugurated by Iain Chalmers and Patricia Atkinson to press for better research for better health care. It ran between 2003 and 2018, when Iain Chalmers retired. During the 15 years of its existence, the JLI developed three strands of work in collaboration with the authors of this paper, and with others.Work themes: The first work strand involved developing a process for use by patients, carers and clinicians to identify shared priorities for research - the James Lind Alliance. The second strand was a series of articles, meetings, prizes and other developments to raise awareness of the massive amounts of avoidable waste in research, and of ways of reducing it. The third strand involved using a variety of approaches to promote better public and professional understanding of the importance of research in clinical practice and public health. JLI work on the first two themes has been addressed in previously published reports. This paper summarises JLI involvement during the 15 years of its existence in giving talks, convening workshops, writing books, and creating websites and databases to promote critical thinking about treatment claims.Conclusion: During its 15-year life, the James Lind Initiative worked collaboratively with others to create free teaching and learning resources to help children and adults learn how to recognise untrustworthy claims about the effects of treatments. These resources have been translated in more than twenty languages, but much more could be done to support their uptake and wider use.

Objectives Claims about interventions are ubiquitous. The ability to assess these claims and make informed choices depends on understanding and applying key concepts that are essential for making judgements about whether a claim is justified, whether comparisons are fair and reliable, and what action to take. The Informed Health Choices (IHC) project has developed educational resources for schoolchildren and their parents to help improve their ability to assess claims about treatment effects. As our starting point, we developed a list of Key Concepts that people need to understand to assess these claims. The list currently includes 36 concepts and serves as a syllabus or curriculum. The objectives of this workshop are to discuss the Key Concepts, their potential uses, whether they are sensible, and their further development. We will also discuss applicability of the concepts to claims and decisions about tests; and to other areas, such as education and environmental management. Method We generated an initial list of concepts by identifying key concepts in literature and tools written for the general public, journalists, and health professionals, and by considering concepts related to assessing the certainty of evidence for treatment effects. We established a project advisory group composed of key researchers, journalists, teachers, and others with expertise in health literacy and teaching or communicating evidence-based healthcare to patients. Based on feedback collected online and at a workshop we have updated the list of IHC Key Concepts yearly (in 2016 and 2017). Results The IHC Key Concepts provide a basis for developing learning resources to help people understand and apply the concepts when claims are made about the effects of treatments (and other interventions), and when health choices are made. The list of concepts also forms the basis for a database of multiple-choice questions which can be used to assess people’s ability to apply the IHC Key Concepts. The IHC Key Concepts serve as standards for judgment, or principles for evaluating the trustworthiness of treatment claims, comparisons, and choices. The list is intended to be universally relevant. The concepts can help people to: Recognise claims about the effects of treatments which have an unreliable basis. Understand whether comparisons of treatments are fair and reliable. Make informed choices about treatments. Conclusions This workshop will be a structured discussion of the potential uses of the IHC Key Concepts, the extent to which they are sensible and useful, and ways in which they might be improved. The Key Concepts are reviewed annually to allow for revisions of existing concepts and identification and inclusion of additional concepts. This discussion will feed into this process.

Many claims about the effects of treatments, though well intentioned, are wrong. Indeed, they are sometimes deliberately misleading to serve interests other than the well-being of patients and the public. People need to know how to spot unreliable treatment claims so that they can protect themselves and others from harm. The ability to assess the trustworthiness of treatment claims is often lacking. Acquiring this ability depends on being familiar with, and correctly applying, some key concepts, for example, that’ association is not the same as causation.’ The Informed Health Choices (IHC) Project has identified 36 such concepts and shown that people can be taught to use them in decision making. A randomised trial in Uganda, for example, showed that primary school children with poor reading skills could be taught to apply 12 of the IHC Key Concepts. The list of IHC Key Concepts has proven to be effective in providing a framework for developing and evaluating IHC resources to help children to think critically about treatment claims. The list also provides a framework for retrieving, coding and organising other teaching and learning materials for learners of any age. It should help teachers, researchers, clinicians, and patients to structure critical thinking about the trustworthiness of claims about treatment effects.

Imagine you are a primary care doctor. A patient comes into your office with acute, atypical chest pain. Immediately you consider the patient’s sex and age, and you begin to think about what questions to ask and what diagnoses and diagnostic tests to consider. You will also need to think about what treatments to consider and how to communicate with the patient and potentially with the patient’s family and other healthcare providers. Some of what you do will be done reflexively, with little explicit thought, but caring for most patients also requires you to think critically about what you are going to do. Critical thinking, the ability to think clearly and rationally about what to do or what to believe, is essential for the practice of medicine. Few doctors are likely to argue with this. Yet, until recently, the UK regulator the General Medical Council and similar bodies in North America did not mention “critical thinking” anywhere in their standards for licensing and accreditation,1 and critical thinking is not explicitly taught or assessed in most education programmes for health professionals. Moreover, although more than 2800 articles indexed by PubMed have “critical thinking” in the title or abstract, most are about nursing. We argue that it is important for clinicians and patients to learn to think critically and that the teaching and learning of these skills should be considered explicitly. Given the shared interest in critical thinking with broader education, we also highlight why healthcare and education professionals and researchers need to work together to enable people to think critically about the health choices they make throughout life.

Background The ability to appraise claims about the benefits and harms of treatments is crucial for informed health care decision-making. This research aims to enable children in East African primary schools (the clusters) to acquire and retain skills that can help them make informed health care choices by improving their ability to obtain, process and understand health information. The trial will evaluate (at the individual participant level) whether specially designed learning resources can teach children some of the key concepts relevant to appraising claims about the benefits and harms of health care interventions (treatments). Methods This is a two-arm, cluster-randomised trial with stratified random allocation. We will recruit 120 primary schools (the clusters) between April and May 2016 in the central region of Uganda. We will stratify participating schools by geographical setting (rural, semi-urban, or urban) and ownership (public or private). The Informed Healthcare Choices (IHC) primary school resources consist of a textbook and a teachers’ guide. Each of the students in the intervention arm will receive a textbook and attend nine lessons delivered by their teachers during a school term, with each lesson lasting 80 min. The lessons cover 12 key concepts that are relevant to assessing claims about treatments and making informed health care choices. The second arm will carry on with the current primary school curriculum. We have designed the Claim Evaluation Tools to measure people’s ability to apply key concepts related to assessing claims about the effects of treatments and making informed health care choices. The Claim Evaluation Tools use multiple choice questions addressing each of the 12 concepts covered by the IHC school resources. Using the Claim Evaluation Tools we will measure two primary outcomes: (1) the proportion of children who ‘pass’, based on an absolute standard and (2) their average scores. Discussion As far as we are aware this is the first randomised trial to assess whether key concepts needed to judge claims about the effects of treatment can be taught to primary school children. Whatever the results, they will be relevant to learning how to promote critical thinking about treatment claims. Trial status: the recruitment of study participants was ongoing at the time of manuscript submission. Trial registration Pan African Clinical Trial Registry, trial identifier: PACTR201606001679337. Registered on 13 June 2016. Electronic supplementary material The online version of this article (doi:10.1186/s13063-017-1958-8) contains supplementary material, which is available to authorized users.

Background\ud People are frequently confronted with untrustworthy claims about the effects of treatments. Uncritical acceptance of these claims can lead to poor, and sometimes dangerous, treatment decisions, and wasted time and money. Resources to help people learn to think critically about treatment claims are scarce, and they are widely scattered. Furthermore, very few learning-resources have been assessed to see if they improve knowledge and behavior.\ud \ud Objectives\ud Our objectives were to develop the Critical thinking and Appraisal Resource Library (CARL). This library was to be in the form of a database containing learning resources for those who are responsible for encouraging critical thinking about treatment claims, and was to be made available online. We wished to include resources for groups we identified as ‘intermediaries’ of knowledge, i.e. teachers of schoolchildren, undergraduates and graduates, for example those teaching evidence-based medicine, or those communicating treatment claims to the public. In selecting resources, we wished to draw particular attention to those resources that had been formally evaluated, for example, by the creators of the resource or independent research groups.\ud \ud Methods\ud CARL was populated with learning-resources identified from a variety of sources—two previously developed but unmaintained inventories; systematic reviews of learning-interventions; online and database searches; and recommendations by members of the project group and its advisors. The learning-resources in CARL were organised by ‘Key Concepts’ needed to judge the trustworthiness of treatment claims, and were made available online by the James Lind Initiative in Testing Treatments interactive (TTi) English (www.testingtreatments.org/category/learning-resources).TTi English also incorporated the database of Key Concepts and the Claim Evaluation Tools developed through the Informed Healthcare Choices (IHC) project (informedhealthchoices.org).\ud \ud Results\ud We have created a database of resources called CARL, which currently contains over 500 open-access learning-resources in a variety of formats: text, audio, video, webpages, cartoons, and lesson materials. These are aimed primarily at ‘Intermediaries’, that is, ‘teachers’, ‘communicators’, ‘advisors’, ‘researchers’, as well as for independent ‘learners’. The resources included in CARL are currently accessible at www.testingtreatments.org/category/learning-resources\ud \ud Conclusions\ud We hope that ready access to CARL will help to promote the critical thinking about treatment claims, needed to help improve healthcare choices.

Background Claims made about the effects of treatments are very common in the media and in the population more generally. The ability of individuals to understand and assess such claims can affect their decisions and health outcomes. Many people in both low- and high-income countries have inadequate aptitude to assess information about the effects of treatments. As part of the Informed Healthcare Choices project, we have prepared a series of podcast episodes to help improve people’s ability to assess claims made about treatment effects. We will evaluate the effect of the Informed Healthcare Choices podcast on people’s ability to assess claims made about the benefits and harms of treatments. Our study population will be parents of primary school children in schools with limited educational and financial resources in Uganda. Methods This will be a two-arm, parallel-group, individual-randomised trial. We will randomly allocate consenting participants who meet the inclusion criteria for the trial to either listen to nine episodes of the Informed Healthcare Choices podcast (intervention) or to listen to nine typical public service announcements about health issues (control). Each podcast includes a story about a treatment claim, a message about one key concept that we believe is important for people to be able to understand to assess treatment claims, an explanation of how that concept applies to the claim, and a second example illustrating the concept. We designed the Claim Evaluation Tools to measure people’s ability to apply key concepts related to assessing claims made about the effects of treatments and making informed health care choices. The Claim Evaluation Tools that we will use include multiple-choice questions addressing each of the nine concepts covered by the podcast. Using the Claim Evaluation Tools, we will measure two primary outcomes: (1) the proportion that ‘pass’, based on an absolute standard and (2) the average score. Discussion As far as we are aware this is the first randomised trial to assess the use of mass media to promote understanding of the key concepts needed to judge claims made about the effects of treatments. Trial registration Pan African Clinical Trials Registry, PACTR201606001676150. Registered on 12 June 2016. http://www.pactr.org/ATMWeb/appmanager/atm/atmregistry?dar=true&tNo=PACTR201606001676150. Electronic supplementary material The online version of this article (doi:10.1186/s13063-016-1745-y) contains supplementary material, which is available to authorized users.

Background: The Informed Health Choices (IHC) Key Concepts are standards for judgement, or principles for evaluating the trustworthiness of treatment claims and treatment comparisons (evidence) used to support claims, and for making treatment choices. The list of concepts provides a framework, or starting point, for teachers, journalists and other intermediaries for identifying and developing resources (such as longer explanations, examples, games and interactive applications) to help people to understand and apply the concepts. The first version of the list was published in 2015 and has been updated yearly since then. We report here the changes that have been made from when the list was first published up to the current (2018) version.Methods: We developed the IHC Key Concepts by searching the literature and checklists written for the public, journalists, and health professionals; and by considering concepts related to assessing the certainty of evidence about the effects of treatments. We have revised the Key Concepts yearly, based on feedback and suggestions; and learning from using the IHC Key Concepts, other relevant frameworks, and adaptation of the IHC Key Concepts to other types of interventions besides treatments.Results: We have made many changes since the Key Concepts were first published in 2015. There are now 44 Key Concepts compared to the original 32; the concepts have been reorganised from six to three groups; we have added higher-level concepts in each of those groups; we have added short titles; and we have made changes to many of the concepts.Conclusions: The IHC Key Concepts have proven useful in designing learning resources, evaluating them, and organising them. We will continue to revise the IHC Key Concepts in response to feedback. We welcome suggestions for how to do this.