Critical thinking education: Evaluation of a science-based intervention based on a cognitive model of critical thinking

Although Critical Thinking Education is strongly encouraged, there are few controlled studies on its effectiveness, especially in the case of young children (Pasquinelli et al., 2020, 2021). Some authors are skeptical that critical thinking education can give what it promises (Willingham, 2007). Moderate optimism comes from recent meta-analyses, which find significant impact for interventions that can broadly described as “teaching to think”, “teaching to read, understand, and interpret”, “teaching to evaluate information” (Abrami et al., 2015; Higgins et al., 2005). Nonetheless, the quality of evidence is quite diverse, and interventions are hardly comparable because they aim at different objectives and put in place a variety of means, for various durations and intensity. In our study we provide a narrow and operational definition of critical thinking and a cognitive functional model of critical thinking. We then propose to test this model by evaluating the impact of an educational intervention aimed at K-5 pupils, in an ecological setting. A definition of critical thinking Critical thinking is here defined as: the capacity of calibrating one's confidence in relationship with new information. In order to calibrate one's confidence one needs to correctly evaluate the epistemic quality of information: whether the sources are trustworthy, whether the content is reliable, whether the argumentation is sound. The correct evaluation of information is thus based on certain criteria that students progressively master and apply skilfully in order to separate wheat from chaff in the mass of information available. This will constitute the necessary ground for distinguishing sound opinions from flawed ones, news from fake, and to form more informed opinions and decisions. The model MODEL / graph According to our model, the motivation and capacity to correctly evaluate information and to calibrate one’s confidence are not only learned, but are based on general cognitive modules and functions that are present in our cognitive architecture. These motivation and capacity are naturally present and develop in every individual from an early age. Then, like many other intellectual skills, it takes some education to reach a proper level of critical thinking (Khun & Dean, 2004). We’ve identified two major modules/groups of functions that constitute the natural ground for critical thinking: the epistemic vigilance module and the use of metacognition in decision making, especially metacognitive confidence. Epistemic vigilance and criteria The epistemic vigilance or open vigilance module is dedicated to assess the trustworthiness of potential informants and the reliability of the contents of information. It includes several components, such as selective trust in relationship with sources, plausibility check and relevance assessment in respect to contents, and capacities such as mind reading, causal reasoning, etc. (Sperber et al. 2010, Mercier 2017). It has been shown that epistemic vigilance is active since early childhood and that children are sensitive to the quality of their sources of information. For example, pre-school children use some general criteria for identifying the adult they can trust as a source of information. Nonetheless, these criteria are quite general and they are used in a quick and dirty manner, via the evaluation of indirect evidence that is immediately available to the child, but does not guarantee that the assessment is correct (Corriveau & Harris, 2011). Based on the literature in developmental psychology, especially in relationship with the development of epistemic vigilance, we can identify the following families of criteria that help assess the epistemic quality of information: The quality of the source (trustworthiness) the benevolence of the source the expertise and objectivity of the source the convergence of multiple (good) sources The quality of the information content (reliability, probability that it corresponds to reality) the plausibility of content the relevance and soundness of the arguments in favor of the content the quality of the evidence that supports the contents. Despite the fact that these criteria are operational in our cognitive functioning since childhood, they fail to protect us from making major mistakes in relationship with contexts and contents that are “new” or complex. The reasons for our mistakes are multiple, and make lots of sense in an evolutionary perspective (Haselton et al. 2015). We can be relatively optimistic about our innate gullibility, since our epistemic vigilance mechanisms seem to protect us most of the time from mass persuasion and fake news (Mercier 2016 ; Acerbi 2019). But there’s the rub, in that we - as adults and of course children - sometimes lack the necessary knowledge and strategies for correctly assessing those beliefs and for adjusting our confidence in respect to them. The information that causes problems is the result of accumulated knowledge that is not immediately available to us. The sources of the information we currently face are often distant and opaque ones, for which our inherited systems of evaluation are not naturally equipped and prepared. Metacognitive confidence We also have to deal, in order to correctly assess the epistemic quality of information, with cognitive biases such as overconfidence and other optimistic biases. For this reason, the advanced and proficient use of the aforementioned criteria requires an adjustment of our metacognitive confidence. Metacognitive confidence is an early function of our mind. Studies show that even babies younger than 2 years-old possess a capacity known as metacognitive sensitivity - the capacity to retrospectively distinguish one’s own correct and incorrect decisions, henceforth of matching their confidence in their actions or beliefs to their accuracy (Goupil & Kuider 2019; Fleming & Lau 2014). Despite the precocious development of implicit forms of metacognition, and the fundamental role they play in decision-making, learning and the updating of information, our confidence judgments are not always calibrated and adjusted. In fact, they suffer both of general biases and of local defects of calibration. These limitations are at least in part context- and content-dependent. What it means to develop critical thinking, according to the model The limitations of epistemic vigilance and metacognitive confidence are the basis for further development via formal and informal education. But before proposing interventions to remediate for the limits of “naive” or “natural” critical thinking and to develop critical thinking into more advanced and sophisticated forms, one needs to understand why these limits exist. From Haselton (2015) and from the existing literature on the failures of educating general skills, we draw the following explanation: critical thinking cannot operate in the absence of contents. Quite the opposite, it is highly content and context-dependent, at the point that some authors doubt that it can ever be educated as a general capacity (Willingham 2007). This is blatant in the framework of the definition of critical thinking proposed here, as the capacity of correctly assessing the epistemic quality of information, i.e. its sources and contents, in order to calibrate one’s confidence. Also, each criterion mentioned above functions in a relatively independent fashion. It is not because one becomes proficient at assessing the expertise of certain sources, which operate in a specific domain of knowledge, that one is also capable of correctly assessing the plausibility or the quality of evidence in the same domain, and even more, in other domains. Because critical thinking is criteria and content-dependent, in order to develop critical thinking in a way that is as general as possible, in accordance to our model, one has to address each criterion, develop specific tools for its advanced use and train the subject to adopt the advanced tools for the criterion in different contexts and for different contents. It is compatible with our model that by tooling one of the criteria and by training to its advanced use one might obtain a much larger and general effect, at least in the short term, than the simple proficiency at using the acquired tools. In fact, the training might “awaken” the motivations that are proper to the epistemic vigilance module and metacognitive confidence operations and create the conditions for the subject to exploit other tools that he or she has acquired at some point of his/her development. However, the main hypothesis that we derive from our model is that the tooling and training of each criterion is a necessary condition for achieving satisfactory performances in critical thinking. The intervention From this model, and from the current understanding of the strengths and limits of epistemic vigilance and of confidence estimation, we’ve derived a pedagogical intervention aimed at tooling the mindware of the child with the necessary knowledge for using the aforementioned criteria in an advanced (as opposed to naïve) way. Contents of the intervention Due to time limitations, the intervention focuses on tooling and training only 1 criterion. Because in our model the criteria maintain a certain independence one from the others - even if they serve their functions in the framework of the epistemic vigilance module and of metacognitive confidence - working on one criterion at a time is not problematic. Quite the opposite, this choice guarantees that the criterion will be addressed properly. The aim of the intervention is that of strengthening the capacity of pupils of discriminating information that is based on good observations. In this way, we aim at tooling the criterion related to the evaluation of evidence in support of a content. All things being equal, multiple observations obtained in a rigorous way provide in fact evidence of a better quality than sparse, improvised observations (see the concept of pyramid of evidence as in Guyatt et al. 1992). Modality of the intervention The proposed intervention consists of 3 lessons for a total of 6 hours aimed at 5th grade pupils. Consistently with the existing literature on the education of “general abilities” (which include critical thinking), in order to comply with the ecological conditions of French schools, and with ethical concerns, we've chosen to embed the contents in the framework of science lessons. The lessons’ contents comply with the French curriculum for 5th grade students. More specifically, the lessons can be used to teach aspects of biodiversity and sustainable development. This is an important ethical consideration when proposing novel teachings, because of the risk of spending time on ineffective interventions, while still having to complete the curriculum. Also, it has practical advantages in that teachers do not have to add new contents to their rich curriculum. During lesson 1, pupils explore the question whether birds have been declining in town during the last years, and are guided to find a method of observation that will provide reliable data for answering the question. During lesson 2, pupils are confronted with the question of discovering which is the preferred habitat of lynx. Once again they discover that multiple rigorous observations constitute a condition for producing reliable knowledge. During lesson 3, pupils reason about the impact of climate change on polar bears. They draw on rigorous observation methods (multiplication of observations at different sites) in order to assess whether different bear populations are affected the same by climate change. Based on the literature on transfer in learning, one might expect that by practicing the advanced use of a criterion in a variety of contexts and contents will help the learner to transfer this ability to new contexts and contents. However, the same literature underlines the importance of explicitation (Perkins & Salomon 1981; Bransford et al. 2000; Willingham 2007; Halpern 2013; Marin & Halpern 2012) in order to favor transfer. We’ve hence proposed an approach in which pupils, in addition to being confronted with concrete examples in the domain of science education, are also explicitly taught about the importance of generalizing the kind of strategies that have been experienced during the lesson and given examples of this generalization to new contents and contexts (daily life). The explicit transfer phase happens at the end of each lesson. It is verbalized and guided by the teacher and supported by a small quiz. In order to assess whether the explicitation and the provision of examples from different domains (including daily life) is a key element in the success of the intervention we’ve included a second group which receives the same intervention in science without “explicitation + transfer examples”. I.e., at the end of each lesson the pupils of the second group (science-focused group) do not reason on how they could transfer the multiplication of observations strategy to other situations outside the three science lessons and especially in daily life. In order to guarantee that the two groups are in all similar but for the explicitation+transfer phase, the science-focused group receives a quiz about the scientific contents that have been developed (the factors that affect biodiversity) and is guided by the teacher to make them explicit. Measurement of progress in critical thinking Improvements in the three groups are measured via a pre-test and post-test, which is proposed at the same time to all the classes participating in the study. The test includes several criteria for critical thinking: some of the items in the test directly assess the criterion included in the intervention, while others don’t. Both the pre-test and post-test and the interventions take place in the classroom. The interventions take place in the framework of science lessons. Our questions Our study aims at putting the proposed model of cognitive thinking to a practical test. We want to know whether critical thinking advanced criteria can be taught in a formal context (schooling). I.e., whether pupils can learn to assess the quality of information in a more correct way, by mobilizing specific criteria that have been tooled and trained (according to the modalities described above). We also want to know if the tooling and training of one criterion helps pupils develop their metacognitive sensitivity (the match between accuracy and confidence). Finally, we want to explore the educational question whether the “explicit+transfer” is more effective in improving critical thinking (evaluation of information, appropriate use of criteria and confidence) as compared to a more implicit approach.