Search

The development and evaluation of valid assessments of scientific reasoning are an integral part of research in science education. In the present study, we used the linear logistic test model (LLTM) to analyze how item features related to text complexity and the presence of visual representations influence the overall item difficulty of an established, multiple-choice, scientific reasoning competencies assessment instrument. This study used data from n = 243 pre-service science teachers from Australia, Canada, and the UK. The findings revealed that text complexity and the presence of visual representations increased item difficulty and, in total, contributed to 32% of the variance in item difficulty. These findings suggest that the multiple-choice items contain the following cognitive demands: encoding, processing, and combining of textually presented information from different parts of the items and encoding, processing, and combining information that is presented in both the text and images. The present study adds to our knowledge of which cognitive demands are imposed upon by multiple-choice assessment instruments and whether these demands are relevant for the construct under investigation--in this case, scientific reasoning competencies. The findings are discussed and related to the relevant science education literature.

Scientific reasoning competencies are highlighted in science education policy papers and standard documents in various countries around the world and pre-service science teachers are asked to develop them during teacher training as part of their professional competencies. In order to monitor the development of pre-service science teachers' scientific reasoning competencies during their course of studies and to enable evidence-based improvements of teacher training, instruments for the assessment of scientific reasoning competencies are needed. However, studies propose that the validity of most instruments for assessing scientific reasoning competencies available so far can be questioned. This study presents an English translation of an already developed German multiple-choice instrument to assess pre-service science teachers' scientific reasoning competencies. A sample of N = 105 Australian pre-service science teachers participated in this study by answering the translated instrument. Quantitative (differential item functioning, Mantel-Haenszel statistic) and qualitative (think-aloud protocols) analyses provide validity evidence for the translated instrument. Furthermore, the interpretation of the data as an indicator for the participating Australian pre-service science teachers' scientific reasoning competencies suggests that there is a need for a more explicit emphasis on scientific reasoning in Australian science teacher training.

Australian science teachers are asked to involve their students in the scientific processes of inquiry and investigation. The abilities needed for scientific problem-solving ("doing") and the capacity to reflect on the process of problem-solving at a meta level ("understanding") are defined as scientific reasoning competencies in this article. Science teachers need to have strong scientific reasoning competencies themselves to model and promote scientific reasoning competencies among students. Hence, scientific reasoning competencies have become an important element of science teachers' professional competencies. This article introduces the relevance of scientific reasoning competencies for science teacher education in Australia. In addition, it provides a theoretical approach of how to develop such competencies in pre-service science teacher education. Finally, an empirical case study on scientific reasoning competencies of pre-service science teachers at one university in Victoria is reported. The findings propose that the pre-service teachers have a reasonable level of scientific reasoning competencies themselves; however, there is no significant increase of this level during the first year of their Master of Teaching degree course. It is concluded that the development of scientific reasoning competencies is a long-term process, which might be supported by providing a sequence of appropriate problem-solving and accompanied opportunities of explicit reflections.

Scientific reasoning competencies are relevant science competencies and therefore the development of assessment instruments for scientific reasoning competencies has become an integral part of science education research. However, some authors have questioned the validity of the instruments available so far, since their psychometric quality has not been evaluated satisfactorily. In this study, an established German multiple-choice instrument to assess scientific reasoning competencies has been translated into English and Spanish and the translated versions were evaluated based on the "translation," "review," "adjudication," "pretesting" and "documentation" (TRAPD) approach. Based on samples of German (N = 497), Australian (N = 103), and Chilean (N = 135) pre-service science teachers, statistical analyses (e.g. dimensionality and DIF analyses) provide psychometric evidence for test equivalence between the German and the translated versions of the instrument. Known groups comparisons provide evidence for the valid interpretation of the test scores as measures of scientific reasoning competencies. It is discussed that further studies might evaluate additional sources of validity evidence for the translated versions of the instrument, particularly focusing on cultural specificities. Independent of this need for further research, it is suggested that the three versions can be used to assess the scientific reasoning competencies of pre-service science teachers whose first language is English, German, or Spanish.