Search

International large-scale assessments, such as Programme for International Student Assessment (PISA), are a crucial source of information for education researchers and policymakers. The assessment also includes a student questionnaire, however, the data can be biased by the differences in reporting behavior between students. In this paper, we analyse differences in response styles of students in the Czech Republic and New Zealand based on data gathered in PISA 2018. The results show that Czech students use, on average, the acquiescence response style (ARS) and extreme response style (ERS) less than their counterparts in New Zealand. Contrarily, Czech students use, on average, the disacquiescence response style (DARS) more than students from New Zealand. A closer analysis according to the school type in the Czech Republic, showed that students from 4-year secondary general schools score, on average, the highest on ARS and ERS, but the lowest on DARS among all school types. The analysis in New Zealand according to the Year of study showed that students in Years 12-13 score, on average, the highest on ARS and ERS among all Year-of-study groups, the DARS values being similar across the Year-of-study groups. The mean ARS and ERS of each of the Czech school types remains below that of any of the New Zealand Year-of-study groups. For DARS, the mean score of the highest-scoring Year-of-study group in New Zealand (Years 9-10) remains below that of the lowest-scoring Czech school type (4-year secondary general school). Analysis of response styles is a crucial tool for the identification of some of the potential biases in student-reported data. Future research should take the differences in student reporting behavior into account and perhaps also employ other methodological approaches for their identification, such as the overclaiming technique. [For the complete Volume 21 proceedings, see ED629259.]

In the context of Programme for International Student Assessment (PISA), paradoxical findings concerning the relationship between questionnaire scales and student achievement are often documented. These questionnaire scales are found to correlate positively with student achievement within countries at the individual level, but negatively at the between-country level (i.e., when correlating questionnaire scale and achievement values aggregated at the level of countries, the countries being the unit of analysis). These anomalous findings can be caused by the differences in reporting behavior of students in different countries and might lead educators and educational policy-makers to erroneous conclusions. In this paper, we examine the relationship between the teacher support scale and student achievement across three PISA cycles -- 2012, 2015, and 2018. Our results show that there is a consistent negative between-country correlation between teacher support and student achievement in all three examined PISA cycles, which is in line with the previously documented paradoxes. We have also found that some countries, which participated in all three PISA cycles under study, consistently contribute to this paradox by having quite high levels of student-reported teacher support but rather low achievement scores. Future research should take into account the differences in reporting behavior between students when making cross-country analyses and consider the application of methodological approaches to identify and adjust for these differences such as the anchoring vignette method and the overclaiming technique. [This conference contribution was made possible with the support from the Czech Science Foundation. For the complete Volume 21 proceedings, see ED629259.]