A Tale of Two Tests: The Role of Topic and General Academic Knowledge in Traditional versus Contemporary Scenario-Based Reading

We compared high school students' performance in a traditional comprehension assessment requiring them to identify key information and draw inferences from single texts, and a scenario-based assessment (SBA) requiring them to integrate, evaluate and apply information across multiple sources. Both assessments focused on a non-academic topic. Performance on the two assessments were moderately correlated (r=0.57), but the SBA was more difficult (Study 1; n=342). The two assessments similarly depended on basic reading skills but diverged in the relation to academic knowledge and (non-academic) topic knowledge (Study 2; n= 1,107). Academic knowledge was highly predictive of traditional comprehension, but less so for SBA. Topic knowledge was more predictive of SBA than traditional comprehension. Thus, the two assessments tap into similar constructs related to comprehension; however, the level of topic knowledge is more important for performance on scenario-based, multiple-source reading tasks, whereas academic knowledge is more important for traditional reading comprehension tasks. [This paper was published in "Learning and Instruction" v73 Article 101462 2021.]