Search

The purpose of this document is to provide information and advice to support states in meeting USED Peer Review Requirements related to demonstrating the comparability of test scores across various devices used for technology-based testing. The document is divided into four main sections. In the first section, we discuss and define the comparability of scores. In the second and third sections, we present a brief summary of relevant research on the comparability of state assessment scores. The second section contains a summary of two key areas of score comparability research: (1) comparability between paper-and-pencil and computer-based assessments and (2) research on score comparability issues associated with student accommodations. The third section of the paper contains a summary of emerging research and analyses of comparability issues dealing directly with the use of various technological devices. The fourth section contains a synthesis and interpretation of the research findings issued related to score comparability across devices, as well as our recommendations for the type of documentation and evidence that states should be compiling in order to address those issues and present a solid case for peer review. [This report was commissioned by the Council of Chief State School Officers (CCSSO) Technical Issues in Large-Scale Assessment (TILSA) State Collaborative on Assessment and Student Standards (SCASS). For an update (2020) to this report, see ED610778.]