1. An Evaluation of the Literacy-Infused Science Using Technology Innovation Opportunity (LISTO): i3 Evaluation (Valid 45) Addendum Report
- Author
-
Johns Hopkins University, Center for Research and Reform in Education (CRRE), Wolf, Rebecca, Cook, Michael, Grant, Ashley, Reid, Alan, Neitzel, Amanda, Ross, Steven, and Risman, Kelsey
- Abstract
The purpose of Project LISTO is to support the instructional capacity of science educators and to validate innovative practices and strategies via previously developed interventions that address literacy-infused science and technology integration with standards-aligned curriculum. The evaluation of LISTO involved a multisite cluster randomized trial (CRT) designed to meet the Every Student Succeeds Act (ESSA) Tier 2 standards for "moderate" evidence, as well as the What Works Clearinghouse (WWC) standards "with reservations." The study estimated program impacts on both student and teacher outcomes and documented the fidelity of implementation and educators' perceptions of program quality. LISTO (Valid 45), and the corresponding VPD, VMC, and curricula resources did not lead to improved student achievement in science or reading for students consented to participate in the study. There was a negative impact on students' science achievement in each of the study years, except for Year 3, when science achievement data was not collected. These quantitative findings conflicted with qualitative data collected from LISTO teachers, who indicated that the program led to improvements in both science vocabulary and engagement and self-efficacy in science for students. LISTO teachers also indicated that the program had benefited their struggling readers, but there was no observed program impact on student reading achievement in any study year. While LISTO may have yielded some benefits for students, these benefits were not well captured on the standardized tests or survey instruments employed. LISTO had positive effects on teacher practices for a subsample of teachers, specifically on increased delivery of research-based instruction to teach science content as rated on a rubric by external reviewers. There were no differences in two other teacher outcomes, however, focused on the share of instructional time spent teaching new science content while performing various activities. The LISTO teachers who participated in the program reported that the Virtual Professional Development (VPD) and Virtual Mentoring and Coaching (VMC) were well-received by teachers. In sum, LISTO appeared to improve instructional practices for a sample of teachers who implemented the program for two years with complete data. Encouragingly, treatment teachers' overall positive reactions to the program suggest its potential to improve student affect and learning, but more extensive implementation experience by teachers and multi-year exposure by students starting in earlier grades may be needed to yield measurable benefits.
- Published
- 2022