Search

Background: There is substantial descriptive and correlational evidence that the COVID-19 pandemic has exacerbated achievement gaps, as historically underserved students have fallen farther behind in reading. Furthermore, contextual factors like concentrated poverty have systematically amplified the negative effects of the pandemic. Recent data from the Grade 3 NWEA MAP reading test (Lewis et al., 2021) revealed larger declines from 2019 to 2021 in reading comprehension performance in high-poverty schools than low-poverty schools. Purpose: We report findings from an effectiveness study that was focused on empowering teachers to make structured adaptations, in which teachers were provided opportunities to adapt program components, to support third-graders' reading comprehension outcomes during the COVID-19 pandemic school year when most US public school districts provided instruction remotely. Setting: The setting for this study is a large urban school district in the southeastern United States serving a diverse student population serving a high proportion of historically underserved and high poverty students (see table 1 for sample characteristics). Intervention: The randomized controlled trial aims to test the efficacy of allowing teachers to make structured adaptations to an evidence-based online science content literacy intervention, Model of Read Engagement (MORE). To prepare teachers to make these adaptations as well as increase their knowledge of the science of reading, the adaptive MORE teachers participated in a series of 4 asynchronous modules in November. The modules were based on Team-Based Learning (TBL) which aims to tightly couple knowledge acquisition with knowledge application (Michaelsen & Sweet, 2008). The knowledge application phase occurred in December 2020 as teachers applied their learnings to make structured and productive adaptations to MORE for their contexts. Teachers were given the flexibility to make both procedural-based and content-based changes to implementation, while keeping the core components of the intervention unchanged. Then in early January all teachers were provided training for the Online MORE curriculum with a focus on MORE's theory of change, the details of the lessons, and best practices based upon previous years in-person results. Classroom implementation of Online MORE curriculum took place from February to March 2021, and posttest assessments were administered in-person in April 2021 when the school district brought students back to their schools for face-to-face instruction. The Online MORE curriculum provides teachers with five weeks of science lessons including 10 synchronous lessons (2x per week) as well as 3 days per week of 30-minute asynchronous lessons that the students are expected to complete on their own. Science lessons focused on human body systems (muscular, skeletal, nervous systems). The Online version of the curriculum adapts the in-person version of the science MORE curriculum. Research Design: A total of 98 third-grade classroom teachers (N = 1,914 third-grade students in the analytic sample) within 26 elementary schools were randomly assigned to (1) a core treatment condition that replicated implementation procedures used in previous studies or a (2) core treatment with structured adaptations (Adaptive MORE). Figure 1 shows the consort flow diagram. Data Collection and Analysis: Our analysis explores three main research questions: 1. First, compared with core MORE, what is the effect of structured teacher adaptations on student and family engagement in MORE activities (online curriculum, and comprehension activities)? 2. Second, what is the effect of structured teacher adaptations on student reading comprehension scores in the Spring? 3. Third, compared with core MORE, what is the effect of structured teacher adaptations on teachers' pedagogical content knowledge and the quality of implementation of the synchronous instruction? To investigate these questions, we employed a two-level hierarchical linear model (HLM) with students at level 1, teachers at level 2, and fixed effects for school. Findings/Results: First, our results revealed no differences in student engagement with the online curriculum or mailed comprehension activities. However, on the comprehension activities that were part of the asynchronous App activities, students in the Adaptive MORE condition scored higher than CORE students on measures of the number of books completed and exposure to target vocabulary words. Second, HLMs indicated that students taught by teachers in the Adaptive MORE condition significantly outperformed students taught by teachers in the core treatment on a domain specific measure of science content knowledge (ES = 0.096, s.e. = 0.038) and science reading comprehension (ES = 0.074, se = 0.033). There were no significant effects on domain general measures of reading comprehension on the NWEA MAP (ES = 0.005, s.e. = 0.032) or the North Carolina End of Grade test (ES = -0.017, s.e. = 0.033) Third, there were no differences on a teachers' pedagogical content knowledge. However, fidelity analyses of the lessons using a quality of teacher-student interaction rating scheme applied to the Zoom recordings indicated that adaptive MORE teachers had higher quality of interactions with their students on indicators of engagement, questioning, and feedback (See Figure 2). Conclusions: The Online MORE intervention was delivered remotely during the COVID-19 pandemic school year (2020-21) and demonstrates how structured teacher adaptations may enhance both the quality of teacher and student engagement and support improvement in science content reading comprehension outcomes.