The Impact of Collaborative Strategic Reading on the Reading Comprehension of Grade 5 Students in Linguistically Diverse Schools. Final Report. NCEE 2011-4001

Collaborative Strategic Reading (CSR) is a set of instructional strategies designed to improve the reading comprehension of students with diverse abilities (Klingner and Vaughn 1996). Teachers implement CSR at the classroom level using scaffolded instruction to guide students in the independent use of four comprehension strategies; students apply the strategies to informational text while working in small cooperative learning groups. The current study is a randomized controlled trial (RCT) examining the effect of CSR on student reading comprehension. Within each participating linguistically diverse school, grade 5 social studies classrooms were randomly assigned to either the CSR condition (using CSR when delivering social studies curricula) or to the control condition (a business-as-usual condition). The implementation period was one school year. This study focused on the following confirmatory research question: In linguistically diverse schools, do grade 5 students in CSR classrooms have higher average reading comprehension posttest scores on the Group Reading Assessment and Diagnostic Evaluation (GRADE) than students in control classrooms? In addition, the study examined three exploratory research questions about CSR's effect on two subgroups of students: (1) Do grade 5 former and current English language learner (FC-ELL) students in CSR classrooms have higher average reading comprehension posttest scores on the GRADE than FC-ELL students in control classrooms?; (2) Do grade 5 non-ELL students in CSR classrooms have higher average reading comprehension posttest scores on the GRADE than non-ELL students in control classrooms?; and (3) Does CSR have a differential impact on GRADE reading comprehension posttest scores for grade 5 FC-ELL and non-ELL students? The intent of these exploratory analyses was to examine whether there is an effect for each subgroup separately, as well as whether there is a differential effect between the subgroups. The primary finding of this study is that CSR did not have a statistically significant impact on student reading comprehension. Nine sensitivity analyses--including alternative statistical approaches, an alternative approach for handling missing data, and different sample specifications--showed that the findings were robust to different analytic approaches. Three exploratory analyses were also conducted to examine the effects of CSR on FC-ELL and non-ELL students. Statistically significant effects on student reading comprehension were not identified for either subgroup, and no statistically significant differential impacts were identified. It is often the case that RCTs, because of their greater rigor, do not support the findings of prior quasi-experiments (Glazerman, Levy, and Myers 2002, 2003). With all other design features held constant, randomization yields stronger evidence about program impacts than do quasi-experiments (Boruch 1997; Shadish, Cook, and Campbell 2002). The current investigation evaluated the impact of CSR in an effectiveness trial designed to approximate a district's implementation of CSR. Data on the fidelity of implementation suggest that professional development was generally delivered according to plan. Data on teacher fidelity of CSR implementation showed that 78.8 percent of teachers reported using CSR two or more times a week, as instructed. However, the single observation conducted for each classroom found that 21.6 percent of CSR teachers were using all five core teacher strategies, which the study defined as full procedural fidelity; 56.8 percent of teachers were observed using three or fewer strategies. Appendices include: (1) Identification and exit criteria for English language learner students in Oklahoma and Texas; (2) Assumptions used to determine statistical power and observed power; (3) Random assignment; (4) Analysis of consent rate at baseline; (5) Estimation methods; (6) Frequently asked questions about contamination; (7) Attrition analyses; (8) Response rates for demographic data; (9) Fall and spring teacher surveys; (10) Fall coaching observation form; (11) Multiple imputation; (12) Assigning students to cooperative learning groups; (13) Critical procedural behaviors for Collaborative Strategic Reading strategies; (14) Observer training for the subscale Expository Reading Comprehension observation instrument and interrater reliability; (15) Descriptive statistics on Group Reading Assessment and Diagnostic Evaluation scores; (16) Baseline equivalence results for multiply imputed analytic dataset; and (17) Full analytic output tables. A glossary is included. (Contains 52 tables, 3 figures and 46 footnotes.