Search

Although some research-based educational practices have shown promise, many fail to be implemented at a scale that affects more than a small proportion of children. Further, research on interventions for young children includes mixed results, with most documenting "fadeout" of effects after several years, but some showing lasting effects. In this study, the authors evaluated the long-term impacts of a model for scaling up early interventions, in this case a successful early mathematics curriculum, testing to see whether the originally-sustained impacts persisted up to 7 years beyond baseline. The original evaluation of TRIAD (Technology-enhanced, Research-based, Instruction, Assessment, and professional Development) employed a cluster randomized trial (CRT) design to test the effectiveness of the TRIAD scale-up approach, using the Building Blocks® early mathematics curriculum (BB) in preschool. In this evaluation, 42 schools were randomly assigned to three groups. One of the groups was a business-as-usual control (CTRL). In preschool, the other two conditions were identical and entailed TRIAD scale-up (intensive training, coaching focused on both BB curriculum and linked assessment and online professional development). After preschool, one of the treatment conditions included follow-through professional development through 1st grade, including knowledge of the intervention and ways to build upon that knowledge using learning trajectories (TRIAD Follow-Through or TRIAD-FT) while the other condition implemented business-as-usual control (TRIAD Non-follow-through or TRIAD-NFT). Students in the two treatment conditions significantly outperformed their peers in the control condition through 1st grade. The primary purpose of the current investigation was to examine the long-term effects of the early mathematics curriculum, but the secondary aim was to explore whether the TRIAD scale-up approach itself, which included aligned professional development for kindergarten and 1st grade teachers, also enhanced the effects of the pre-K curriculum beyond 1st grade. Research questions were as follows: (1) What are the differences in the average math achievement of the three original study groups (TRIAD-NFT, TRIAD-FT, and CTRL) in 3rd, 4th, and 5th grade; (2) Do the differences in the 3rd, 4th, and 5th grade math achievement of the original study groups vary by the socio-economic and racial/ethnic backgrounds of the students in the groups; (3) What are the differences in the 3rd, 4th, and 5th grade math achievement of the subset of children who remained in the group to which their original school was randomly assigned throughout the intervention period (pre-K through 1st grade); and (4) Do the differences in the in the 3rd, 4th, and 5th grade math achievement of the subset of children who remained in the group to which their original school was randomly assigned throughout the intervention period (pre-K through 1st grade) vary by the students' socio-economic status and racial/ethnic backgrounds? Figure 1 presents results addressing the first research question that pertain to the differences between the three groups at the end of pre-K through 5th grade. Figures 2a and 2b inform the second research question and show the effects of the FT and NFT conditions separately for African-American, Hispanic, other minority, and white students. Figures 3a and 3b show the FT and NFT effect estimates separately for the low and high SES subgroups formed based on students' eligibility for FRPL. Figures 4, 5a, 5b, 6a, and 6b inform the 3rd and 4th research questions and show parallel results for the stayers (students who stayed in the original conditions between pre-K and 1st grade). Impacts at the end of 5th grade were statistically significant and considerably larger than impacts in 3rd and 4th grade for both conditions and all subgroups. Tables and figures are appended.

This paper presents results from the longitudinal experimental study conducted on North Carolina's early college model described in an earlier paper. The primary purpose of this paper is to present the impact of the early college model on outcomes related to postsecondary enrollment. The specific research questions driving this study include: (1) What is the impact of the early college model on students' enrollment in postsecondary education? (2) What is the impact on students' attainment of postsecondary credentials? One of the main ways the early college model accomplished its goal was by enrolling high school students into college classes as early as ninth grade. By the time students were in their junior or senior years of high school, they were taking mostly college classes with other college students. The early college also provided explicit instruction in college readiness skills and provided extensive supports to students. The study results showed early college is increasing students' access to postsecondary education, driven primarily by their experience in high school. The increase in enrollment is primarily in the area of two-year colleges because most of the early colleges are located on community college campuses. This increase in two-year enrollment is not happening at the expense of four-year enrollment, given that there is a slight increase in that arena as well. Tables and figures are appended. [For the related article: "The Small, Stand-Alone Early College: Impact on High School Outcomes," see ED567039.]

North Carolina's Early College model is the subject of an IES-funded eleven-year longitudinal experimental study that utilized a lottery process to assign early college applicants to either treatment or control groups. This paper presents findings related to high school outcomes. The primary goal of the early college model is to increase the number of students who graduate from high school and who continue on to and succeed in college. The specific research questions addressed in this paper include the following: (1) What is the impact of the early college model on the percentage of students successfully completing a college preparatory course of study in high school? (2) What is the impact of the early college model on the number of college credits earned while in high school? (3) What is the impact of the early college model on students' graduation from high school? (4) How does the impact vary for specific sub-groups including those who are low-income, first in their family to go to college, members of underrepresented minority groups, and those who entered high school below grade level? Results from this study show that the Early College High School model is having positive impacts on students' college preparatory course-taking, their attainment of college credits, and their graduation from high school. Tables and figures are appended. [For the related article: "The Small, Stand-Alone Early College: Impact on Postsecondary Outcomes," see ED567041.]

The changing nature of the U.S. economy has fostered concerns that too few students are successfully completing postsecondary education (Achieve, 2004). Three quarters of those who enter high school graduate within four years, with approximately 70 percent of those graduates enrolling immediately in some form of postsecondary education (Ross et al., 2012). Of those who do attend, insufficient numbers complete a degree with only a little less than half (49 %) of beginning postsecondary students attaining some sort of a postsecondary credential within six years of enrolling (Ross, et al., 2012). As a result, there have been numerous initiatives to increase the number of students who graduate from high school prepared to enroll and progress in postsecondary education. One approach is the Early College High School (ECHS or early college) model, small schools that blur the line between high school and college. The primary goal of the early college model is to increase the number of students who graduate from high school and who continue on to and succeed in college. This paper presents results from a longitudinal experimental study that is examining the impact of early colleges on students' outcomes in high school and in postsecondary (PS) education, specifically on postsecondary enrollment and college credit accrual during and after high school. Early results from this study show that the Early College High School model is increasing students' enrollment in postsecondary education, primarily by the required exposure in high school. The results show that, as is inherent in its design, the program is successful in providing early access to college. This paper also reports results from a detailed cost and benefit analysis of the early college model. The following are appended: (1) References; and (2) Tables and Figures.

The Comparative Short Interrupted Time Series (C-SITS) design is a frequently employed quasi-experimental method, in which the pre- and post-intervention changes observed in the outcome levels of a treatment group is compared with those of a comparison group where the difference between the former and the latter is attributed to the treatment. The increase in the availability and quality of extant data (e.g., state test scores, graduation rates, and college application rates in primary and secondary education and cognitive, language, and socio-emotional assessments in pre-school settings) has made the use of C-SITS designs a more viable option for assessing the impacts of interventions. Despite the recent growth in its use, the existing resources on how to estimate minimum detectable effects for this design are still very limited. This paper aims to address the limitations of existing research on this topic by: (1) deriving expressions for the variance of the various C-SITS estimators; and (2) providing plausible values for the critical variance parameters calculated using school-level test scores from state assessments. The study is ongoing at the time of this publication. Preliminary findings show that when appropriate plausible values are entered into the formulae provided in this paper, there is a close correspondence between the standard errors estimated from the formulae and those obtained from fitting models to actual data. The formulae and plausible values in this paper provide analysts with a flexible basis for estimating MDES for various C-SITS designs, and serves as a template for accumulation of relevant information that can be used to build databases of plausible values that inform the designs of future studies. Tables and figures are appended.

Education needs generalizable models to scale up evidence-based practices and programs and longitudinal research evaluating the persistence of the effect of their implementation. This is particularly important given the "deep, systemic incapacity of U.S. schools, and the practitioners who work in them, to develop, incorporate, and extend new ideas about teaching and learning in anything but a small fraction of schools and classrooms." The authors synthesized research to create a model called TRIAD (Technology-enhanced, Research-based, Instruction, Assessment, and professional Development) for the scale of successful interventions. The specific goal of the implementation of the TRIAD model was to increase math achievement in young children, especially those at risk, by means of centering aspects of the curriculum--mathematical content, pedagogy, technology, and assessments--on a common core of learning trajectories. Results indicate that TRIAD was effective during the treatment period. The TRIAD implementation included a complete intervention in pre-K, and the impact was strong at pre-K. The TRIAD follow-through component was important for persistence of effects. The fact that at least some effects last for African-American children 3 years after the end of all interventions support the need for continued follow through interventions in mathematics for at-risk children. Tables and figures are appended.

In experimental studies, researchers are often interested in secondary research questions that explore important aspects of main findings, such as whether or not program effects vary according to the level of fidelity in which the program has been implemented; or according to the dosage received by individuals participated in the program. In this evaluation, stakeholders questioned whether higher fidelity of implementation was related to greater impact of the intervention on student outcomes. This paper addresses this question and compares the methodology and results of three approaches for linking implementation fidelity to impacts. Exhibits are appended.

Many educational Randomized Controlled Trials (RCTs) collect baseline versions of outcome measures (pretests) to be used in the estimation of impacts at posttest. Although pretest measures are not necessary for unbiased impact estimates in well executed experimental studies, using them increases the precision of impact estimates and reduces sample size requirements to detect desired effect sizes. This paper provides a theoretical overview of the late pretest issue and empirically assesses the bias-precision tradeoff in the experimental evaluation of the efficacy of adding a curriculum designed to support the development of children's self-regulation skills (Scaffolded Self-Regulation, SSR) to an established early mathematics curriculum (Building Blocks, BB) to form a synthesized curriculum, BBSSR. The RCT described in this paper is conducted in three large districts in Southern California. The analytic sample includes 807 students who participated in both baseline and posttest data collection. These students were in 84 4-year-old classrooms across the three districts. The authors analyzed a three-armed cluster RCT in which classrooms in early childhood centers or schools were randomized to the two intervention conditions and a business-as-usual control condition. Randomization was conducted separately for schools with one or two participating classrooms (groups A and B). Schools/centers in group A were placed into five randomization blocks such that each block consisted of all half-day or full-day PreK classrooms in one of the three study districts. Within each block, schools/centers were sorted with respect to prior math achievement, % reduced price lunch eligible, and %ELL and randomly assigned to the three conditions three at a time, starting with a randomly chosen point in the sorted list. This process was used to ensure balanced experimental groups. In each group B center/school, the two study classrooms were randomly assigned to two conditions that were randomly determined. Analyses conducted thus far show that controlling for contaminated pretest measures in impact regressions can lead to substantially biased impact estimates while their effect on the precision of impact estimates is less profound, causing one to question the overall merit of using them. Figures and exhibits are appended.

The authors' goal is twofold. First, they wished to produce a theoretically-based approach to this synthesis. Child-centered programs have a long history. However, concerns about children's achievement, and the pressure of content-specific standards, have set up a perceived conflict, in which educators believe they are being asked to abandon child-centered approaches, or, at least, to compromise and squeeze in, as one teacher put it, "Literacy on Monday-Wednesday-Friday, math on Tuesday-Thursday, and socio-emotional during our shortened play periods." They hope that their approach, if shown to be efficacious, will serve as a model that others can use to successfully and synergistically combine these strategies so the whole is more than the sum of its integrated, not conflicting, parts. Second, and more importantly, they are producing a rigorous evaluation of the efficacy of this approach. The research is also designed to answer which components are responsible for its effects, and why these components led to the outcomes. Thus, they are empirically testing their hypotheses, which will provide an evidentiary basis for researchers and practitioners. This report is the first (and only partial) description of their findings. (Contains 1 footnote.)

The proposed paper studies the bias in the two-stage least squares, or 2SLS, estimator that is caused by the compliance-effect covariance (hereafter, the compliance-effect bias). It starts by deriving the formula for the bias in an infinite sample (i.e., in the absence of finite sample bias) under different circumstances. Specifically, it considers the following cases: (1) A single site study with one mediator; (2) A multiple site study with one mediator; and (3) A multiple site study with multiple mediators. The formulas demonstrate how the magnitude of the compliance-effect bias varies with different parameters (e.g., compliance-effect correlation, mean and variance of the compliance and effect) in infinite samples. However, as the situation under consideration gets more complicated, the bias formula quickly becomes intractable. The second part of the paper, therefore, uses simulations to demonstrate the relationship between the compliance-effect bias and various parameters, as well as the behavior of the estimated 2SLS standard errors. Furthermore, the simulation exercise assesses how the compliance-effect bias interacts with the finite sample bias when the analysis sample is small or when the instrument is weak. The paper also uses simulations to compare the properties of the 2SLS estimator with those of the ordinary least squares (OLS) estimator in the presence of the compliance-effect bias, the finite sample bias, and the omitted variable bias. (Contains 6 footnotes.)

To address concerns about low graduation rates and a lack of workforce readiness, states and districts have been experimenting with different models of high school reform. One of the most popular models has been the Early College High School (ECHS) model, small schools that blur the line between high school and college. Since 2002, over 200 ECHSs have been created under the auspices of the Early College High School Initiative, which is primarily funded by the Bill & Melinda Gates Foundation. North Carolina has the largest concentration, with over 70 ECHSs across the state. The purpose of this study is to rigorously examine the implementation and impact of the Early College High School model in North Carolina. This study is the first to utilize a longitudinal experimental design to assess the impact of the ECHS model on student outcomes. The study has three main goals: (1) Determine the impact of the model on selected student outcomes; (2) Determine the extent to which impacts differ by student characteristics; and (3) Examine the implementation of the model by site and the extent to which variation in specific model components is associated with impacts. Nineteen ECHS in North Carolina are participating in the study. Early results from this study show that the Early College High School model is having a positive impact on many outcomes associated with remaining in school and becoming ready for college. These results suggest that the ECHS is making substantial progress towards its goal of graduating more students who are ready for college and work. Although these findings are very positive, they are restricted to the model as implemented in North Carolina. Other early colleges around the country may not follow the same design principles and may not receive the same level of assistance in implementation as the schools in North Carolina; as a result, their results may differ. (Contains 1 figure and 4 tables.)

"The No Child Left Behind Act of 2001" (PL 107-110) established the Reading First Program (Title I, Part B, Subpart 1), a major federal initiative designed to help ensure that all children can read at or above grade level by the end of third grade. Reading First (RF) builds upon research indicating that effective reading instruction on specific early reading skills, particularly in the primary grades, significantly reduces the number of students who experience reading difficulties in later years. This study was designed to answer two broad sets of questions, each focusing on a different aspect of sustainability of the program. These policy questions are critical to the ongoing conversations about the directions federal education policy should take to close achievement gaps and promote high achievement. The first set of questions is the following: "What are states' planned responses to the Reading First budget reduction?" and "Which RF program elements do state-level staff believe can be sustained beyond Reading First?" These questions address the planned and likely sustainability of the program from the perspective of state-level program administrators. The second set of questions is the following: "Does student achievement in later grades differ for students exposed to Reading First in K-3 compared with students in Title I Schoolwide Program (SWP) schools?" and "Is there evidence that the effect of RF spilled over to students in non-RF Title I SWP schools in RF districts?" These questions address the sustainability of program effects on students by examining patterns of student performance after students have completed the targeted grades. Findings are embargoed pending release of report results by the U.S. Department of Education (anticipated early 2011). (Contains 1 footnote.)

This report, which has been prepared by Abt Associates for the Institute of Education Sciences' National Center for Education Evaluation and Regional Assistance, takes an important first step in sorting out the implications of relying on state tests for general, student-level measures of reading and math achievement in evaluations of educational effectiveness. More specifically, this report provides empirical evidence that may help evaluation designers decide whether to rely on state tests to measure student achievement in studies of educational effectiveness. This study is designed to inform some of the choices that evaluation designers face with respect to state and study-administered standardized tests for a broad but well-defined class of study designs. This class, which covers many IES-sponsored evaluations, has two key defining features: (1) randomization (at either the student or cluster level) and (2) measurement of student achievement at two points in time--at "baseline" (prior to the implementation of the intervention) and (2) at "follow-up" (at least one point after the intervention has been implemented). (Contains 1 exhibit.)

This paper addresses methodological issues arising from an experimental study of North Carolina's Early College High School Initiative, a four-year longitudinal experimental study funded by Institute for Education Sciences. North Carolina implemented the Early College High School (ECHS) Initiative in response to low high school graduation rates. The goal of the initiative is to increase the number of students graduating from high school and who continue on and succeed in college. The study has three main goals: (1) Determine the impact of the model on selected student outcomes, including course-taking patterns, achievement, attitudes, and dropout and leaving rates; (2) Determine the extent to which outcomes differ by student characteristics; and (3) Examine the implementation of the model and the extent to which specific model components are associated with positive outcomes. Schools participating in the study identify an eligible pool of student applicants. The research team then randomly assigns students to either the treatment group (attending the ECHS) or the control group (business as usual). The outcomes for students in the two groups are then tracked and compared. The study follows an intent-to-treat model in that once a student is assigned to ECHS, he or she remains in the treatment group regardless of whether he or she ends up enrolling in ECHS, or leaves ECHS. Findings suggest that ECHS has the same impact on pass-rates for the Always-takers as the traditional high school. Where the ECHS are making difference is by increasing the number of students who are taking these courses. (Contains 5 figures, 6 tables and 4 footnotes.)

The purpose of this study is to rigorously examine the implementation and impact of the Early College High School (ECHS) model in North Carolina. The primary goal of the ECHS model is to increase the number of students who graduate from high school and who continue on and succeed in college. Therefore, the anticipated long-term outcomes for the program include increased high school graduation rates, increased enrollment in college, and increased graduation from college. To track progress toward those long-term outcomes, the conceptual framework identifies intermediate outcomes that are associated with continued enrollment in high school and enrollment in and success in college. Highlighting early findings from both the impact and implementation results, the authors found that schools are implementing the Design Principles as intended, where students are reporting higher levels of support and interactions with school staff. According to the logic model, this should lead to higher course ECHS students have high expectations of college attendance and frequents college facilities. ECHS students also take more Algebra courses than the control group, indicating early positive impacts on ECHS. However, the authors are finding limited impacts on student's engagement, expectations, and attitudes towards school work. (Contains 1 figure, 4 tables, and 2 footnotes.)

The Reading First Program is a central element of the No Child Left Behind legislation (No Child Left Behind Act, 2001). It builds on findings reported in a national consensus report (NICHD, 2000), about proven strategies to reduce the prevalence of reading difficulty in the early grades in order to help children read at or above grade level by the time they reach third grade. The program provides resources to schools for professional development, for purchasing reading programs/materials, for assessments of students, and for salaries of reading coaches who can help classroom teachers. Reading First departs from historical precedent in its reach into the content of instruction. Traditionally, the federal education agency has been focused on issues of access, and determination of actual curriculum content has been left to states and localities. The legislation contained a savvy blend of federal guidance for local decision-making; it did not endorse any reading materials or professional development or assessments, but rather articulated the procedures states and districts were required to use as they made local decisions about which districts and schools to fund as well as which specific tools those districts and schools selected help to implement the program. The Reading First Impact Study was designed to assess the impact of the Reading First Program on two core domains: classroom reading instruction and student reading achievement. This paper will summarize the methods used and the study's findings on the two questions of program impact. (Contains 4 exhibits and 2 footnotes.)