Search

The Massachusetts Early Childhood Support Organization (ECSO) initiative seeks to improve the quality of early education programs (EEPs) by supporting leaders to strengthen their organizational climate, provide job-embedded professional learning opportunities for educators, support the use of instructional curriculum and child assessments in their program, and engage in continuous quality improvement. This report presents findings from an implementation study conducted by Abt Associates, an independent research firm. The findings are based on data collected from ECSOs and participating EEPs in the 2021-2022 school year, including surveys, classroom observations, and administrative data. Although the findings are descriptive and changes over time cannot be attributed to the initiative because there is no comparison group, they show promise and warrant further study.

Classroom quality measures, such as the Early Childhood Environment Rating Scale, Revised (ECERS-R), are widely used in research, practice, and policy. Increasingly, these uses have been for purposes not originally intended, such as contributing to consequential policy decisions. The current study adds to recent evidence of problems with the ECERS-R standard stop-scoring by analyzing eight studies offering 14 waves of data collection in approximately 4,000 classrooms. Our analyses, which featured the nominal response model, generalized partial credit model, partial credit model, within-category averages of total scores, and point-biserial correlations, revealed that all 36 items had categories that did not follow an ordinal progression with respect to quality. Additionally, our results showed that the category problems accumulated to the scale score. The results caution against the use of the standard raw scoring and encourage development of alternative scoring methods for the ECERS-R. [This paper was published in "AERA Open" v4 n1 p1-16 Mar 2018.]

Many states have implemented or expanded state-funded prekindergarten programs in the last decade, encouraged by claims about the benefits that can be expected and the importance of early experiences for children's development, especially for economically disadvantaged children. However, there is remarkably little methodologically adequate evidence about the effects of such programs. Using a subsample of children with parental consent from a larger sample of children randomly assigned to attend the Tennessee pre-k program or not, this study examined effects on cognitive and noncognitive outcomes through third grade. At the end of the pre-k year, program participants showed better outcomes than comparable nonparticipants on achievement measures and ratings of school readiness by kindergarten teachers. But those effects were not sustained in subsequent years and, indeed, by the end of third grade the pre-k participants scored lower on the achievement measures than nonparticipants. These results raise questions about the way state pre-k programs have been designed and implemented.

Early mathematics knowledge is a strong predictor of later academic achievement, but children from low-income families enter school with weak mathematics knowledge. An Early Math Trajectories model is proposed and evaluated within a longitudinal study of 517 low-income American children from age 4 to 11. This model includes a broad range of math topics, as well as potential pathways from preschool to middle-grades mathematics achievement. In preschool, nonsymbolic quantity, counting and patterning knowledge predicted fifth-grade mathematics achievement. By the end of first grade, symbolic mapping, calculation and patterning knowledge were the important predictors. Further, the first-grade predictors mediated the relation between preschool math knowledge and fifth-grade mathematics achievement. Findings support the Early Math Trajectories model among low-income children. [At time of submission to ERIC this article was in press with "Child Development."]

In 2009, Vanderbilt University's Peabody Research Institute, in coordination with the Tennessee Department of Education's Division of Curriculum and Instruction, initiated a rigorous, independent evaluation of the state's Voluntary Prekindergarten program (TN- VPK). TN-VPK is a full-day prekindergarten program for four-year-old children expected to enter kindergarten the following school year. The program in each participating school district must meet standards set by the State Board of Education that require each classroom to have a teacher with a license in early childhood development and education, an adult-student ratio of no less than 1:10, a maximum class size of 20, and an approved age-appropriate curriculum. TN-VPK is an optional program focused on the neediest children in the state. It uses a tiered admission process, with children from low-income families who apply to the program admitted first. Any remaining seats in a given location are then allocated to otherwise at-risk children, including those with disabilities and limited English proficiency. The current report presents findings from this evaluation summarizing the longitudinal effects of TN-VPK on pre-kindergarten through third grade achievement and behavioral outcomes for an Intensive Substudy Sample of 1076 children, of which 773 were randomly assigned to attend TN-VPK classrooms and 303 were not admitted. Both groups have been followed since the beginning of the pre-k year.

Research Findings: The Early Childhood Environment Rating Scale, Revised (ECERS-R) is widely used, often to evaluate whether preschool programs are of sufficient quality to improve children's school readiness. We examined the validity of the measure for this purpose. Item response theory (IRT) analyses revealed that many items did not fit together to measure single dimensions, particularly when rated by consultants as indicating aspects of quality relevant for multiple domains of child development. IRT results also conflicted with scale developers' expectations in terms of whether markers that they attached to higher response categories represented higher quality empirically. When reanalyzed based on experts' ratings, IRT results also showed relatively few indicators captured the moderate to high range of quality. Practice or Policy: Our results suggest that policymakers should carefully consider whether measures designed for specific purposes are appropriate for other high-stakes uses. We encourage continued refinement of existing quality measures, development of new measures, and the accumulation of evidence for their various uses. [This paper was published in "Early Education and Development" (EJ1072524).]

This paper presents evidence on the amount of classroom variation in gains across various domains of achievement within the Tennessee Voluntary Pre-Kindergarten (TN-VPK) program; the data are taken from the randomized control trial (RCT) design of the TN-VPK evaluation. This study explores areas where those differences are largest, and offers speculations about possible causes of these differences. The full randomized sample included children from 111 randomized school applicant lists in 28 districts across Tennessee. The consented subsample of that group came from 76 applicant lists in 58 schools from 21 districts. Ten of those 58 schools were in the West region of the state, 24 were in the Central West region, 12 were in the Central East region, and 12 were in the East. Children were individually assessed using the set of Woodcock Johnson III achievement tests (Letter-Word Identification, Spelling, Understanding Directions, Applied Problems, Quantitative Concepts, Passage Comprehension, and Oral Comprehension). Data presented were analyzed through first calculating each individual child's gain by differencing the beginning and end of pre-k scores on each subscale, and those differences were aggregated to the classroom level for the children who attended VPK. The focus of this paper is not just on mean gain, but on standard deviations and ranges of gain. The amount of gain made during the pre-k year in TN-VPK classrooms varied substantially from classroom to classroom. Correlations of gains across subscales revealed that though classrooms that tended to make more relative gain on one subtest also tended to make relatively more gain on other subtests, the strength of the correlations did not suggest that classrooms making the most gains always made the most gains regardless of subject matter. Tables and figures are appended.

In 2009, Vanderbilt University's Peabody Research Institute, with the assistance of the Tennessee Department of Education's Division of Curriculum and Instruction, initiated a rigorous, independent evaluation of the state's Voluntary Prekindergarten program (TN- VPK). TN-VPK is a full-day prekindergarten program for four-year-old children expected to enter kindergarten the following school year. The program in each participating school district must meet standards set by the State Board of Education that require each classroom to have a teacher with a license in early childhood development and education, an adult-student ratio of no less than 1:10, a maximum class size of 20, and an approved age-appropriate curriculum. TN-VPK is an optional program focused on the neediest children in the state. It uses a tiered admission process with children from low-income families who apply to the program admitted first. Any remaining seats in a given location are then allocated to otherwise at-risk children including those with disabilities and limited English proficiency. The current report is the second in a series that presents findings from this evaluation. The prior report described outcomes at the end of the pre-k year for the children in the Intensive Substudy sample who participated in TN-VPK in comparison to those who did not participate. The present report summarizes the longitudinal effects of TN-VPK on kindergarten outcomes and those first grade outcomes that are currently available. At the end of prekindergarten, TN-VPK effects could be examined only on early achievement measures and teacher ratings of academic skills and behavior obtained at the very beginning of the kindergarten year. A notable addition in this current report is that results are now available on several other "non-cognitive" academic outcomes, including grade retention, attendance, recorded disciplinary actions, and special education services.

This report summarizes the results of a scale-up project funded by the Institute of Education Sciences in 2006. "Scaling up TRIAD: Teaching Early Mathematics for Understanding with Trajectories and Technologies" was a project that took a preschool mathematics intervention to scale across three sites, following children from their Prekindergarten year--in which the intervention was delivered--through first grade. This study was designed to explore the following research questions: (1) What are the immediate and long-term effects of the intervention on children's math skills?; (2) How much variation was there in effects across sites?; (3) Were curricular effects different for different subgroups of children?; and (4) What are the effects of the math environment and the fidelity of implementation on children's immediate and long term math gains, overall and across sites? [This research was conducted in partnership with Doug Clements and Julie Sarama at the University at Buffalo SUNY (recently moved to the University of Colorado).]

This study includes a randomized control experiment, but that component could only be implemented in a limited number of schools with more applicants than seats in the pre-k program. Informative as that component is about the effects of Tennessee Voluntary Pre-Kindergarten (TN-VPK), the participating schools do not provide a representative sample of the TN-VPK classrooms in Tennessee. To provide a more representative statewide picture, a stratified random sample of schools with TNVPK classrooms was drawn and enrolled in an age cutoff regression-discontinuity designs. This design was made possible because TN-VPK has explicit age requirements that are implemented as a strict age cutoffs for TN-VPK eligibility in all schools. Using the two designs concurrently, this study seeks to determine the statewide effect of TN-VPK on both the kindergarten readiness of the participating children and their long-term cognitive and behavioral skills, including performance on the state-administered achievement tests in the third grade. A secondary purpose is to investigate the relationships between those outcomes and the characteristics of the TN-VPK classrooms, e.g., the curriculum used, teacher credentials, and classroom quality, and structural support and monitoring at the district and school levels, detecting impact heterogeneity by these characteristics.

Relatively few rigorous studies of the effectiveness of contemporary public prekindergarten programs have been conducted despite the growing number of programs and large monetary investments they require. The study on which this presentation is based was launched in partnership with the Tennessee State Department of Education's Division of School Readiness and Early Learning to provide an assessment of the effects of the statewide Tennessee Voluntary Prekindergarten (TN-VPK) program on the readiness for kindergarten of the economically disadvantaged population it serves. Research studies have reported the rapid neurological, cognitive, and social-emotional growth that takes place during the first five years of life and the positive effects of high-quality early childhood educational programs. In recent years, however, TN-VPK has become a controversial program in Tennessee, with some legislators expressing doubts about its value in the context of severe budget shortfalls and still others referring to it even more skeptically as expensive babysitting. This study interleaves a longitudinal randomized control trial (RCT) and an age-cutoff regression discontinuity (RD) design to evaluate the effectiveness of the TN-VPK program. Though the project is still underway, this presentation will summarize results for two cohorts of RCT participants.

Studies show that the mathematics test-score gap is evident at every level of schooling and can be linked to students' earlier performance. For example, a mathematics performance gap was found in children as young as three years of age (Case & Griffin, 1990; Jordan, Huttenlocher, & Levine, 1992). Addressing the mathematics performance gap early on, before children start school, has therefore become a priority for preschool programs serving children from low-income backgrounds (Clements, 2004). The authors created a research-based model to meet this challenge in the area of mathematics, with the intent that the model generalize to other subject matter areas and other age groups. The specific goal of their implementation of the TRIAD (Technology-enhanced, Research-based, Instruction, Assessment, and professional Development) model was to increase math achievement in young children, especially those at risk, by means of a high-quality field-centered implementation of the Building Blocks math curriculum, with all aspects of the curriculum--mathematical content, pedagogy, teacher's guide, technology, and assessments--based on a common core of learning trajectories. The primary research question of interest is as follows: Do children who are exposed to the Building Blocks mathematics curriculum in preschool perform better on measures of mathematics skills through the end of first grade than do children who were not exposed to that curriculum? This scale-up intervention took place in preschool classrooms in three urban school districts: the Buffalo Public School system in Buffalo, New York, the Boston Public School system in Boston, Massachusetts, and a combination of the Metropolitan Nashville Public School system and the Metropolitan Action Council Head Start system in Nashville, Tennessee. Research has suggested that early curricular effects may fade over time, resulting in very little, if any, discernable difference in elementary school between students who had been exposed to a given curriculum prior to formal schooling and students who were not exposed to such a program, as those without early curriculum exposure "catch up" to their peers (Barnett et al., 1995). In the Preschool Curriculum Evaluation Research (PCER) project, across all 14 curricula, kindergarten effects were nonexistent, prompting a decision not to collect any further longitudinal data. Similarly, with the scale-up project, the authors saw evidence of curricular effects across outcomes at the end of prekindergarten, but very few differences at the end of kindergarten, and virtually none at the end of first grade. Longitudinal research, including follow through interventions in these grades, is needed to determine if these early gains truly "fade," or if, as the authors posit, the problem is that primary grade curricula and teachers do not build upon them.

The "Building Blocks PreK Math Curriculum" (Clements & Sarama, 2007) was designed to facilitate children's engagement in math and talk about math. Much research investigates the effect of curriculum on classrooms or teacher practices. This study used a mediational model to look at a curriculum's effect on children's achievement gain, operating through specific child behaviors in the classroom. Specifically, this study looked at how a math curriculum affected children's focus in math alone or in all learning activities (math, literacy, science, social-studies, and other), talking during math-related activities or in all learning activities, and engagement during math or during all learning activities. Additionally, this study examined how those child behaviors predicted children's math achievement gain. It is hypothesized in the existing literature that much of the variability in student achievement across prekindergarten programs can be explained by the amount of time children are engaged in learning through talking, listening, or sustained focus on academic content. Behaviors like a child's focus on instruction (Barr & Dreeben, 1983; NCES, 2002), verbal behaviors (Winsler & Naglieri, 2003; Dickinson & Tabors, 2001; Snow, Burns, & Griffin, 1998), and educational engagement (Brophy & Good, 1986; Howse, Lange, Farran, & Boyles, 2002) are all considered critical elements of learning. This study was based on the hypothesis that a curriculum which encourages teachers to focus on such critical elements in the classroom can lead to changes in child achievement if changes in children's behaviors are also affected. This study was conducted as part of the SUNY Buffalo/Vanderbilt scale-up of the "Building Blocks Prekindergarten Math Curriculum" in Nashville, Tennessee. Fifty-seven classrooms from twenty sites, 16 Public Schools and 4 Head Start centers, participated in one of two study conditions. Thirty-one classrooms participated in the new math curriculum while twenty-six classrooms conducted business as usual. Across both conditions, children were observed in their classrooms and during mealtimes on three typical days--once in fall, once in winter, and once in spring,--but not when they were outdoors. The "Building Blocks" curriculum, designed to facilitate children's engagement in math and talk about math, was predictive of children's participation in math-focused activities, which predicted higher gains on standardized math measures. Without the resulting change in children's behaviors in the classroom, however, changes in achievement would not have been possible through curriculum implementation alone. Although getting children to talk more about math was a goal of the curriculum, children's talk with a math focus did not predict their gain. It is possible that teachers in the treatment condition actually spent more time in learning related activities, but spent that time instructing rather than probing children's thinking or asking higher-order inferential questions. The results of this study demonstrate the difficult task of helping teachers promote an environment where children can investigate mathematics through talk and play, rather than through direct instruction. The results suggest that classrooms organized to keep children focused in learning activities will better equip children with math-readiness skills, and thereby improve early math education in large-scale public programs for children from low-income households. (Contains 4 tables.)

This research investigated the effects of two contrasting pre-k curricula, relative to practice as usual, on subsequent academic achievement. One curriculum had a strong literacy focus, the other was a less didactic "developmentally appropriate" curriculum that allowed children to have more influence on classroom activities. The purpose of the research was to determine if either of these curricula provided advantages for improving the academic performance of economically disadvantaged children in rural Tennessee. (Contains 3 tables.)