Search

Most studies of test-based promotion policies focus on measuring the effect of retention (being left back) on later student outcomes, and the evidence is fairly mixed. However, test-based promotion policies do not only affect the students who are retained. Presumably, they also affect students and schools as they try to improve reading performance in order to avoid being retained or having to retain students. The threat of retention could plausibly have either positive or negative effects on students within the gateway grade. On the one hand, the pressure to score above a particular threshold on a standardized test might backfire by overwhelming both students and schools. On the other hand, test-based promotion policies might incentivize students and schools to make academic improvements within the targeted grade in order to avoid retention. The effect of test-based promotion policies on student performance prior to the retention decision has not been studied enough. Filling this hole in the literature is important in order to understand the full impact that these policies have on students within a given jurisdiction. Even if relatively few students are actually retained under these policies, many students are in danger of scoring below the benchmark when they enter the gateway grade in the fall, and thus might be motivated by the policy to do better during the year than they would have otherwise. Therefore, even a small effect on students within the gateway grade could have a larger overall effect on student learning in a school system. This study applies a difference-in-difference design to statewide longitudinal school-by-grade data from two states (Florida and Arizona) and to longitudinal student-level data from a large public school district (Hillsborough County, Florida) to investigate the effect of introducing a third-grade test-based promotion requirement on third-grade test scores. It measures whether there was an increase in third-grade test scores that occurred in the policy's first year relative to scores in other grades within the school that were not directly targeted by the policy. It finds evidence that enacting the policy led to a statistically significant and meaningful increase in average third-grade test scores in both states. The magnitude of the effect is very similar across the two states, despite their being enacted nearly a decade apart, as are the differences in the policy details.

Previous research suggests that there are academic benefits when students and teachers share the same race/ethnicity because such teachers can serve as role models, mentors, advocates, or cultural translators. In this paper, we obtain estimates of achievement changes as students are assigned to teachers of different races/ethnicities from grades 3 through 10 utilizing a large administrative dataset provided by the Florida Department of Education that follows the universe of test-taking students in Florida public schools from 2001-02 through 2008-09. We find small but significant positive effects when black and white students are assigned to race-congruent teachers in reading (0.004 to 0.005 standard deviations) and for black, white and Asian/Pacific Island students in math (0.007 to 0.041 standard deviations). We also examine the effects of race matching by students' prior performance level, finding that lower-performing black and white students appear to particularly benefit from being assigned to a race-congruent teacher. An appendix provides a test of the authors' hypothesis and supplemental tables.

Public school teachers in the United States are famously difficult to dismiss. The reason is simple: after three years on the job, most receive tenure--after a brief and subjective evaluation process in which few receive negative ratings. In recent years, some school districts have experimented with changes in tenure rules. They seek the power to remove ineffective teachers and, in some jurisdictions, to reevaluate teachers throughout their careers. A keystone of this reform movement is the replacement of subjective evaluation with quantifiable measures of each teacher's effectiveness. The quantitative method is known as value-added modeling (VAM), a statistical analysis of student scores that seeks to identify how much an individual teacher contributes to a pupil's progress over the years. The use of VAM in teacher evaluations is growing, but the method remains extremely controversial. Critics often claim that it does not and cannot measure actual teacher quality. This paper addresses that claim. Part I analyzes data from Florida public schools to show that a VAM score in a teacher's third year is a good predictor of that teacher's success in his or her fifth year. Having established that VAM is a useful predictive tool, Part II of the paper addresses the most effective ways that VAM can be used in tenure reform. (Contains 9 figures, 2 tables and 8 endnotes.)

In a major address and policy white paper on education, Massachusetts governor Mitt Romney proposed to allow students enrolled in special education programs to use the federal dollars tied to their disability to attend a public, charter, or (when permitted by state law) private school of their choice. The basic principle behind Romney's plan--that the dollars spent to educate a child should be portable as students move to their preferred school--is a powerful one with the potential to fundamentally improve the relationship between students and their schools. Of course, special education is primarily a state and local responsibility and any president's policies will have effect only at the margins. If enacted, Romney's proposed policy could set an important precedent that further pushes states in a direction many of them have drifted over the last decade. At worst, Romney's plan would better utilize the relatively small amount of federal resources in education. At best, this policy might encourage states to buttress the effectiveness of federal reform by allowing state and local education dollars to follow students as well. Florida's decade-long experience with special education vouchers suggests that Romney's proposal would indeed be effective policy. The details for Romney's proposal are described in this Issue Brief. (Contains 14 endnotes.)

State and municipal policymakers are increasingly addressing the practice of social promotion in schools--moving children along to the next grade whether or not they have mastered the curriculum--by mandating test-based grade promotion. This paper draws conclusions about the effects of a policy limiting social promotion. To do so, it employs a methodology known as regression discontinuity, which is capable of producing causal estimates of policy effects to study the impact of Florida's test-based promotion policy on later student achievement. Under this program, students must take an exam to automatically pass from third to fourth grade (some students scoring below the automatic promotion threshold may still advance at teacher discretion). Students who are retained in third grade also receive a rigorous remediation regime aimed at improving their long-term performance. By studying the long-term performance of children who just barely passed the test, as well as those who were just barely left behind, it was possible to compare two essentially identical populations: one set of students who moved forward despite only borderline understanding of the material; and another set who stayed behind a year and received tutoring, mentoring, and other remedial interventions. On average, the students who were remediated did better academically, in both the short and long term, than those who were promoted. Tellingly, the benefits of the remediation were still apparent and substantial through the seventh grade (which is as far as the data can be tracked at this point). These results contrast with previous work cited by supporters of social promotion finding that grade retention has strong negative consequences for the student's later academic outcomes. This paper takes the view that there is considerable reason to question the validity of much of that research because most prior studies on grade retention use methods that are flawed or inadequate. Notably, these studies do not take into account "unobserved differences" between students studied. Unobserved differences are characteristics, such as maturity level or home environment, that aren't accounted for in the researchers' datasets, but which may have an enormous bearing on student performance. The results of this study demonstrate that a test-based promotion policy structured similar to Florida's policy should be expected to improve student performance relative to a policy of social promotion. Florida's system is an example for policy makers across the country to emulate. (Contains 6 figures, 2 tables and 10 endnotes.)

Given the challenges facing American public education today, identifying effective teachers is a more vital task than ever before. In the U.S. public school system today, the method used to determine teacher effectiveness--and thus to drive salary, promotion, and tenure decisions--is based on a few external credentials: certification, advanced degrees, and years of experience in the classroom. Yet according to a new analysis of student performance in Florida that two colleagues and the author conducted, little to no relationship exists between these credentials and the gains that a teacher's students make on standardized math and reading exams. The expansive study included all test-taking public elementary school students in the state of Florida over a period of four years. This study, to be published in the peer-reviewed journal "Economics of Education Review," builds on two decades of research from a variety of school systems and confirms a consistent finding: external teacher credentials tell next to nothing about how well a teacher will perform in the classroom. Along with coauthors Jay Greene and Bruce Dixon (professors at the University of Arkansas), the author of this Issue Brief utilized a data set provided by the Florida Department of Education to study the relationship between elementary school teachers' training and experience, on the one hand, and the learning gains made by their students in a given year, on the other. (Contains 10 endnotes.)

In 2006-07, New York City, the largest school district in the United States, decided it would follow several other school systems in adopting a progress report program. Under its program, the city grades schools from A to F according to an accumulating point system based on the weighted average of measurements of school environment, students' performance, and students' academic progress. The implementation of these progress reports has not been without controversy. While many argue that they inform parents about public school quality and encourage schools to improve, others contend that grades lower morale at low-performing schools. To date there has been too little empirical information about the program's effectiveness to settle these questions. This paper incorporates student-level data in a regression-discontinuity design to study the impact of a school's receipt of a particular grade--A, B, C, D, or F--on student proficiency in math and English one year later. The main findings of the paper are as follows: (1) Students in schools earning an F grade made overall improvements in math the following year, though these improvements occurred primarily among fifth-graders; and (2) Students in F-graded schools did no better or worse in English than students in schools that were not graded F. The paper is divided into six sections. Following an introduction, Section 2 provides a brief overview of previous research evaluating a similar progress report program in Florida. Section 3 discusses the design of New York City's policy. In order to give results on the relative improvement of D- and F-graded schools greater context, the paper presents some information about overall school progress in New York City in Section 4. It then devotes Section 5 to discussing methodology and data. The paper reports results from estimation in Section 6, including a replication of results obtained by Rockoff and Turner (2008), who employed a similar design to study New York's policy but used aggregate data. Section 7 states the conclusions. (Contains 6 tables and 5 endnotes.)

School systems across the nation have adopted policies that reward or sanction particular schools on the basis of their students' performance on standardized math and reading tests. One of the most frequently raised concerns regarding such "high-stakes testing" policies is that they oblige schools to focus on subjects for which they are held accountable, but to neglect the rest. Many have worried that the limited focus of these policies could have an unintended negative effect on student proficiency in other subjects, such as science, that are important to the development of human capital and thus to future economic growth. This paper evaluates whether the F-grade sanction in Florida's A+ program has led schools to increase student learning in the high-stakes subjects of math and reading to the detriment of learning in the important, but low-stakes subject of science. Results indicate that the F-grade sanction led to substantial student gains in the learning of math, reading, and science. Finally, a simple model is produced to explain the impact of high-stakes testing on student learning in low-stakes subjects. Evidence is provided suggesting that virtually all the positive findings in science are attributable to complementarities in the learning of math and reading. (Contains 2 tables and 3 endnotes.)

This paper evaluates the impact of exposure to a voucher program for disabled students in Florida on the academic performance of disabled students who remain in the public school system. The authors utilize student-level data on the universe of public school students in the state of Florida from 2000-01 through 2004-05 to study the effect of the largest school voucher program in the United States, the McKay Scholarship Program for Students with Disabilities (McKay), on achievement in math and reading by students who have been diagnosed as disabled and remain in the public school system. The report is an empirical evaluation of the impact of exposure to a voucher program designed to allow students with disabilities to enroll in schools other than their local public schools on the achievement of disabled students who remain in their local public schools. Vouchers for disabled students are the fastest-growing type in the United States. Programs similar to McKay are currently operating in Ohio, Georgia, and Utah and have been recently considered by other states. Highlights of the study include: (1) Public school students with relatively mild disabilities made statistically significant test score improvements in both math and reading as more nearby private schools began participation in the McKay program, contrary to the hypothesis that school choice harms students who remain in public schools; (2) Disabled public school students' largest gains as exposure to McKay increased were made by those diagnosed as having the mildest learning disabilities; and (3) Academic proficiency of students diagnosed with relatively severe disabilities was neither helped nor harmed by increased exposure to the McKay program. (Contains 7 endnotes and 6 tables.)

Social promotion has long been the normal practice in American schools. Critics of this practice, whereby students are promoted to the next grade regardless of academic preparation, have suggested that students would benefit academically if they were made to repeat a grade. Supporters of social promotion claim that retaining students (i.e, holding them back) disrupts them socially, producing greater academic harm than promotion would. A number of states and school districts, including Florida, Texas, Chicago, and New York City, have attempted to curtail social promotion, by requiring students to demonstrate academic preparation on a standardized test before they can be promoted to the next grade. This study analyzes the effects of Florida's test-based promotion policy on student achievement two years after initial retention. It builds upon our previous evaluation of the policy in two ways. First, we examine whether the initial benefits of retention observed in the previous study continue, expand, or contract in the second year after students are retained. Second, we determine whether discrepancies between our evaluation and the evaluation of a test-based promotion policy in Chicago are caused by differences in how researchers examined the issue, or by differences in the nature of the programs. Our analysis shows that, after two years of the policy, retained Florida students made significant reading gains relative to the control group of socially promoted students. These academic benefits grew substantially from the first to the second year after retention. That is, students lacking in basic skills who are socially promoted appear to fall further and further behind over time, whereas retained students appear to be able to catch up on the skills they are lacking. Further, we find these positive results in Florida, both when we use the same research design that we used in our previous study, and when we use a design similar to that employed by the evaluation of the program in Chicago. The differences between the Chicago and Florida evaluations appear to be caused by differences in the details of the programs, and not by differences in how the programs were evaluated. (Contains 5 tables and 10 footnotes.)

Social promotion has long been the normal practice in American schools. Critics of this practice, whereby students are promoted to the next grade regardless of academic preparation, have suggested that students would benefit academically if they were made to repeat a grade. Supporters of social promotion claim that retaining students disrupts them socially, producing greater academic harm than promotion would. A number of states and school districts have attempted to curtail social promotion, by requiring students to demonstrate academic preparation on a standardized test before they can be promoted to the next grade. This study analyzes the effects of Florida's test-based promotion policy on student achievement two years after initial retention. It builds upon a previous evaluation of the policy in two ways. First, the authors examine whether the initial benefits of retention observed in the previous study continue, expand, or contract in the second year after students are retained. Second, the researchers determine whether discrepancies between their evaluation and the evaluation of a test-based promotion policy in Chicago are caused by differences in how researchers examined the issue, or by differences in the nature of the programs. Analysis shows that, after two years of the policy, retained Florida students made significant reading gains relative to the control group of socially promoted students. These academic benefits grew substantially from the first to the second year after retention. That is, students lacking in basic skills who are socially promoted appear to fall farther behind over time, whereas retained students appear to be able to catch up on the skills they are lacking. Further, these positive results in Florida were found both when we use the same research design that used in the previous study and when using a design similar to that employed by the evaluation of the program in Chicago. The differences between the Chicago and Florida evaluationsappear to be caused by differences in the details of the programs, and not by differences in how the programs were evaluated. (Contains 10 endnotes and 5 tables.)

This report shows that New York can reduce special ed costs and enrollment--and improve parental satisfaction with the program as part of the bargain--by adopting two simple reforms: changing the formula funding special education from a "bounty" system to a "lump-sum" system; and implementing a voucher program for children in special education. The authors note that the problem of increased number of children in special-ed is largely a self-inflicted one. There is little evidence to support contentions that increased disability rates are to blame. Indeed, the special education category which has grown the most in the 1990s, Special Learning Disability (SLD), is the one with the fewest objective standards governing its diagnosis, and hence the one most open to unintentional manipulation. The authors also examined the results of Florida's McKay Scholarship Program, which, gives parents of every special education child the choice to take a voucher for 100 percent of the amount the state spends on that child and use it at a private school. The authors cite research showing that parents who use a McKay voucher are overwhelmingly satisfied with their child's private school experiences. The authors conclude that adopting these two reforms--changing the special education funding formula to a lump-sum system and creating a universal special-ed voucher program--will improve New York's educational outcomes and save taxpayers money. (Contains 11 figures and 24 notes.) [This Special Report was written with Greg Forster.]

This study evaluates the effect that the size of a state's school districts has on public high school graduation rates. The authors calculate the graduation rate over the last decade and examine the relationship between these graduation rates and changes in each state's average school district size. The study finds that decreasing the size of school districts has a substantial and statistically significant positive effect on graduation rates. Conversely, consolidation of school districts into larger units leads to more students dropping out of high school. The results of the analysis indicate that decreasing the average size of a state's school districts by 200 square miles leads to an increase of about 1.7 percentage points in its graduation rate. This finding is particularly important for states with very large school districts. For example, if Florida decreased the size of its school districts to the national median, it would increase its graduation rate from 59% to 64%. Decreasing the size of school districts could improve educational outputs, including graduation rates, because it would increase the choice that parents have in the school system that educates their child. By making it easier to relocate from one school system's jurisdiction to the next, smaller school districts make it possible for a larger number of families to exercise choice among different school districts. The more families are able to move from district to district, the less students can be taken for granted by schools, which, for a variety of reasons, don't want to lose enrollment. This study provides empirical evidence that increasing the choice parents have in their child's school district contributes to higher public high school graduation rates. (Contains 4 tables and 23 endnotes.)

This study examined whether the existence or threat of competition would cause public schools to improve, focusing on Florida's A+ Program, which combined educational vouchers and high stakes testing. The theory behind the program is that chronically failing public schools will have an incentive to improve if they must compete with other schools for students and the funding they generate. The study identified five categories of low-performing schools based on the degree of threat each school faced from voucher competition: voucher eligible schools, voucher threatened schools, formerly threatened schools, and two categories of similarly low-performing schools not facing any immediate threat of voucher competition. It examined test score improvements on the Florida Comprehensive Assessment Test and the Stanford-9 to see whether low-performing schools facing a greater degree of threat from voucher competition made more improvements than low-performing schools facing a lesser degree of threat from vouchers. Results indicate that Florida's low-performing schools are improving in direct proportion to the challenge they face from voucher competition. Schools already facing voucher competition show the greatest improvements of all five categories. Threatened schools show the second greatest improvements. An appendix offers 14 tables. (Contains 13 references.) (SM)

Many states have implemented high-stakes testing since the enactment of the No Child Left Behind Act of 2001. Yet the question remains whether high-stakes tests effectively measure student proficiency. This report describes a study that compared results on high-stakes tests with results on other standardized tests not used for accountability purposes and thus considered low-stakes tests. Data for the comparisons were gathered from test scores from 5,587 schools in 9 school systems in 8 states. Scores were compared on each test given in the same subject in the same school year. When possible, the results of high-stakes and low-stakes tests given at the same grade levels were also compared. For all the school systems examined in the study, high correlations between score levels on high-stakes and low-stakes tests were found. Also found were some high correlations for year-to-year gains in scores on high-stakes and low-stakes tests. But the correlations of score gains were not as consistently high, and in some places were quite low. The report concludes that stakes of the tests do not distort information about the general level at which students are performing. (Contains 10 tables and 23 references.) (WFA)

Many American states require that students lacking basic reading proficiency after third grade be retained and remediated. We exploit a discontinuity in retention probabilities under Florida's test-based promotion policy to study its effects on student outcomes through high school. We find large positive effects on achievement that fade out entirely when retained students are compared to their same-age peers, but remain substantial through grade 10 when compared to students in the same grade. Being retained in third grade due to missing the promotion standard increases students' grade point averages and leads them to take fewer remedial courses in high school but has no effect on their probability of graduating.