Search

This policy brief, a quick look at some aspects of the debate, illustrates the differences in New York City public schools that would result when layoffs are determined by seniority in comparison to a measure of teacher effectiveness. Due to data limitations and an interest in simplicity, this analysis employs the value added of teachers using the 4th and 5th grade math and ELA (English Language Arts) achievement of their students. Unsurprisingly, the authors find that layoffs determined by a measure of teacher effectiveness result in a more effective workforce than would be the case with seniority-based layoffs. However, they were surprised by facets of the empirical results. First, assuming readily available measures of teacher effectiveness actually measure true teacher effectiveness, an assumption to which they return in this brief, the differences between seniority and effectiveness-based layoffs are larger and more persistent than they anticipated. Second, even though seniority-based layoffs imply laying off more teachers, the differential effect on class size is very small in their simulations, though it would be larger for larger budget reductions. Third, there is a somewhat greater school-level concentration of layoffs in a seniority-based system, though with a few notable exceptions, both methods result in fairly dispersed layoffs, with the vast majority of schools having no more than one layoff in grades four and five combined. As a result of the limited applicability of teacher value-added measures to the full population of teachers as well as concerns about potential mismeasurement of effectiveness associated with using value-added measures even when available, neither seniority nor measures of value added to student achievement should be the sole criterion determining layoffs. However, ignoring effectiveness measures completely, as seniority-based systems do, is also problematic. Instead, the use of multiple measures of effectiveness for layoff decisions holds promise for softening the detrimental effect of layoffs. Teacher Value-Added Estimation is appended. (Contains 1 figure and 9 notes.)

Even as research documents that teachers matter, there is less certainty about the attributes of teachers that make the most difference in raising student achievement. Many studies have estimated the relationship between teachers' characteristics (i.e., experience and academic performance) and their value-added to student achievement. Few have explored whether instructional practices predict student test score gains. In this study, the authors ask what classroom practices, if any, differentiate teachers with high impact on student achievement in middle school English Language Arts from those with lower impact. The study further explores the extent to which value-added measures signal differences in instructional quality. Even with the small sample used in the analysis, the authors find evidence that high value-added teachers have a different profile of instructional practices than do low value-added teachers. Teachers in the top quartile as measured by value-added scores score higher than second-quartile teachers on all 16 elements of instruction that were measured. The differences are statistically significant for a subset of practices including explicit strategy instruction. (Contains 4 footnotes, 6 figures, and 5 tables.)

This paper analyzes attrition patterns among teachers in New York City public elementary and middle schools and explores whether teachers who transfer among schools, or leave teaching entirely, are more or less effective than those who remain. We find that the first-year teachers who are less effective in improving student math scores have higher attrition rates than do more effective teachers. The first-year differences are meaningful in size; however, the pattern is not consistent for teachers in their second and third years. Attrition patterns differ between schools having disproportionate numbers of low- vs. high-scoring students. A relatively high percentage of the ineffective first-year teachers in low-scoring schools leave teaching altogether; whereas inefficient first-year teachers in higher-scoring schools disproportionately transfer within NYC. In general, first-year teachers who transfer, on average, are less effective than their peers in their new schools, as was the case in their original schools. Furthermore, the more effective first-year teachers who transfer differentially move to schools with fewer low-scoring, poor, Black and Hispanic students, possibly contributing to achievement gaps. As discussed in the paper, these findings raise important questions about policies and policy proposals intended to reduce teacher attrition. Value-added analysis and estimation of attrition model are appended. (Contains 12 figures, 8 tables and 19 footnotes.) [This research was supported by the National Center for the Analysis of Longitudinal Data in Education Research (CALDER).]

There are fierce debates over the best way to prepare teachers. Some argue that easing entry into teaching is necessary to attract strong candidates, while others argue that investing in high quality teacher preparation is the most promising approach. Most agree, however, that we lack a strong research basis for understanding how to prepare teachers. This paper is one of the first to estimate the effects of features of teachers' preparation on teachers' value-added to student test score performance in Math and English Language Arts. Our results indicate variation across preparation programs in the average effectiveness of the teachers they are supplying to New York City schools. In particular, preparation directly linked to practice appears to benefit teachers in their first year. Sample Results for Math with Pathway/Institution Effects and Description of Variables are appended. (Contains 6 footnotes, 9 tables and 4 figures.) [Funding for this report was provided by the City University of New York.]

Value-added models in education research allow researchers to explore how a wide variety of policies and measured school inputs affect the academic performance of students. Researchers typically quantify the impacts of such interventions in terms of "effect sizes", i.e., the estimated effect of a one standard deviation change in the variable divided by the standard deviation of test scores in the relevant population of students. Effect size estimates based on administrative databases typically are quite small. Research has shown that high quality teachers have large effects on student learning but that measures of teacher qualifications seem to matter little, leading some observers to conclude that, even though effectively choosing teachers can make an important difference in student outcomes, attempting to differentiate teacher candidates based on pre-employment credentials is of little value. This illustrates how the perception that many educational interventions have small effect sizes, as traditionally measured, are having important consequences for policy. In this paper we focus on two issues pertaining to how effect sizes are measured. First, we argue that model coefficients should be compared to the standard deviation of gain scores, not the standard deviation of scores, in calculating most effect sizes. The second issue concerns the need to account for test measurement error. The standard deviation of observed scores in the denominator of the effect-size measure reflects such measurement error as well as the dispersion in the true academic achievement of students, thus overstating variability in achievement. It is the size of an estimated effect relative to the dispersion in the true achievement or the gain in true achievement that is of interest. Adjusting effect-size estimates to account for these considerations is straightforward if one knows the extent of test measurement error. Technical reports provided by test vendors typically only provide information regarding the measurement error associated with the test instrument. However, there are a number of other factors, including variation in scores associated with students having particularly good or bad days, which can result in test scores not accurately reflecting true academic achievement. Using the covariance structure of student test scores across grades in New York City from 1999 to 2007, we estimate the overall extent of test measurement error and how measurement error varies across students. Our estimation strategy follows from two key assumptions: (1) there is no persistence (correlation) in each student's test measurement error across grades; (2) there is at least some persistence in learning across grades with the degree of persistence constant across grades. Employing the covariance structure of test scores for NYC students and alternative models characterizing the growth in academic achievement, we find estimates of the overall extent of test measurement error to be quite robust. Returning to the analysis of effect sizes, our effect-size estimates based on the dispersion in gain scores net of test measurement error are four times larger than effect sizes typically measured. To illustrate the importance of this difference, we consider results from a recent paper analyzing how various attributes of teachers affect the test-score gains of their students (Boyd et al., in press). Many of the estimated effects appear small when compared to the standard deviation of student achievement--that is effect sizes of less than 0.05. However, when measurement error is taken into account, the associated effect sizes often are about 0.16. Furthermore, when teacher attributes are considered jointly, based on the teacher attribute combinations commonly observed, the overall effect of teacher attributes is roughly half a standard deviation of universe score gains--even larger when teaching experience is also allowed to vary. The bottom line is that there are important differences in teacher effectiveness that are systematically related to observed teacher attributes. Such effects are important from a policy perspective, and should be taken into account in the formulation and implementation of personnel policies. An appendix is included. (Contains 34 footnotes, 4 figures, and 9 tables.) ["Overview of Measuring Effect Sizes: The Effect of Measurement Error. Brief 2" (ED508284) was based on this report.]

Studies have found substantial sorting of teachers across schools, with the schools with the highest proportions of poor, non-white, and low-scoring students having the least qualified teachers as measured by certification, exam performance, and inexperience (Lankford, Loeb and Wyckoff, 2002). Yet, there have been substantial changes in the educational policy landscape over the past five years. New laws, including the No Child Left Behind Act of 2001 (NCLB), have changed requirements for teachers. Assessment-based accountability policies at the state-level have created standards and increased oversight of schools, especially those with low-achieving students. New routes into teaching, many with fewer requirements before teaching, have changed the cost for individuals to enter the teaching profession. These changes have affected teacher labor markets profoundly. In this paper the authors examine these changes, asking how the distribution of teachers has changed in recent years and what the implications of these changes are for students. They examine three questions: (1) How has the distribution of teaching qualifications between schools with concentrations of poor students and those with more affluent students changed over the last five years?; (2) What effects are the changes in observed teacher qualifications likely to have on student achievement?; and (3) What implications do these findings have for improving policies and programs aimed at recruiting highly effective teachers? This study uses data on New York City teachers, students, and schools to address these questions. The authors find that measurable characteristics of teachers are more equal across schools in 2005 than they were in 2000. Schools with large proportions of poor students and students of color, on average, have teachers whose observable qualifications are much stronger than they were five years ago. Nonetheless, a meaningful number of schools with large proportions of poor students did not demonstrate such improvement. Changes in these observed qualifications of teachers account for a modest improvement in the average achievement of students in the poorest schools. More importantly, the results suggest that recruiting teachers with stronger observed qualifications, e.g., math SAT scores or certification status, could substantially improve student achievement. The following are appended: (1) Average School Qualifications of Teachers by Percent of Students in School Who Are Black or Hispanic, 2000 and 2005; (2) Average School Qualifications of Teachers by Percent of Students in School Who Scored at Level 1 on 4th Grade ELA Exam, 2000 and 2005; (3) Average School Qualifications of Teachers in Elementary Schools by Student Poverty, 2000 and 2005; (4) Average School Qualifications of Teachers In Middle Schools by Student Poverty, 2000 and 2005; and (5) Average School Qualifications of Teachers in High Schools by Student Poverty, 2000 and 2005. (Contains 10 figures, 11 tables, and 10 footnotes.) [This working paper was published by the National Center for Analysis of Longitudinal Data in Education Research (CALDER). CALDER is a program of research by the Urban Institute with Duke University, Stanford University, University of Florida, University of Missouri-Columbia, University of Texas at Dallas, and University of Washington. It is supported by an Institute of Education Sciences (IES) grant to the Urban Institute.]

This paper explores a little-understood aspect of labor markets, their spatial geography. Using data from New York State, it finds teacher labor markets to be geographically very small. Teachers express preferences to teach close to where they grew up, and, controlling for proximity, they prefer areas with characteristics similar to their hometown. The paper discusses implications of these preferences for the successful recruitment of teachers, including the potential benefits of local recruiting and training. It also discusses implications for the modeling of teacher labor markets, including the possible biases that arise in estimates of compensating differentials when distance is omitted from the analyses. This study contributes to the literature on the geography of labor markets more generally by employing data on residential location during childhood instead of current residence, which may be endogenous to job choice. The final section of the paper summarizes findings on the geographic scope of teacher labor markets and examines their application to policy issues in the recruitment of teachers.(Contains 37 references and 10 tables/figures.) (SM)

Understanding what makes an effective teacher, as well as how teachers sort by their effectiveness across schools, is central to understanding and addressing student achievement gaps. Prior studies have found substantial sorting of teachers across schools, with the schools with the highest proportions of poor, non-white, and low-scoring students having the least qualified teachers as measured by certification, exam performance, and inexperience. Yet there have been substantial changes in the educational policy landscape over the past five years. In this paper, the authors examine these changes, asking how the distribution of teachers has changed in recent years and what the implications of these changes are for students. The authors address these questions using data on New York City teachers, students, and schools. While the findings may be specific to New York City, they may mirror changes in other large urban districts, many of which have seen similar policy changes over the past decade. The authors find that measurable characteristics of teachers are more equal across schools in 2005 than they were in 2000. Schools with large proportions of poor students and students of color, on average, have teachers whose observable qualifications are much stronger than they were five years ago. Nonetheless, a meaningful number of schools with large proportions of poor students did not demonstrate such improvement. The authors find that changes in these observed qualifications of teachers account for a modest improvement in the average achievement of students in the poorest schools. More important, the results suggest that recruiting teachers with stronger observed qualifications--for example, math SAT scores or certification status--could substantially improve student achievement. (Contains 6 tables, 9 figures, and 9 footnotes.)

We are in the midst of what amounts to a national experiment in how best to attract, prepare, and retain teachers, particularly for high poverty urban schools. Using data on students and teachers in grades three through eight, this study assesses the effects of pathways into teaching in New York City on the teacher workforce and on student achievement. We ask whether teachers who enter through new routes, with reduced coursework prior to teaching, are more or less effective at improving student achievement than other teachers and whether the presence of these alternative pathways affects the composition of the teaching workforce. Results indicate that in some instances the new routes provide teachers with higher student achievement gains than temporary license teachers, though more typically there is no difference. When compared to teachers who completed a university-based teacher education program, teachers with reduced course work prior to entry often provide smaller initial gains in both mathematics and English language arts. Most differences disappear as the cohort matures and many of the differences are not large in magnitude, typically 2 to 5 percent of a standard deviation. The variation in effectiveness within pathways is far greater than the average differences between pathways.

When states prepare their budgets, they usually base revenue and expenditure projections upon forecasts of national and state economic and demographic trends. This brief presents findings of a Center for the Study of the States survey that asked state budget offices what they were assuming for many key variables. The survey obtained 41 state forecasts, most of which were used to prepare official executive budget projections for 1996-97. The most commonly forecasted state-level variables included employment (39 states), nominal personal income (41), Medicaid caseloads (25), Aid to Families with Dependant Children (AFDC) caseloads (29), and prison populations (27). State budget-office forecasts of the national economy were slightly more conservative than those of private forecasters. The median budget forecast of 1996 real GDP growth was 2.4 percent, slightly below the comparable Blue Chip consensus of 2.6 percent. The states' median forecast of 1996 consumer price inflation was 2.8 percent, compared to the median Blue Chip forecast of 2.9 percent. States expect a significant slowdown in corporate profits over the next 2 years, with eight states expecting declines in 1996. States do not appear to suffer from a "provincial illusion": fast-growing states were not more optimistic about the national economy than slow-growing states. Finally, many states are expecting declines in AFDC caseloads in the next 2 years. More than 60 percent of reporting states expect a decline in 1997, with Delaware's anticipated decline of 22 percent by far the largest. Two tables and three figures are included. (LMI)