Search

Choosing a school for a child can be a rewarding, but difficult task for parents. A recent survey reported that parents often feel confused by the information available to them as part of the school choice process. This can be exacerbated by limited time and resources to devote to the process--constraints that can particularly affect low-income and otherwise disadvantaged families. If confusion results in uninformed decision-making then parents might not select schools that best fit their children's needs and the possible benefits of school choice could be undermined. There is a new demand for making school information available to families. The federal Every Student Succeeds Act (ESSA) passed in 2015 requires states and districts to provide public report cards that include information on state, district, and school performance and progress in an "understandable" and "user-friendly" manner. These report cards are likely to build on the school choice directories (online, on paper, or both) that many districts, states, and other education organizations already offer to help families navigate their options. This guide is a resource for those considering how to present information to parents ("information providers"). The recommendations in the guide are "evidence-based" because they emerged from a randomized study that is described below. The intended audience for the guide includes, for example, education officials responsible for deciding which information about schools to release publicly, along with designers responsible for translating that information into carefully formatted materials. Some information providers work in public agencies, like school districts or state departments of education, but many others work in nonprofit organizations or other private companies. The purpose of this guide is to help information providers--whoever and wherever they are--incorporate evidence into the many decisions involved in presenting information about schools. For the study, researchers worked with a firm specializing in presenting education data to create an online "information display" of individual "school profiles" for a hypothetical school district. Parents who participated in the study were each randomly assigned to review one variation of the information display--as if they were considering elementary school options for their youngest child--and then answered questions about the display and the schools it described. This type of experiment is commonly used for market research and by behavioral and social scientists to examine the effects of specific variations (e.g., color of a product, wording of instructions, etc.) in controlled environments. For this study, the research team compared responses to the questions for parents who saw different information displays to determine how the design of the displays affected key outcomes: parents' understanding of the school information, their satisfaction with the display and its perceived ease of use, and the choices they made when ranking the schools. 3,500 low-income parents were assigned to see one of 72 different web pages displaying information for a hypothetical district. All displays included profiles for 16 schools with information about distance from the parent's home, academics, safety, and resources. [For the study snapshot to this report "Presenting School Choice Information to Parents: An Evidence-Based Guide. Study Snapshot. NCEE 2019-4003," see ED589103. For the study highlights to this report "Presenting School Choice Information to Parents: An Evidence-Based Guide. Study Highlights. NCEE 2019-4003," see ED589106.]

More than ever, school districts and other organizations need to provide parents with information about school quality and their school choices. Across the country school choice is expanding in ways that provide opportunities--and challenges--for parents to select the schools that best fit their children's needs. The federal Every Student Succeeds Act of 2015 (ESSA) now calls for every school to have a report card that makes data available to families in an "understandable" and "user-friendly" manner. Yet little is currently known about how variations in the ways school information is presented might help or hinder parents as they make choices. Research in fields outside of education provides some hints about how the visual representation, amount, and organization of school information might affect parents' understanding of and satisfaction with the information and their choices. In testing some possible strategies and publishing this guide, the Institute of Education Sciences hopes to make decisions easier for information providers, who develop and distribute school directories, choice websites, and report cards and, ultimately, for parents. This guide illustrates the importance of carefully considering how school information is presented. [For the full report "Presenting School Choice Information to Parents: An Evidence-Based Guide. NCEE 2019-4003," see ED589101. For the study highlights to this report "Presenting School Choice Information to Parents: An Evidence-Based Guide. Study Highlights. NCEE 2019-4003," see ED589106.]

This brief describes (1) what parents look for when they choose a school and (2) how these preferences affect the sorting of students into schools under different school-choice policies. The findings are based on lists of preferred schools submitted by over 20,000 applicants to a citywide lottery for more than 100 traditional and charter public schools in Washington, DC. The results confirm previous findings that commuting distance, school demographics, and academics play important roles in school choice. However, we also found considerable variation in parents' preferences. [This brief summarizes findings from "Market Signals: Evidence on the Determinants and Consequences of School Choice from a Citywide Lottery. Working Paper 45," (ED568709).]

The Teacher Incentive Fund (TIF) provides grants to support performance-based compensation systems for teachers and principals in high-need schools. The study measures the impact of pay-for-performance bonuses as part of a comprehensive compensation system within a large, multisite random assignment study design. The treatment schools were to fully implement their performance-based compensation system. The control schools were to implement the same performance-based compensation system with one exception--the pay-for-performance bonus component was replaced with a one percent bonus paid to all educators regardless of performance. This second report provides implementation and impact information. Ninety percent of all TIF districts in 2012-2013 reported implementing at least 3 of the 4 required components for teachers, and only about one-half (52 percent) reported implementing all four. This was a slight improvement from the first year of implementation. In a subset of 10 districts participating in the random assignment study, educators' understanding of key program components improved during the second year, but many teachers still did not understand that they were eligible for a bonus. The pay-for-performance bonus policy had small, positive impacts on students' reading achievement; impacts on students' math achievement were not statistically significant but similar in magnitude. Appended are: (1) Supplemental Information on Study Sample Design, Data, and Methods for Chapter II; (2) Supplemental Information on Analytic Methods for Chapter II; (3) Supplemental Findings on Programs and Experiences of All TIF Districts For Chapter III; (4) Supplemental Findings of TIF Implementation in Evaluation Districts for Chapter IV; (5) Supplemental Findings on Impacts Of Pay-For-Performance on Educators' Attitudes and Behaviors For Chapter V; (6) Supplemental Findings in Impacts of Pay-For-Performance on Educator Effectiveness and Student Achievement for Chapter VI; and (7) Supplemental Findings on Relationships between TIF Program Characteristics and the impacts of Pay-For Performance for Chapter VI.

The Teacher Incentive Fund (TIF) provides grants to support performance-based compensation systems for teachers and principals in high-need schools. The study measures the impact of pay-for-performance bonuses as part of a comprehensive compensation system within a large, multisite random assignment study design. The treatment schools were to fully implement their performance-based compensation system that included four required components. The control schools were to implement the same performance-based compensation system with one exception--the pay-for-performance bonus component was replaced with a one percent bonus paid to all educators regardless of performance. This first of four planned reports provides implementation information prior to educators receiving annual performance measure information or payouts. Fewer than half of all 2010 TIF districts reported implementing all four required program components, although 85 percent reported implementing at least three of the four. In a subset of 10 districts who participated in the random assignment study, educators' reporting of the program indicated most misunderstood the performance measures and the amount of pay-for-performance bonus that they were eligible for. Most educators were satisfied with their professional opportunities, school environment, and the TIF program. Educators in those schools that offered the pay-for-performance aspect of TIF tended to be less satisfied than those in schools that did not offer such bonuses. However, educators in schools offering pay-for-performance bonuses were more satisfied with the opportunity to earn additional pay, and a greater percentage indicated feeling increased pressure to perform due to the TIF program. Appendices include: (1) Supplementary Information on Study Sample and Design; (2) Survey Response Rates and Characteristics of Respondents; (3) Analytic Methods and Sensitivity Analyses; (4) Supplemental Findings on TIF Design and Implementation for Chapters III and IV; and (5) Supplementary Findings for Chapter V. [For the executive summary of this report, see ED546821. For the NCEE Snapshot of this report, see ED546823.]

The Teacher Incentive Fund (TIF) provides grants to support performance-based compensation systems for teachers and principals in high-need schools. The study measures the impact of pay-for-performance bonuses as part of a comprehensive compensation system within a large, multisite random assignment study design. The treatment schools were to fully implement their performance-based compensation system that included four required components. The control schools were to implement the same performance-based compensation system with one exception--the pay-for-performance bonus component was replaced with a one percent bonus paid to all educators regardless of performance. This executive summary provides highlights from the first of four planned reports providing implementation information prior to educators receiving annual performance measure information or payouts. Fewer than half of all 2010 TIF districts reported implementing all four required program components, although 85 percent reported implementing at least three of the four. In a subset of 10 districts who participated in the random assignment study, educators' reporting of the program indicated most misunderstood the performance measures and the amount of pay-for-performance bonus that they were eligible for. Most educators were satisfied with their professional opportunities, school environment, and the TIF program. Educators in those schools that offered the pay-for-performance aspect of TIF tended to be less satisfied than those in schools that did not offer such bonuses. However, educators in schools offering pay-for-performance bonuses were more satisfied with the opportunity to earn additional pay, and a greater percentage indicated feeling increased pressure to perform due to the TIF program. [For the full report, see ED546820.]

Lack of researcher consensus on how to measure disadvantaged students' access to effective teaching has made it challenging for practitioners to draw lessons from the data. This brief aims to help policymakers understand the emerging evidence by synthesizing findings from three peer-reviewed studies that collectively span 17 states. The studies provide two lessons: (1) on average, disadvantaged students received less effective teaching than other students, equivalent to about four weeks of learning for reading and two weeks for math, or about 2 to 4 percent of the student achievement gap between these groups; and (2) access to effective teaching for disadvantaged students varied across districts, with a statistically significant difference between more and less disadvantaged students' access in some districts and no statistically significant difference in access in others. (Contains 11 endnotes, 1 table, and 4 figures.) [For the technical appendix, see ED544676.]

One way to improve struggling schools' access to effective teachers is to use selective transfer incentives. Such incentives offer bonuses for the highest-performing teachers to move into schools serving the most disadvantaged students. In this report, we provide evidence from a randomized experiment that tested whether such a policy intervention can improve student test scores and other outcomes in low-achieving schools. The intervention, known to participants as the Talent Transfer Initiative (TTI), was implemented in 10 school districts in seven states. The highest-performing teachers in each district--those who ranked in roughly the top 20 percent within their subject and grade span in terms of raising student achievement year after year (an approach known as value added)--were identified. These teachers were offered $20,000, paid in installments over a two-year period, if they transferred into and remained in designated schools that had low average test scores. The main findings from the study include: (1) The transfer incentive successfully attracted high value-added teachers to fill targeted vacancies; (2) The transfer incentive had a positive impact on test scores (math and reading) in targeted elementary classrooms; and (3) The transfer incentive had a positive impact on teacher-retention rates during the payout period; retention of the high-performing teachers who transferred was similar to their counterparts in the fall immediately after the last payout. Seven appendixes are included: (1) Supplemental Materials for Chapters I and II; (2) Value-Added Analysis to Identify Highest-Performing Teachers; (3) Supplemental Materials for Chapter III; (4) Identification of Focal Teachers; (5) Supplemental Materials for Chapter IV; (6) Supplemental Materials for Chapter V; and (7) Supplemental Materials for Chapter VI. (Contains 114 footnotes, 61 figures, and 92 tables.) [For the executive summary, see ED544268.]

One way to improve struggling schools' access to effective teachers is to use selective transfer incentives. Such incentives offer bonuses for the highest-performing teachers to move into schools serving the most disadvantaged students. In this report, we provide evidence from a randomized experiment that tested whether such a policy intervention can improve student test scores and other outcomes in low-achieving schools. The intervention, known to participants as the Talent Transfer Initiative (TTI), was implemented in 10 school districts in seven states. The highest-performing teachers in each district--those who ranked in roughly the top 20 percent within their subject and grade span in terms of raising student achievement year after year (an approach known as value added)--were identified. These teachers were offered $20,000, paid in installments over a two-year period, if they transferred into and remained in designated schools that had low average test scores. The main findings from the study include: (1) The transfer incentive successfully attracted high value-added teachers to fill targeted vacancies; (2) The transfer incentive had a positive impact on test scores (math and reading) in targeted elementary classrooms; and (3) The transfer incentive had a positive impact on teacher-retention rates during the payout period; retention of the high-performing teachers who transferred was similar to their counterparts in the fall immediately after the last payout.(Contains 5 figures, 1 box, and 5 footnotes.) [For the full report, see ED544269.]

This report describes the implementation and intermediate impacts of an intervention designed to provide incentives to induce a school district's highest-performing teachers to work in its lowest-achieving schools to improve student achievement. The report is part of a larger study that used random assignment to form equivalent groups of classrooms ("teacher teams") that either had the chance to participate in the intervention or did not. A future report will focus on the impacts of the intervention on student achievement and other outcomes. By focusing on implementation, this report will provide important information for policymakers and district administrators wishing to understand issues associated with teacher transfer incentives. Chapter II describes the formation of the study sample. Chapter III documents implementation information from the period of intensive activity that takes place during the spring and summer leading up to the transfers, when transfer teacher candidates are identified and recruited, and principals at the receiving schools interview and make offers to them. Chapter IV describes the teachers who transferred, the teachers who would fill the vacancies in high-need schools in the absence of a transfer incentive, the degree of collaboration and interaction among teachers, and the changes that happen inside the receiving school as a result of the transfer incentive. The last two phenomena--collaboration and changes in resource allocation--relate to the indirect and resource allocation effects discussed in the logic model above. Finally, Chapter V provides a brief summary of findings and next steps for the study. Appended are: (1) Analysis of Survey Nonresponse; (2) Value-Added Analysis to Identify Highest-Performing Teachers; and (3) Supplemental Tables and Figures. (Contains 24 tables, 27 figures and 59 footnotes.) [For "Moving Teachers: Implementation of Transfer Incentives in Seven Districts. Executive Summary. NCEE 2012-4052," see ED530749.]

This report describes the implementation and intermediate impacts of an intervention designed to provide incentives for a school district's highest-performing teachers to work in its lowest-achieving schools. The report is part of a larger study in which random assignment was used to form two equivalent groups of classrooms organized into teacher "teams" that are composed of teachers in the same grade level and subject (math, reading, or both in the case of an elementary school grade). Teams were assigned to either a treatment group that had the chance to participate in the intervention described below and or a control group that did not. Intermediate outcomes, measured for both the treatment and control teams, include the mix of teachers who make up the team, the climate of collaboration and cooperation in the team, and the way in which resources are allocated within the teacher team. A future report will focus on the impacts of the intervention on student achievement and other outcomes like retention. (Contains 2 tables, 2 figures and 9 footnotes.) [For the full report, "Moving Teachers: Implementation of Transfer Incentives in Seven Districts. NCEE 2012-4051," see ED530746.]

There are many ways to use student test scores to evaluate schools. This paper defines and examines different estimators, including regression-based value-added indicators, average gains, and successive cohort differences in achievement levels. Given that regression-based indicators are theoretically preferred but not always feasible, we consider whether simpler alternatives provide acceptable approximations. We argue that average gain indicators potentially can provide useful information, but differences across successive cohorts, such as grade trends, which are commonly cited in the popular press and used in the Safe Harbor provision of federal school accountability laws, are flawed and can be misleading when used for school accountability or program evaluation. (Contains 5 tables, 2 figures and 7 endnotes.)

The body of research on the design, implementation, and effects of performance-based compensation systems has influenced the design and evaluation of the 2010 Teacher Incentive Fund (TIF) grants. In the sections presented here, the authors describe the key components of 2010 TIF grants and the conceptual framework for the evaluation. The remainder of this report describes the plans for the study design in greater detail. In Chapter II, the authors provide context for the 2010 TIF grants and describe the treatment and counterfactual conditions--that is, the conditions that would exist in the absence of the performance pay systems. In Chapter III, they discuss data collection, the sample for the evaluation, and how they will conduct random assignment. Finally, they explain the analysis plans in Chapter IV. (Contains 12 tables, 1 figure and 15 footnotes.) [This report was submitted to the Institute of Education Sciences.]

This brief describes the prevalence of highest-performing teachers in ten purposely selected districts across seven states. The overall patterns indicate that low-income students have unequal access, on average, to the district's highest-performing teachers at the middle school level but not at the elementary level. However, there is evidence of variation in the distribution of highest-performing teachers within and among the ten districts studied. Some have an under-representation of the highest-performing teachers in high-poverty elementary and middle schools. Others have such under-representation only at the middle school level, and one district has a disproportionate share of the district's highest-performing teachers in its high-poverty elementary schools. (Contains 3 figures and 11 endnotes.) [For related report, "Do Low-Income Students Have Equal Access to the Highest-Performing Teachers? Technical Appendix", see ED517967.]

In 2004, the U.S. Department of Education's Institute of Education Sciences contracted with Mathematica Policy Research to conduct a large-scale evaluation of comprehensive teacher induction. The purpose of the study was to determine whether augmenting the set of services districts usually provide to support beginning teachers with a more comprehensive program improves teacher and student outcomes. This is the study's third and final report on the program's impacts. This report compares retention, achievement, and classroom practices of teachers who were offered comprehensive induction services to teachers who were offered the support normally offered by the school. Teachers assigned to receive comprehensive induction for either one or two years were supported by a full-time mentor who received ongoing training and materials to support the teachers' development. The teachers also were offered monthly professional development sessions and opportunities to observe veteran teachers. The teachers were followed for three years. Data was collected from 1,009 beginning teachers in 418 schools in 17 districts. Districts included in the study were not already offering comprehensive induction services, including paying for full-time mentors. Novice teachers in approximately half of the schools were assigned by lottery to receive comprehensive induction services. In 10 of the districts, these teachers were provided one year of comprehensive induction services; in the remaining 7 districts, the teachers were provided two years of services. Teachers in the schools not assigned to receive comprehensive induction services were provided the support normally offered to novice teachers by the school. Teacher practices were measured via classroom observations conducted in the spring of 2006. Data on teacher retention were collected via surveys administered in the fall of 2006, 2007, and 2008. Student test scores were collected from district administrative records for the 2005-06, 2006-07, and 2007-08 school years. Key findings include: (1) During the comprehensive induction program, treatment teachers received more support than control teachers; (2) The extra induction support for treatment teachers did not translate into impacts on classroom practices in the first year; (3) For teachers who received one year of comprehensive induction, there was no impact on student achievement; (4) For teachers who received two years of comprehensive induction, there was no impact on student achievement in the first two years. In the third year, there was a positive and statistically significant impact on student achievement; and (5) Neither exposure to one year nor exposure to two years of comprehensive induction had a positive impact on retention or other teacher workforce outcomes. The following are appended: (1) Supplemental Information for Chapters II and III; (2) Supplemental Information for Chapter IV; (3) Sensitivity Analyses and Supplemental Information for Chapter V; and (4) Sensitivity Analyses and Supplemental Information for Chapter VI.

This report set out to profile the Missouri Career Ladder program by delving beneath the surface and comparing how the program works in practice (based on stakeholder accounts) with how it works in theory (based on published statutes and regulations). The authors found that in many respects the program operates as one might expect. Districts implement rules that tend to follow the state guidelines, with some local tailoring. Teachers of a given experience level must perform a set of allowed activities in order to receive payments at one of the three Career Ladder levels. While Career Ladder provided opportunities to supplement salaries by offering extra pay for extra work, in most cases it would not qualify as a performance incentive program for teachers. The use of an evaluation tool (the PBTE) to promote teachers along the career ladder suggests that it should provide incentives for teachers to improve their practice. However, the authors found little evidence that the Performance-Based Teacher Evaluation (PBTE) is applied rigorously or implemented uniformly in a way that would motivate teachers to raise their performance. Instead, Career Ladder payments were seen as a reward for longevity and completion (with documentation) of a set of allowable activities such as tutoring, after-school activities, or approved professional development. The authors also sought to explain how districts and teachers came to be in the program. They found that district size matters. Finally, a goal of the analysis was to identify the mechanisms by which Career Ladder might produce positive impacts. They identified plausible routes whereby the program could raise student achievement, improve teacher satisfaction/morale, and improve teaching via productivity and composition effects. Teachers reported that they get to know their students better as a result of tutoring them outside of class. Three appendices are included: (1) Data and Methods; (2) Protocol for District-Level Interviews; and (3) Focus Group Protocol. (Contains 2 tables, 3 figures and 11 footnotes.)

In practice, teacher induction is common, but induction that is intensive, comprehensive, structured, and sequentially delivered in response to teachers' emerging pedagogical needs is less so. Congressional interest in formal, comprehensive teacher induction has grown in recent years. The National Center for Education Evaluation and Regional Assistance within the U.S. Department of Education's (ED) Institute of Education Sciences (IES) contracted with Mathematica Policy Research, Inc. (MPR), to evaluate the impact of structured and intensive teacher induction programs. The study examines whether comprehensive teacher induction programs lead to higher teacher retention rates and other positive teacher and student outcomes as compared to prevailing, generally less comprehensive approaches to supporting new teachers. More specifically, the study is designed to address five research questions on the impacts of teacher induction services: (1) What is the effect of comprehensive teacher induction on the types and intensity of induction services teachers receive compared to the services they receive from the districts' current induction programs?; (2) What are the impacts on teachers' classroom practices?; (3) What are the impacts on student achievement?; (4) What are the impacts on teacher retention?; and (5) What is the impact on the composition of the district's teaching workforce? Statistically significant differences between the treatment and control groups were identified in the amount, types, and content of induction support teachers reported having received, both in the fall and the spring of the intervention year. Although treatment teachers reported receiving more mentoring than did control teachers; were more likely than control teachers to report participating in specific induction activities; and spent more time in certain professional activities than did control teachers during the three months prior to the spring survey, summarized comparisons between treatment and control groups found: (1) No impacts on teacher practices; (2) No positive impacts on student test scores; (3) No impacts on teacher retention; and (4) No positive impacts on composition of district teaching workforce. This report focused on the first year of findings only. The research team is conducting longer term follow-up to include additional collection of test score and teacher mobility data. Eight appendices are included: (1) National Data on Teacher Induction; (2) Analysis Weights; (3) Impact Estimation Methods; (4) Classroom Observation Methods; (5) Reference Tables for Chapter II; (6) Supplemental Tables for Chapter IV; (7) Supplemental Tables for Chapter V; and (8) Supplemental Figures. (Contains 43 footnotes, 15 figures and 101 tables.)

This report examines the career trajectories of those who have successfully completed the Passport to Teaching certification offered by the American Board for Certification of Teacher Excellence (ABCTE) during its first years of existence. To elicit information on the career choices of Passport alumni, Mathematica Policy Research, Inc. (MPR) designed a survey that addressed the following questions: (1) Who obtains a Passport and what career choices did they make? (2) What happens to those who become teachers? and (3) What other experiences did they have after receiving a Passport to Teaching? As of October 1, 2007, 614 individuals have been certified, and the numbers of alumni and the diversity of states and subject areas of certification are increasing each year, as are the applicants from overseas. The number of African American and Hispanic alumni has grown, although each of those two groups continues to represent less then two percent of the alumni. The states from which Passport alumni come has been expanding gradually beyond the base of Idaho and Florida, where the certificate was first accepted in public schools. Passport alumni sought the credential for a variety of reasons. A growing majority aspired to enter the teaching profession, which is presumably most consistent with the goal of the program. Of those who said they were seeking to retain their position or obtain another position, the program may have served to document their expertise in the field they were already teaching, or it may have enabled them to move laterally into a shortage area or a field in which they preferred to be teaching. The current employment status of Passport alumni includes teachers in a variety of positions as well as many who are not teaching. Alumni who were teaching at the time of the survey were in a variety of schools, including traditional public schools as well as charter and private schools. The majority of alumni who are teaching are doing so in their fields of certification, where mentoring is common, but not universal. The report concludes that the future direction of Passport to Teaching will depend on the acceptance of the credential in more states and growing its reputation among charter and private school administrators. It will also depend on the ongoing recruitment efforts of ABCTE to actively pursue minority candidates. Additional research is recommended to continue following Passport alumni into the classroom and to measure their job performance relative to teachers who reach the classroom through more traditional pathways and to assess the impact Passport teachers have on student achievement and other outcomes. A Table with the data for States that Accepted Passport to Teaching Certification(s) is appended. (Contains 1 footnote, 5 figures and 11 tables.)

States vary in the degree to which they require prospective teachers to complete coursework, accumulate student teaching experience, and perform on written examinations before they are granted entry into the classroom. In recent years, the American Board for Certification of Teacher Excellence (ABCTE) has devised a primarily examination-based approach called the Passport to Teaching to address this problem, but to date, only five states have recognized it. One major impediment to more widespread adoption of the Passport is the need for more rigorous and objective evidence on the ability of the Passport program to identify high-quality or even minimal-quality teachers. This report is the first in a series of reports from an evaluation study designed to help address this need. Here, the authors provide some early evidence on the performance of American Board-certified teachers based on a recent survey of their supervisors. (Contains 8 tables and 1 figure.)

Education policymakers have long sought to establish teaching standards that will measure new or continuing teachers against these standards. The problem is, existing methods for certifying teachers have been criticized for being either so onerous as to deter good candidates or so lax as to keep weak teachers in the profession. To provide another approach, the U.S. Department of Education recently funded an organization called the American Board for Certification of Teacher Excellence (ABCTE, or American Board) to develop portable teacher credentials that would identify qualified beginning teachers and recognize accomplished veteran teachers. In 2005, ABCTE selected Mathematica Policy Research, Inc. (MPR) to conduct an independent, five-year evaluation of the certification programs. This paper describes results from the first six months and plans for the remainder of the evaluation. The first sections provide background on certification processes in use today ("traditional") and ABCTE itself; subsequent sections discuss findings from the early use of American Board certification, as well as a design for a rigorous evaluation of the credentials, the results of which will be included in future reports. By issuing this report on the early descriptive findings and design of the evaluation, the hope is to promote transparency in this approach and invite comments from the field. (Contain 4 tables and 2 figures.) [This report was submitted to the American Board for Certification of Teacher Excellence.]

How public school teachers are paid in the U.S. has been a controversial issue for many years. Critics of the traditional system, in which teachers are paid on the basis of years of experience and educational attainment, claim that it does not reward or promote good teaching as fairly as systems that tie pay to performance: having certain skills, or being willing to teach in high-need areas. Proponents of the traditional system argue that experience and education are important predictors of teacher performance, and that the simplicity, transparency and fairness of the system make it the logical choice. In an attempt to achieve the best of both worlds, educators and policymakers have devised numerous approaches to reforming the teacher payment system. Choosing between the various approaches is difficult because scientific evidence on their effectiveness is limited. This report presents findings and recommendations from a review of how secondary data could be used to study a variety of teacher incentive programs to address the question of whether teacher incentive programs improve student learning, either by making teachers more productive (productivity effort) or by attracting and retaining good teachers at higher rates (composition effect.) Following background information on attempts to study these programs, the preliminary steps of identifying and categorizing teacher incentive programs and the process of narrowing down the list to the most promising are discussed. Detailed profiles of the seven candidates selected for further study are provided. Cross-cutting issues of sample size adequacy and cost are identified, and a final section summarizes the findings and offers recommendations for next steps. (Contains 8 tables.) [This report was submitted to the US Department of Education, Institute of Education Sciences/National Commission on Excellence in Education by Mathematica Policy Research, Inc. It was prepared by Mathematica Policy Research, Inc. and Chesapeake Research Associates.]

In recent years, researchers have argued that the shortage of highly qualified teachers in poor school districts may have less to do with attracting new teachers than with retaining them. To provide the scientific evidence that will support sound decisions about teacher induction, Mathematica is examining whether high-intensity teacher induction programs lead to higher retention rates and other positive teacher and student outcomes. This report lays out the study design in detail, documenting the process for building the sample of districts, schools, and students, and for conducting random assignment. In addition, it describes the data collection and analysis plans. (Contains 7 tables and 3 figures.)

This four-page issue brief summarizes findings from Mathematica's evaluation of Teach For America. The researchers found that TFA teachers outperformed non-TFA teachers in the same schools in math and achieved the same results in reading. However, TFA teachers also reported greater challenges in managing their classrooms. TFA and non-TFA teachers had very different educational and teacher preparation backgrounds, with TFA teachers generally coming from more competitive undergraduate backgrounds but with less formal teacher training. However, non-TFA teachers came from a diversity of backgrounds and preparation, with many entering the profession without substantial teacher training. (Contains 2 figures.)

In October 2002, the Institute of Education Sciences (IES) contracted with Mathematica Policy Research, Inc. (MRP) to help identify issues pertinent to the evaluation of Title I and to propose feasible evaluation design strategies. This design effort took its lead from two sources: (1) the Title I Independent Review Panel (IRP); and (2) a more specialized technical working group (TWG) composed of education research and evaluation experts assembled for the design effort itself. The MPR design team worked with the TWG, the IRP, other outside experts, and IES to develop a series of recommendations concerning: (1) the topic of study for the evaluation; (2) strategies for selecting intervention and schools, and (3) the basic elements of a research design. This document lists the key recommendations that came out of this effort. The report itself provides the rationale for the recommendations and discusses design and measurement issues, including sample sizes. The following are appended: (1) TWG and IRP Members; (2) Teaching of Comprehension in Content Areas: Design Criteria (Thomas Trabasso); and (3) Desirable Characteristics of Comprehension Interventions (Michael Kamil). (Contains 2 tables and 1 figure.)

A study involving random assignment of all youth eligible for Job Corps to either a Job Corps program or to a control group was conducted to assess the impact of Job Corps on key participant outcomes. Participants in the study were nationwide youth eligible for Job Corps who applied for enrollment for the first time between November 16, 1994, and December 17, 1995. The study sought to determine the following:(1) how effectively Job Corps improves the employability of disadvantaged participants, (2) whether Job Corps impacts differ for youths with different baseline characteristics, and (3) how effective the residential and nonresidential components of Job Corp are. Findings over the first 4 years after random assignment include the following: (1) Job Corps provided extensive education, training, and other services to the program group and improved their educational attainment; (2) Job Corps generated positive employment and earnings impacts by the beginning of the third year after random assignment and the impacts persisted through the fourth year; (3) employment and earnings gains were found broadly across most subgroups of students; (4) the resident and nonresidential programs were each effective for the youths they served; (5) Job Corps significantly reduced youths' involvement with the criminal justice system; (6) Job Corps had small beneficial impacts on the receipt of public assistance and self-assessed health status, but no impacts on illegal drug use; and (7) Job Corps had no impacts on fertility or custodial responsibility, but it slightly promoted independent living and mobility. (The report include numerous tables and charts, 31 references, and five appendixes concerning the study methodology.) (KC)

A benefit-cost analysis of the Job Corps program compared groups randomly assigned to either enroll in the program or to constitute a control group that did not enroll. Youth who participated in the study were those found eligible for Job Corps nationwide between November 1994 and February 1996. Interviews with participants and the assignment of dollar values to costs and benefits were among the research methods used. Benefits and costs measured included the following: (1) benefits of increased output resulting from the additional productivity of Job Corps participants; (2) benefits from reduced use of other programs and services; (3) benefits from reduced crime committed by or against participants; and (4) program costs and costs of resources used by Job Corps. Benefits and costs were measured from the perspectives of society as a whole, participants, and the rest of society (non-participants in Job Corps). The study's findings indicate that Job Corps is a good investment. The benefits to society exceed the costs of the program by nearly $17,000 per participant, assuming that the observed earnings impacts do not decline rapidly as participants get older. The researchers say evidence from other studies suggests the impacts will persist without rapid decay. The study concluded that Job Corps is a valuable program whose benefits exceed costs over a wide spectrum of student groups and for several areas of society. (The report includes 28 tables and 11 figures.) (KC)

The National Job Corps Study is based on a national random sample of all eligible applicants to Job Corps in late 1994 and 1995. The sampled youth were assigned randomly to either a program group whose members could enroll in Job Corps or a control group whose members could enroll in all other programs available to them in their communities. Findings related to delivering services indicated Job Corps centers effectively deliver the planned services called for by the program model and Job Corps provides extensive education, training, and other services. Findings related to making a difference show Job Corps substantially increases the education and training services that youths receive and improves their skills and educational attainment; Job Corps generates employment and earnings gains; employment and earnings gains are found across most groups of students; the residential and nonresidential programs are each effective for the youths they serve; Job Corps significantly reduces involvement with crime; and Job Corps has modest or no impacts on a range of other outcomes. Outcomes regarding whether Job Corps is a good investment whose benefits exceed costs are Job Corps is cost effective despite its high costs; benefits during the study period are modest; benefits should continue; and Job Corps is a good investment. (YLB)

This report examines the extent to which Job Corps improves literacy and numeracy skills. Chapter I provides an overview of Job Corps and discusses key policy issues related to basic skills and study objectives. Chapter II addresses study design issues, interview response rates, and analytic methods used to estimate program impacts on literacy scores. Chapter III presents Job Corps impacts on three domains of literacy skills (prose, document, and quantitative) and provides a descriptive analysis, impact findings, and estimates of literacy for key subgroups. It reports that the typical youth Job Corps serves has lower functional literacy scores than the typical young adult, especially in the quantitative literacy domain. Job Corps' impacts on participants' functional literacy skills were positive in all three domains. Chapter IV discusses the extent to which estimated impacts on literacy skills are consistent with impact findings on key outcomes associated with literacy skills. A proposed theoretical framework is used to examine the link among educational attainment, labor market outcomes, and literacy skill scores. Empirical results on these relationships are presented: literacy levels are influenced by schooling and employment measures; and some association exists between literacy assessment scores and labor market outcomes, but a large residual variation in earnings remains. Appendixes contain 13 references, analysis of nonresponse, administration and scoring of the assessment, and supplementary tables. (YLB)

The Child Nutrition Act of 1998 authorized demonstration pilot projects in up to six school food authorities and a rigorous evaluation to assess the effects of providing free school breakfasts to elementary school children. This report describes the evaluation strategy and data collection plans. Part 1 of the report provides background information, describes the school breakfast program (SBP), summarizes the literature on breakfast and learning, and discusses the rationale for the Universal-Free School Breakfast Program (USBP). This part also identifies the requirements of the demonstration and evaluation, provides a conceptual framework of how participation in a USBP may affect student outcomes, and provides an overview of the full set of evaluation design components. Part 2 describes the design for the impact study. Research questions and hypotheses to be tested are discussed, as are outcomes and measures that will be used to assess program effectiveness. This part also describes the approach for evaluating the USBP demonstration programs, discusses evaluation and sampling design issues and strategies, presents an analysis plan for assessing program impacts, and describes data collection plans. Part 3 presents the approach for conducting the implementation study of the USBP pilot project, describing study topics and research questions and outlining plans for process data collection. Part 4 provides an assessment of the feasibility of implementing the preferred evaluation design. This part reviews the statistical precision achieved, provides an estimate of the demonstration costs, and identifies strengths, weaknesses, and risks. (Contains 67 references.) (KB)

Those who favor expansion of consumer choice in education claim that competition would force schools to improve. Critics claim that it would sort students by race and class. A competitive market will provide what consumers demand, yet neither side has empirical evidence on such consumer preferences to back up their claims. This paper offers such evidence. This paper estimates a conditional logit model using data from a public school choice program in Minneapolis, Minnesota, in order to infer how families trade off the convenience of a shorter commute with school quality and peer group characteristics. The evidence suggests that consumer choice alone would not raise schools' academic performance. Parents in Minneapolis were not more likely to choose schools with high test scores or greater value added. Rather they preferred schools relatively close to home and ones where they were better represented ethnically and racially. The only discernable test score effect was one where families sought a match between their own child's ability and the mean ability level of similar students at the prospective school. Simulations suggest that expanding choice could ultimately lead to severe, but not total segregation by race and ethnicity. Unrestricted choice could present a tradeoff between consumer satisfaction today and increased racial segregation in the future. (Author/RJM)

This report addresses the comparison of teacher evaluation systems in the context of a particular administrative and legislative challenge: How a state or the federal government could achieve a uniform standard for dispensing funds to school districts for the recognition of exceptional teachers without imposing a uniform evaluation system on those districts. The authors address and provide practical procedures for determining the reliability of local teacher evaluation systems. They then demonstrate that the reliability of the evaluation system determines the proportion of teachers that a system can identify as exceptional. Thus a school district wanting to accurately recognize the top quartile of teachers as highly effective would only be able to identify some portion of the top 25 percent with confidence given the lack of perfect reliability in the measures of teacher effectiveness that are deployed by the district. Further the portion of the top quartile that could be identified would be greater in an identically sized district that has more reliable measures. A technical appendix is included. (Contains 4 figures and 14 endnotes.) [This paper was written with the assistance of Michelle Croft.]

The evaluation of teachers based on the contribution they make to the learning of their students, "value-added", is an increasingly popular but controversial education reform policy. In this report, the authors highlight and try to clarify four areas of confusion about value-added. The first is between value-added information and the uses to which it can be put. One can, for example, be in favor of an evaluation system that includes value-added information without endorsing the release to the public of value-added data on individual teachers. The second is between the consequences for teachers vs. those for students of classifying and misclassifying teachers as effective or ineffective--the interests of students are not always perfectly congruent with those of teachers. The third is between the reliability of value-added measures of teacher performance and the standards for evaluations in other fields--value-added scores for individual teachers turn out to be about as reliable as performance assessments used elsewhere for high stakes decisions. The fourth is between the reliability of teacher evaluation systems that include value-added vs. those that do not--ignoring value-added typically lowers the reliability of personnel decisions about teachers. They conclude that value-added data has an important role to play in teacher evaluation systems, but that there is much to be learned about how best to use value-added information in human resource decisions. (Contains 16 footnotes.)