Search

Research shows that effective teachers are one of the most important in-school contributors to student learning. To improve low-income minority students' access to effective teaching, the Bill & Melinda Gates Foundation launched the Intensive Partnerships for Effective Teaching (IP) initiative in the 2009-2010 school year. The program was implemented in three public school districts and four charter management organizations (CMOs) across the United States. Researchers examined a variety of outcomes, but analysis suggested that, overall, the initiative did not reach its goals. In particular, student achievement, low-income minority students' access to effective teachers, and dropout rates were not substantially better in the IP sites than in similar sites that did not participate in the initiative. Although this analysis of impact, which was published in 2018, did not find that the initiative was effective overall, the researchers did observe that some IP schools had improved more than others. The study team decided to dig deeper into the variation in outcomes among schools and look for school-level factors associated with that variation. Newly collected qualitative data, as well as new analyses of survey data from the original study, allowed the team to more closely investigate the factors that differed between the improving and nonimproving IP schools and develop recommendations for future variation studies of multisite efforts. [This brief describes work documented in "Improving Teaching Effectiveness: Variation in Improvement among Schools in the Intensive Partnerships for Effective Teaching." For this report, see ED609290.]

In 2018, The RAND Corporation and the American Institutes for Research (AIR) published an evaluation of the Bill & Melinda Gates Foundation's Intensive Partnerships for Effective Teaching (IP) initiative, which was designed to improve achievement among low-income minority (LIM) students. The initiative provided support for several reforms that were theorized to result in improved teacher effectiveness, such as teacher workforce conditions (e.g., hiring, retention, dismissal), teacher-evaluation policies, individualized professional development (PD), strategic compensation, and career ladders. However, the study found that the initiative did not meet its goals of improved teacher effectiveness and greater access to effective teachers among LIM students. Although, on average, there was little improvement, the Gates Foundation, along with researchers from RAND and AIR, speculated that lessons could be learned from variation among schools within the IP sites because some schools had improved more than others. To investigate the factors that might be associated with positive student outcomes at the school level, researchers conducted a qualitative study and a survey study. For the qualitative study, the authors selected 11 pairs of schools from the original study that were similar at the beginning of the initiative but showed differing levels of improvement during the initiative. Researchers interviewed teachers and administrators at these schools and analyzed their responses to identify ways that improving and nonimproving schools varied from the staff perspective. In the survey study, the authors used teacher surveys that were administered in the spring of each year from 2013 through 2016 to look for relationships between staff beliefs and school-level improvement over the course of the IP initiative. Key findings from the study include: (1) In both the qualitative and the survey study, leadership was related to schools' success; (2) The qualitative study found that unity among staff was associated with school improvement, while staff perceptions of contextual factors beyond their control were associated with nonimprovement; (3) Unlike the qualitative study, the survey study found that several factors associated with the IP initiative were significantly related to improvement; and (4) Although an after-the-fact examination of variation among schools that undertook a reform initiative can be useful, planning ahead for such a study would be beneficial.

The authors of this report aim to understand the health of the science, technology, engineering, and mathematics (STEM) education and labor market in the Appalachia region -- defined as the intersection of Ohio, Pennsylvania, and West Virginia -- with a focus on the mining and extraction industry. This report is the third in a three-part series. The first two reports provided initial examinations of STEM education and the labor market in the Appalachia Partnership Initiative (API) region. In this report, the authors address the same questions as the first two reports but devote more attention to examining whether and how any of the indicators have changed over time. The authors also tighten the analysis specifically to the API region and introduces a few new measures, such as participation in career and technical education (CTE). Key Findings: (1) Middle school students in the API states have outperformed students nationwide in mathematics, and trends are improving over time; (2) CTE in the API region has room for growth; (3) High school graduation rates are high and improving; (4) Although postsecondary enrollment decreased in the API region over time, the percentage of students graduating in STEM has been increasing and outstripping the national average; (5) The working-age population declined within the API region; (6) Median earnings were higher in suburban areas of the API region and lowest in parts of the urban centers, but there was no clear geographic pattern of growth in median earnings; (7) Earnings were high and growing in the mining and utilities industry in the API region; (8) The mining and extraction industry was particularly successful at compensating workers who held a high school degrees or had less education; and (9) New hires from outside the extraction industry in API states primarily came from within the region but from other industries and enjoyed large earnings increases. [The research was conducted with the RAND Social and Economic Well-Being Division. For "Wages, Employment, and STEM Education in Ohio, Pennsylvania, and West Virginia. Report No. 1 (2016). Research Report. RR-1357-CHC," see ED605413. For "Wages, Employment, and STEM Education in Ohio, Pennsylvania, and West Virginia. Report No. 2 (2017). Research Report. RR-1863-CHC," see ED605433.]

Research across the decades has confirmed that effective school leadership is associated with better outcomes for students and schools. A high-quality school leader affects dozens of teachers and hundreds or thousands of students. It is a pivotal role. From 2011 to 2016, The Wallace Foundation, through its Principal Pipeline Initiative (PPI), provided funding and technical assistance to support six large school districts in their efforts to put in place systematic processes for the strategic management of school leaders. The purpose was to examine whether a comprehensive principal pipeline would be more effective than business-as-usual approaches to the preparation and management of school leaders. The term principal pipeline is shorthand for the range of talent management activities that fall within a school district's scope of responsibility when it comes to school leaders, including leader standards, preservice preparation opportunities for assistant principals and principals, selective hiring and placement, and on-the-job induction, evaluation, and support. This report documents what the PPI districts were able to accomplish, describing the implementation of the PPI and its effects on student achievement, other school outcomes, and principal retention. The authors found that all six PPI districts were able to implement comprehensive pipelines, and they did so in different ways. The PPI had positive effects on a wide range of outcomes that school districts care about, and evidence of these positive effects was widespread. The authors also found that the work is affordable: It cost a PPI district about $42 per pupil per year, or less than 0.5 percent of the district's budget in each school year, to operate and enhance its principal pipeline. The authors conclude that districts looking for ways to enhance school outcomes and improve the retention of newly placed principals should be encouraged by the experiences of PPI districts. The findings of this study suggest that when districts focused attention on activities related to principal pipelines, then principals, schools, and students benefited. Key findings include: (1) The work is feasible: (a) PPI districts were able to implement all components of a principal pipeline at scale; (b) PPI districts approached pipeline enhancement in different ways depending on their starting point, needs, and opportunities; (2) The work is effective: (a) After three or more years, schools with newly placed principals in PPI districts outperformed comparison schools with newly placed principals by 6.22 percentile points in reading and 2.87 percentile points in math; (b) Newly placed principals in PPI districts were 5.8 percentage points more likely to remain in their school for at least two years and 7.8 percentage points more likely to remain in their school for at least three years than newly placed principals in comparison schools; (c) The PPI had statistically significant, positive effects on achievement in elementary and middle schools and some evidence of positive effects for high schools; (d) PPI effects on achievement were positive and statistically significant for schools in the lowest quartile of the achievement distribution; and (e) PPI effects were positive and statistically significant in reading for five PPI districts and in mathematics for three districts. The PPI effect was negative and statistically significant in mathematics in one district; (3) The work is affordable: PPI districts spent about $42 per student per year on pipeline activities during the initiative. The lowest-cost components were the development of leader standards and selective hiring and placement; and (4) What drove these effects: The entire package of PPI components appears to have worked as a cohesive whole, much as it was designed to do. There was little evidence that individual components were uniquely correlated with larger or smaller effect sizes.

The Office of Workforce Development (OWD) in New Orleans, Louisiana, with a grant from the U.S. Department of Labor, implemented Career Pathways, an innovative program designed to increase the local talent pool and help lower-skilled, unemployed, and underemployed individuals train for work in growing fields. RAND Corporation researchers evaluated the program to find out whether it was succeeding in its mission: helping trainees learn industry-valued skills and find related jobs. The research team also examined the broader costs and benefits of the program in relation to the city of New Orleans. The team found that the New Orleans Career Pathways program produced meaningful positive results in several areas. These included individuals' wage growth, job satisfaction, and the government's and society's return on investment. There were also areas that had no significant change, such as arrest rates and the duration of individuals' employment. This brief summarizes the main study findings as well as related recommendations to help workforce development boards, employers, training organizations, and other stakeholders striving to create or improve workforce development programs in their region. [For the full report, "Effectiveness of Screened, Demand-Driven Job Training Programs for Disadvantaged Workers an Evaluation of the New Orleans Career Pathway Training. Research Report. RR-2980-DOL," see ED618674.]

This appendix supplements the report "Effectiveness of Screened, Demand-Driven Job Training Programs for Disadvantaged Workers an Evaluation of the New Orleans Career Pathway Training. Research Report. RR-2980-DOL" (ED618674). The appendix reproduces the instruments used in the analysis presented in the full report. The following are presented: (1) Career Pathways Program Implementation: Timeline of Modifications to the Proposed Design; (2) Prescreening Protocol for Cultural Organizations; (3) Office of Workforce Development and ResCare Screening Materials; (4) New Screening Materials (Effective August 2017); (5) Advanced Manufacturing Cohorts (Training Provided by Delgado Community College); and (6) Information Technology Cohorts (Training Provided by New Horizons Computer Learning Center).

A growing body of research points to the ways in which principals influence teachers, classrooms, and, ultimately, student achievement. New Leaders aims to prepare transformational school leaders by partnering with districts and charter schools to offer rigorous, research-based training for aspiring principals. The Aspiring Principals program is New Leaders' signature program and has three core features: selective recruitment and admission, training and endorsement, and support for principals early in their tenure. This report is a follow-up to the 2014 evaluation of New Leaders' Aspiring Principals program. Focusing on the revised program, which was first implemented in 2012, the authors present evidence of the effectiveness of the revised Aspiring Principals program and share lessons that can inform principal-preparation policy and practice. To assess the effect of New Leaders' Aspiring Principals program, researchers analyzed whether schools and students led by graduates of the program outperformed comparison schools and students in the same district, focusing on student achievement and principal retention. They also examined program graduate placement and satisfaction with the Aspiring Principals program. [For the appendixes, see ED605724.]

The basic concept of personalized learning (PL) -- instruction that is focused on meeting students' individual learning needs while incorporating their interests and preferences -- has been a longstanding practice in U.S. K-12 education. Options for personalization have increased as personal computing devices have become increasingly affordable and available in schools and developers created software to support individual student learning. In recent years, it has become more common for schools to embrace schoolwide models of PL. We collected data from schools in the Next Generation Learning Challenges (NGLC)'s Breakthrough School Models program. Our study seeks to describe the practices and strategies these schools used to implement PL, understand some of the challenges and facilitators, and consider these alongside achievement findings to discern patterns that may be informative. Teachers and students reported higher levels of many aspects of personalization than their counterparts in a national sample. These included time for one-on-one tailored support for learning; using up-to-date information on student progress to personalize instruction and group students; students tracking their own progress; competency-based practices; and flexible use of staff, space, and time. However, some more-difficult-to-implement aspects did not appear to differ from practices in schools nationally, such as student discussions with teachers on progress and goals; keeping up-to-date documentation of student strengths, weaknesses, and goals; and student choice of topics and materials. We estimate study students gained about 3 percentile points in mathematics relative to a comparison group of similar students. In reading, there was a similar trend, though it was not statistically significant. Low-performing and high-performing students appeared to benefit.

This first of five annual reports focuses on employment and wages in energy and advanced manufacturing--related industries and on science, technology, engineering, and mathematics (STEM) education indicators in the Appalachia Partnership Initiative (API) region. Together, these reports will capture trends over the next four years to (1) inform the regional stakeholder community across the 27-county API region about which localities might have greater demand for educating or employing local talent in STEM careers and (2) guide API investments and collaborative work. Key Findings: API Regional Trends in the Working-Age Population and in Wages and Employment in Energy and Advanced Manufacturing Sectors: (1) The API county with the greatest growth in working-age population was Monongalia County, West Virginia, which grew by 29.2 percent from 2000 to 2013. At this same time, the working-age population in the United States grew by 13.6 percent. In all but five counties of the API region, the size of the working-age population decreased; (2) Median wages were relatively high in some suburban Pittsburgh areas but highest in portions of a West Virginia county that experienced high growth in its working-age population; (3) The utilities industry was the STEM-related industry category that provided the highest median wages in the region; (4) The occupations of engineering and architecture enjoyed the highest median wages of STEM-related occupations across all industries, but there was a wide range of median wages in the 27-county region; (5) The number of regional jobs in STEM-related industries and occupations increased, while those in other industries decreased; (6) West Virginia lagged behind Pennsylvania, Ohio, and the national average in mathematics and science assessment scores, with 76 percent of eighth graders scoring at basic or below, only 3 percent scoring advanced in mathematics, and more than three in four students scoring basic or below basic in science; (7) Ohio and West Virginia had high school graduation rates comparable to the nation, while Pennsylvania outperformed the national average. Graduation rates in the three API states, as well as nationally, improved by 2 to 3.5 percentage points between 2010-2011 and 2012-2013; and (8) Ohio and Pennsylvania exceeded the nation in percentages of students gaining associate's degrees in STEM-related fields, while West Virginia trailed. All three states, and the nation, had roughly comparable percentages of students gaining bachelor's degrees in STEM-related fields.

The adoption of personalized learning approaches has increased significantly in recent years. This report examines achievement in 62 public charter and district schools that are pursuing a variety of personalized learning practices, and examines implementation details in 32 of those schools. Researchers obtained achievement data for personalized learning students and a matched comparison group of students attending other schools serving similar populations. They also collected and analyzed data from site visits, interviews, and surveys to create a broad picture of the schools' efforts to implement personalized learning and the perceptions of teachers and students. The achievement findings indicate that compared to peers, students in schools using personalized learning practices are making greater progress over the course of two school years, and that those students who started out behind are catching up to perform at or above national averages. The study finds that teachers at most schools were using data to understand student progress and make instructional decisions, all schools offered time for individual academic support, and the use of technology for personalization was widespread. However, some strategies, such as competency-based progression, were less common and more challenging to implement. Key findings are grouped into four sections. The first section on student achievement finds that there were positive effects on student mathematics and reading performance and that the lowest-performing students made substantial gains relative to their peers. The second section on implementation and the perceptions of stakeholders finds that adoption of personalized learning practices varied considerably. Personalized learning practices that are direct extensions of current practice were more common, but implementation of some of the more challenging personalized learning strategies was less common. The third section relates implementation features to outcomes and identifies three elements of personalized learning that were being implemented in tandem in the schools with the largest achievement effects. Finally, the fourth section compares teachers' and students' survey responses to a national sample and finds some differences, such as teachers' greater use of practices that support competency-based learning and greater use of technology for personalization in the schools in this study with implementation data. Appended are: (1) Methods for Achievement Analysis; (2) Methods for Implementation Analyses; (3) Student Survey Scales, Personalized Learning Schools; (4) Teacher Survey Scales, Personalized Learning Schools; and (5) Personalized Learning Attributes, Definitions, and Coding.

The purpose of this document is to describe the methods RAND used to analyze achievement for 23 personalized learning (PL) schools for the 2012-13 through 2013-14 academic years. This work was performed at the request of the Bill & Melinda Gates Foundation (BMGF), as part of a multi-year evaluation contract. The 23 schools were selected from a larger portfolio of PL schools funded directly or indirectly by BMGF because they implemented PL school-wide during both of the two academic years and they also administered Northwest Evaluation Association's (NWEA) Measures of Academic Progress (MAP) both years.

As part of its effective teaching initiative, the Bill & Melinda Gates Foundation partnered with three urban school districts across the U.S. and a group of four charter management organizations to undertake a strategic set of human capital reforms. A key objective of the "Intensive Partnerships for Effective Teaching" program was to promote increased access among disadvantaged students to the most effective teachers in their respective sites. The IP initiative includes an integrated set of programs or levers designed to improve teacher human capital in each site and to improve the access of the most disadvantaged students to the best teachers working within a site. These levers include efforts to improve how teachers are recruited, hired, and professionally developed; how they are evaluated, rewarded, and dismissed; and how they are assigned to--or incentivized to work with--high-need students. This paper discusses trends over time in the correlation between a teacher's value-added estimate and the proportion of students in her caseload and school who are both low-income and minority (LIM). This report is the third in a series that RAND has published that documents teachers' value-added estimates in mathematics and reading by the proportion of their students who are LIM. The paper also includes an appendix containing additional tables.

New Leaders is dedicated to promoting student achievement by developing outstanding school leaders to serve in urban schools. RAND Corporation researchers conducted a formative and summative external evaluation of the New Leaders program, its theory of action, and its implementation from 2006 through 2013. This document presents technical appendixes to supplement our main evaluation report (Gates et al., 2014). The intended audience for these appendixes is individuals who seek additional information on the methods used in our analysis or additional district-specific detail. The intended audience includes other researchers and officials in school districts. This research has been conducted in RAND Education, a unit of the RAND Corporation, under a contract with New Leaders. [For the full report, see ED561152. For the research brief, "Principal Preparation Matters: How Leadership Development Affects Student Achievement. Research Brief," see ED561155.]

New Leaders is a nonprofit organization with a mission to ensure high academic achievement for all students by developing outstanding school leaders to serve in urban schools. Its premise is that a combination of preparation and improved working conditions for principals, especially greater autonomy, would lead to improved student outcomes. Its approach involves both preparing principals and partnering with school districts and charter management organizations (CMOs) to improve the conditions in which its highly trained principals work. As part of the partnerships, New Leaders agrees to provide carefully selected and trained principals who can be placed in schools that need principals and to provide coaching and other support after those principals are placed. The districts and CMOs agree to establish working conditions that support, rather than hinder, the principals' efforts to improve student outcomes. This report describes how the New Leaders program was implemented in partner districts, and it provides evidence of the effect that New Leaders has on student achievement. [The research in this report was produced within RAND Education. For the appendices that accompany this report, see ED561154. For the research brief, "Principal Preparation Matters: How Leadership Development Affects Student Achievement. Research Brief," see ED561155.]