Search

The research featured in this paper is part of the CORE-PACE Research Partnership, which is focused on producing research that informs continuous improvement in the CORE districts as well as policy and practice in California and beyond. In this report, the authors used data from multiple interim assessments to examine how the rate of student learning from fall 2019 through winter 2020-21 differs from that of student learning before COVID-19. Specifically, the authors assessed the degree to which approximately 100,000 students across 19 local education agencies (LEAs) in California experienced slower academic growth compared to previous school years (i.e., a learning lag) by the time they completed winter 2021 interim assessments (NWEA MAP Growth, Renaissance Learning Star, and Curriculum Associates i-Ready) in Grades 4-8. To understand equity gaps in the degree to which students have experienced lost instructional opportunities, the authors disaggregated these results for students who were economically disadvantaged, students who were English learners, students with disabilities, students of different racial/ethnic backgrounds, students with low prior achievement, and students who were experiencing homelessness. The results show that by the time students completed winter interim assessments in the 2020-21 school year, they had experienced a learning lag of approximately 2.6 months in English language arts (ELA) and 2.5 months in math. It was further found that students who were economically disadvantaged, English learners, and Latinx experienced greater learning lag than students who were not in these groups. The authors position these findings in the context of other recent studies that have estimated COVID-19 impacts on student learning, discuss caveats of using interim assessments (including those administered remotely), and highlight the importance of examining differences in the effects for students in different groups. These findings can be useful for guiding decision-making and resource allocation at the state and local levels.

The research featured in this paper is part of the CORE-PACE Research Partnership, which is focused on producing research that informs continuous improvement in the CORE districts as well as policy and practice in California and beyond. Since spring 2020, the COVID-19 pandemic has been abruptly interrupting regular instruction in almost all schools in the U.S. One year later, policymakers, district administrators, and educators are still balancing the benefits and risks of returning K-12 students to fully in-person school. Many are concerned about the pandemic's disruption to students' academic progress. In California, educators have been focused equally on students' mental and emotional health, social relationships, and learning environment, given that many students have been learning remotely since the onset of the pandemic. Anticipating the need for schools and districts to understand their students' social-emotional well-being, the CORE Districts in California developed survey instruments to collect such information in the 2020-21 school year. The surveys were administered remotely, as all participating districts offered distance-only instruction. This report draws on data from the "Student well-being and learning conditions diagnostic survey" taken by 32,000 students in Grades 4-12 from three districts at the beginning of the 2020-21 school year. The results from 15,000 students who also responded to the "Comprehensive interim survey" a few months later are also highlighted. This report brings data to bear on the areas of student well-being that are most relevant at each developmental stage, so that all education stakeholders are able to focus their efforts on meeting the most pressing needs of each group of students.

This brief is one in a series aimed at providing K-12 education decision makers and advocates with an evidence base to ground discussions about how to best serve students during and following the novel coronavirus pandemic. It addresses one central question: How can schools and districts monitor students' social and emotional well-being across the year? In order to answer this question, the brief breaks down the issue into two points: (1) Disruptions to students' mental and emotional health, social systems of support, and learning environments require a new focus on social and emotional well-being; and (2) Although the need to assess students' social and emotional well-being in a virtual environment is new, we can still draw from assessments that were developed and validated prior to the pandemic. Based on these points, the brief provides four strategies to consider and two strategies to avoid.

Education researchers use surveys widely. Yet, critics question respondents' ability to provide high-quality responses. As schools increasingly use student surveys to drive policymaking, respondents' (lack of) motivation to provide quality responses may threaten the wisdom of using surveys for data-based decision-making. To better understand student satisficing (suboptimal responding on surveys) and its impact on data quality, we examined the pervasiveness and impact of this practice on a large-scale social-emotional learning survey administered to 409,721 students in grades 2-12. Findings indicated that despite the prevalence of satisficing in our sample, its impact on data quality appeared more modest than anticipated. We conclude by providing an accessible approach for defining and calculating satisficing for researchers, practitioners, and policymakers working with large-scale datasets.

Teachers play a critical role in establishing classroom and school environments that contribute to students' social and emotional development. This paper explores whether we can estimate a classroom-level measure of student growth in SEL by applying value-added models to students' [social-emotional learning] SEL. We analyze data from the 2016 and 2017 administrations of student self-report surveys, which contain responses from roughly 40,000 students in Grade 5 within five of California's CORE Districts. We estimate separate value-added models for each of the four SEL constructs assessed--growth mindset, self-efficacy, self-management, and social awareness--and for math and [English language arts] ELA academic growth. We find across-classroom-within-school variance of students' SEL outcomes, even after accounting for school-level variance. The magnitude of classroom-level impacts on students' growth in SEL appears similar to impacts on students' growth in ELA and math, although the growth models of SEL do not perform as well as growth models of academic outcomes. Results suggest that across-classroom-within-school impacts may be larger in magnitude than across-school impacts on students' SEL growth. Finally, we show that there are generally low correlations between classroom-level growth in SEL and classroom-level growth in ELA or math; however, growth mindset stands apart from the other three SEL constructs in that there is a moderately strong relationship. By assessing whether we can develop a sound approach for measuring classroom-level impacts on students' SEL, we aim to contribute to the growing body of knowledge about appropriate and innovative uses of data on students' non-cognitive and social-emotional learning. [For the policy brief, "Can We Measure Classroom Supports for Social-Emotional Learning?," see ED600449.]

The increasing importance of a college degree has led policymakers and educators to focus on strengthening the path to college beyond college entry. In this research brief the authors review four factors key to educational attainment: (1) aspirations and beliefs, (2) academic preparation, (3) knowledge and information, and (4) fortitude and resilience. This brief also highlights the ways in which these are reflected in practice and discusses steps that schools might consider given their role in developing students' skills and dispositions for successfully navigating the pathways to and through college.

California's shift towards continuous improvement in education makes understanding how districts and schools can learn to improve a more pressing question than ever. The CORE Improvement Community (CIC), a network of California school districts engaged in learning about improvement together, is an important testing ground to learn about what this work entails. This report continues drawing lessons from the CIC's second year as its districts work together towards a common aim: to improve the mathematics achievement of African American and Latinx students in Grades 4-8. The CIC applies a specific continuous improvement approach, called improvement science, to support teams in reaching the aim. Improvement science, unlike many approaches to reform, is not a specific "program" designed to fix educators' performance in a particular aspect of their work (e.g., mathematics instruction). Instead, it is an approach and tools through which educators can better understand the causes of lagging performance, select ideas that they believe will lead to improvement, test them, and collect and analyze data from those tests to systematically see if their ideas indeed yield better outcomes. In 2016-17, the work of the CIC consisted of building district teams that conducted systems analysis to understand the achievement gap in their respective contexts. In 2017-18, the CIC launched local improvement teams (LITs) at schools, which used tools and protocols of improvement science to identify strategies that could impact the problem of practice, test those ideas, and gather data about the impacts of those change ideas. The first section of the report briefly explains the policy context in California and the history of the CORE districts' collaboration. The second part of the report details four major lessons learned from the CIC this year: (1) The simultaneous goals of improving math achievement while building capacity for continuous improvement offer both benefits and challenges for the CORE Districts; (2) Districts have a pivotal role to play in supporting and sustaining continuous improvement efforts focused on classroom instruction; (3) Context matters. Preexisting structures and processes, time for educator collaboration, and supportive leadership all influence continuous improvement efforts; and (4) District and school leaders are excited about the potential of continuous improvement to spur deep and lasting improvement.

Under emerging policy structures in California, the responsibility for school improvement is increasingly placed upon local school districts, with County Offices of Education (COEs) playing a critical support role. In this system, districts are responsible for school improvement, with counties in charge of ensuring quality across districts and providing feedback and support where necessary. Underlying this major policy shift is the idea that local leaders are in the best position to drive real educational improvement and ensure quality across multiple schools and contexts. As California supports districts and counties statewide to embark on this improvement journey, there are important lessons to be learned from the CORE districts, six of which developed an innovative accountability system under a waiver from No Child Left Behind (NCLB). The CORE districts are early adopters of the new accountability paradigm: local leaders using multiple measures of school performance and working together to figure out collectively what works best for struggling schools. Now deepening their work together as a Networked Improvement Community (NIC), the CORE districts have simultaneously expanded access to their multiple-measures data and learning system by inviting other California districts to join their "Data Collaborative." Districts who join contribute their own student data to the CORE measurement system and are then able to benchmark student performance against other schools in the state. Data Collaborative districts engage in structured network learning activities that enhance their capacity to use data to drive system improvements. Currently, over 50 California school districts representing nearly a million students have joined the Data Collaborative. This report first provides a framework for how data use for improvement is different from data use for accountability and how data should be used by actors at different levels of the system. Next, it discusses the policy context in California and the current state of data use based on interviews conducted in the summer of 2017 with 41 leaders from state education agencies, COEs, school districts, technical assistance providers, education advocacy organizations, and education associations. Finally, the report shares lessons from the CORE Data Collaborative about how data can be used for improvement within networked structures, including what data is needed and how learning and collaboration can be facilitated. [For "Using Data for Improvement: Learning from the CORE Data Collaborative. Facts-at-a-Glance," see ED594745.]

Measures of school-level growth in student outcomes are common tools used to assess the impacts of schools. The vast majority of these measures are based on standardized tests, even though emerging evidence demonstrates the importance of social-emotional skills (SEL). This paper uses the first large-scale panel surveys of students on SEL to produce and evaluate school-level value-added measures by grade for growth mindset, self-efficacy, self-management, and social awareness. We find substantive differences across schools in SEL growth, of magnitudes similar to those for academic achievement. This result suggests that schools might contribute to students' SEL. However, we also find that the models are not as well specified for SEL as they are for achievement gains, raising the possibility that the estimated school effects include school-level measurement error and potential omitted variables bias. In addition, the across-school variance in the average level of the SEL measures is proportionally much smaller than for academic measures, which would not be expected if substantial impacts of schools on SEL outcomes persisted over time. These findings recommend caution in interpreting measures as the causal impacts of schools on SEL, though they also do not rule out important school effects.

California's Local Control Funding Formula (LCFF) requires districts to report multiple measures of student performance that reflect success in the goal of preparing students for college, career, and citizenship. As they engage in the Local Control Accountability Plan (LCAP) process, they are expected to use state and local indicator data from California's School Dashboard to monitor student progress. When Dashboard indicators identify student subgroups as low performing or low growth, districts are encouraged to engage in a process of continuous improvement to develop strategies and then monitor their effectiveness. At this early stage of implementation, education leaders have an opportunity to learn from early adopters who are already using continuous improvement principles. In this case study, we share how Fresno Unified School District (FUSD) developed and utilized its data dashboard and the principles of Improvement Science to increase college access for their students, in partnership with the University of California, Merced.

The research featured in this paper is part of the CORE-PACE Research Partnership, through which Policy Analysis for California Education (PACE) has partnered with the CORE districts to conduct research designed to support them in continuous improvement while simultaneously helping to improve policy and practice in California and nationwide. Through this partnership, PACE coordinates and executes research with partners from all of California's top universities, including Stanford University, the University of Southern California, and the University of California, Davis, in addition to engaging researchers from universities and research organizations nationwide. This policy brief and the accompanying report show that there is good reason to pursue the measurement of social-emotional learning (SEL) and school culture/climate (CC) as a way to better understand student and school performance. Using data from California's CORE districts--districts serving nearly one million students who have embraced systematic measurement of SEL and CC--this study shows that SEL and CC measures demonstrate reliability and validity, distinguish between schools, are related to other academic and non-academic measures, and also illuminate dimensions of student achievement that go beyond traditional indicators. This report also shows how the SEL and CC measures can be used to identify areas of improvement within schools, such as identifying subgroup gaps or differences in reports between various respondent groups. [For the accompanying report, "Using Surveys of Students' Social-Emotional Learning and School Climate for Accountability and Continuous Improvement," see ED574847.]

The research featured in this paper is part of the CORE-PACE Research Partnership, through which Policy Analysis for California Education (PACE) has partnered with the CORE districts to conduct research designed to support them in continuous improvement while simultaneously helping to improve policy and practice in California and nationwide. Through this partnership, PACE coordinates and executes research with partners from all of California's top universities, including Stanford University, the University of Southern California, and the University of California, Davis, in addition to engaging researchers from universities and research organizations nationwide. This report and accompanying policy brief show that there is good reason to pursue the measurement of social-emotional learning (SEL) and school culture/climate (CC) as a way to better understand student and school performance. Using data from California's CORE districts--districts serving nearly one million students who have embraced systematic measurement of SEL and CC--this study shows that SEL and CC measures demonstrate reliability and validity, distinguish between schools, are related to other academic and non-academic measures, and also illuminate dimensions of student achievement that go beyond traditional indicators. This report also shows how the SEL and CC measures can be used to identify areas of improvement within schools, such as identifying subgroup gaps or differences in reports between various respondent groups. A figure and tables are appended. [For the accompanying policy brief, "Using Surveys of Students' Social-Emotional Learning and School Climate for Accountability and Continuous Improvement. Policy Brief 17-1," see ED574846.]

California and the nation are at the crossroads of a major shift in school accountability policy. At the state level, California's Local Control and Accountability Plan (LCAP) encourages the use of multiple measures of school performance used locally to support continuous improvement and strategic resource allocation. Similarly, the federal Every Student Succeeds Act (ESSA) reinforces this local control, requiring more comprehensive assessment of school performance and a less prescriptive, local approach to school support. These changes represent a major cultural shift for California schools and districts. Ahead of the curve, six California districts, known as the California Office to Reform Education (CORE) waiver districts, have implemented an innovative measurement system and supports for school and district improvement under an NCLB waiver, and thus provide a unique opportunity to examine and learn from the enactment of a system supported by accountability policy in this new era. This report examines the early implementation and effects of the CORE reform and seeks to inform the ongoing efforts within CORE as well as the development and implementation of future accountability policy in other states and districts. Additional details about the research methods are appended. [For the accompanying policy brief, "Local Control in Action: Learning from the CORE Districts' Focus on Measurement, Capacity Building, and Shared Accountability. CORE-PACE Research Partnership. Policy Brief 16-4," see ED574788.]

California and the nation are at the crossroads of a major shift in school accountability policy. At the state level, California's Local Control and Accountability Plan (LCAP) encourages the use of multiple measures of school performance used locally to support continuous improvement and strategic resource allocation. Similarly, the federal Every Student Succeeds Act (ESSA) reinforces this local control, requiring more comprehensive assessment of school performance and a less prescriptive, local approach to school support. These changes represent a major cultural shift for California schools and districts. As California supports districts statewide to embark on this improvement journey, there are important lessons to be learned from the California Office to Reform Education (CORE) districts, six of which developed an innovative accountability system under a waiver from No Child Left Behind (NCLB). The CORE districts are early adopters of the new accountability paradigm: local leaders using multiple measures of school performance and working together to figure out collectively what works best for struggling schools. This study examines how the CORE districts understood, implemented, and responded to the new accountability system implemented under the waiver. The research indicates that a shift to greater flexibility and locally-determined capacity building efforts brings its own set of challenges, but substantial benefits as well. The CORE districts present an opportunity to learn how to effectively utilize multiple measures of school quality, develop shared accountability, and build capacity for schools and districts to improve. In summary, the authors found that: (1) district and school administrators greatly appreciated the shift toward a more holistic approach to measurement and an emphasis on support over sanctions; (2) most waiver districts adapted CORE's accountability system to their local needs, revealing a tension between shared accountability and local variation; and (3) CORE's measurement system and district-level collaboration hold promise for improving local systems, while efforts to improve schools through collaboration and capacity building remain a work in progress. [For the accompanying report, "At the Forefront of the New Accountability Era: Early Implementation Findings from the CORE Waiver Districts," see ED574787.]

The Every Student Succeeds Act (ESSA) makes sweeping changes to the way school performance is measured. Using the innovative measurement system developed by the CORE Districts in California, the authors explore how schools can be identified for support and improvement using a multiple measures framework. They show that 1) Different academic indicators measure very different aspects of school performance, suggesting that states should be allowed and encouraged to make full use of multiple measures to identify schools in the way they see fit instead of reporting a summative rating; 2) The ESSA regulations effectively restrict the weighting of the non-academic "School Quality and Student Success" indicators to zero, which is not in the spirit of the expanded measurement; and 3) The majority of schools will be identified for targeted support under the current regulations, suggesting the need for a clarification in federal policy. [This memo represents work underway as part of the CORE-PACE Research Partnership. For a summary of this policy memo, see ED574852.]

In May 2016, the authors released a policy memo comparing the effect of reporting subgroups at an n-size of 20+ compared to 100+. In response to this original memo, the U.S. Department of Education released a rule notice proposing changes to ESSA regulation §200.17 allowing states "to establish a range of n-sizes, not to exceed 30." To support state-level policymaking under these new requirements, this update to the original policy memo will illustrate the trade-offs between subgroup sizes of 20+ and 30+. In this supplementary analysis, the authors show that: (1) The effect of moving from a subgroup size of 30+ to 20+ is not as stark as the effect of moving from a subgroup size of 100+ to 20+; (2) A substantially higher percentage of student data is reported at smaller subgroup sizes; (3) At a subgroup size of 20+, approximately 38 percent more schools report results for "all student subgroups" than at a subgroup size of 30+; and (4) The lowest performing racial/ethnic subgroup in the school is often excluded from schools reporting at the higher subgroup size. [This memo represents work underway as part of the CORE-PACE Research Partnership. For the original memo, see ED574823.]

With the passage of the Every Student Succeeds Act (ESSA) of 2015, California state policymakers are tasked with determining the subgroup threshold for school-level reporting. To inform this decision, this policy brief explores the implications of utilizing various subgroup sizes using data from the CORE Districts. The authors find that the 20+ subgroup size presents clear advantages in terms of the number of students represented, particularly in making historically underserved student populations visible. [This memo represents work underway as part of the CORE-PACE Research Partnership. For an update to this memo, see ED574821.]

With the passage of the Every Student Succeeds Act (ESSA) of 2015, California must integrate additional measures of student and school performance into the state-wide accountability system. To support the conversation as policymakers consider if/how to include chronic absenteeism data in the state's accountability system, PACE has conducted an analysis of the CORE Districts' student chronic absenteeism data. Chronic absence is feasible for inclusion in California's accountability measurement system using the state's approach for rating school achievement based on outcome and improvement, or alternatively through an approach that simply looks at performance in a given school year. [This memo represents work underway as part of the CORE-PACE Research Partnership.]

To evaluate the extent to which California's school districts are engaging in continuous improvement, Policy Analysis for California Education (PACE) interviewed 41 educational leaders across the state. Three main ideas surfaced: (1) State data is insufficient for continuous improvement; (2) Some school districts are developing local data systems, but not every district has the time, money, or expertise to do so; and (3) Capacity to use data for improvement remains challenging. This two-page infographic highlights how the CORE Data Collaborative meets all three of these concerns. [For the full report, see ED594744.]

The Every Student Succeeds Act (ESSA) requires states to develop an accountability system that includes multiple measures of student academic performance and at least one additional indicator of "School Quality or Student Success" (SQSS). To support policymaking at both the state and federal level, the authors of "Identity Crisis: Multiple Measures and the Identification of Schools under ESSA. Policy Memo 16-3" use the innovative measurement system developed by the CORE Districts in California to explore one important aspect of ESSA, the identification of schools for support and improvement using a multiple measures framework. They find that: (1) Different academic indicators measure very different aspects of school performance, and states should therefore be allowed and encouraged to make full use of multiple measures to identify schools instead of reporting a single summative rating; (2) The ESSA regulations effectively restrict the weighting of the non-academic "School Quality and Student Success" indicators to zero, which is not in the spirit of the law's attention to multiple measures of school performance; and (3) The majority of schools will be identified for Targeted support and Improvement (TSI) under the current regulations, which suggests that the rules for identification should be clarified. [This memo represents work underway as part of the CORE-PACE Research Partnership. For the full policy memo, see ED574851.]