Search

The COVID-19 virus brought on a public health emergency that massively disrupted school systems and learning nationwide. During the 2020-2021 school year, many local education agencies (LEAs) in Pennsylvania and other states adopted remote learning to help slow the virus's spread. However, remote learning came with challenges for students, families, and LEAs. These challenges included very practical ones like not having a reliable internet connection or the correct link to log into classes, as well as fatigue from so many remote meetings and feelings of social isolation. This brief examines remote learning in Pennsylvania and its role in shaping education outcomes and COVID-19 case rates in local communities during the 2020-2021 school year. Key findings for the 2020-2021 school year: (1) Vulnerable groups of students were more likely than students overall to attend LEAs where the predominant instructional modes included remote learning; (2) The same groups of students were also more likely to be in LEAs where larger shares of the student population had trouble accessing reliable internet; (3) More remote learning was associated with lower assessment performance in grades 5-8, especially in schools with higher levels of economic disadvantage. It was also associated with lower suspension rates in grades 6-12 and did not appear to harm or help graduation rates; and (4) Remote learning in high school was associated with lower subsequent COVID-19 rates within an LEA's boundary, suggesting it had an important public health benefit. [Contributions from Candy Miller, Rhonda Johnson, and Rosemary Riccardo.]

The COVID-19 pandemic raised concerns about the possibility of increased staff attrition from the public school sector, as it did for other occupations during the Great Resignation that followed the pandemic's onset. Recruiting and retaining education staff were pressing challenges even before the pandemic began. Since March 2020, education staff have needed to adapt to new job expectations and risk exposure to an unknown virus. These factors might have contributed to staff leaving public education jobs due to mental and physical health concerns and other issues. This research brief describes staff attrition during the pandemic from Pennsylvania local education agencies (LEAs). The brief describes annual rates of staff leaving the statewide public school sector from several jobs--including teachers, health and counseling staff, school administrators, and LEA administrators--and how changes in attrition during the pandemic varied with staff, school, and LEA characteristics. Key findings for teachers, health and counseling staff, and administrators include: (1) five percent of staff working in Pennsylvania local education agencies (LEAs) in fall 2020 left the statewide public school sector by fall 2021, an increase from 4 percent attrition in preceding years; (2) pre-pandemic attrition rates for staff of color were generally higher than for non-Hispanic White staff, and these differences grew during the pandemic except for school administrators, where they narrowed; and (3) gaps in attrition rates were also exacerbated during the pandemic for teachers and health and counseling staff early in their careers and in charter schools and for LEA administrators in large LEAs. [This brief includes contributions from Candy Miller, Rhonda Johnson, and Rosemary Riccardo.]

The disruptions to schooling created by the COVID-19 pandemic prompted nationwide concern about negative impacts on academic proficiency, particularly for vulnerable student populations. Pennsylvania's 2021 statewide assessments offered the first opportunity since the pandemic began to assess how students across the Commonwealth were faring, whether achievement gaps had widened, and how remote learning affected student performance. As in other states, the raw assessment data did not tell the full story of the pandemic's impacts on academic proficiency. Fewer Pennsylvania students participated in the 2021 assessments than in prior years, and test takers were not a representative sample of all students. In addition, test administration dates varied widely across the state in 2021, with some local education agencies (LEAs) postponing the usual spring assessments to the fall. Using a predictive model that accounts for the unique assessment issues in 2021 and the instructional modes that Pennsylvania LEAs offered, this research brief provides a fuller picture of the pandemic's impacts on academic proficiency in grades 5 through 8. Key findings for Pennsylvania students in grades 5 through 8 include: (1) students in vulnerable groups and in fully remote learning had lower assessment participation rates than other students in 2021; (2) the pandemic led to a substantial reduction in academic proficiency statewide in 2021, reducing academic achievement even more than the results for tested students indicated (3) the pandemic had consistently large negative academic effects across student groups, including for vulnerable groups that had low rates of proficiency before the pandemic; and (4) LEAs that were fully remote after winter break saw lower proficiency rates in 2021, compared with LEAs that offered fully in-person learning to similar students. [This brief includes contributions from Candy Miller, Rhonda Johnson, and Rosemary Riccardo.]

The COVID-19 pandemic put unprecedented pressure on Pennsylvania schools to find new ways of educating students amid a worldwide health crisis. Terms like remote learning and hybrid learning became part of everyday language as local education agencies (LEAs) across the Commonwealth implemented different approaches to safely deliver instruction and support students' needs. This research brief describes how Pennsylvania LEAs (school districts and charter schools for purposes of this research) responded to public health and other challenges during the 2020--2021 school year.

Proficiency rates on Pennsylvania's statewide assessments in 2022 remain below pre-pandemic levels, especially in math, but there is evidence that student learning has begun to recover. Proficiency rates for grades 5-8 in 2022 improved in both English language arts (ELA) and math relative to rates predicted for 2021 from a Mathematica research brief that accounted for unusual test conditions that year. The results are encouraging, but more work is needed to help students recover from the pandemic. [For a related report "How the COVID-19 Pandemic Affected Academic Proficiency Rates in Pennsylvania in 2021: Findings from a Predictive Model. Education Research Brief," see ED628488.]

Like many states throughout the nation, Pennsylvania is in the midst of major reforms to its teacher evaluation system. Under the new system, the state will base annual evaluations on several measures, including supervisor observations using the Framework for Teaching (FFT) and, for many teachers, their contributions to student achievement growth from a value-added model (VAM). In the past, there was concern that supervisor observations did not differentiate performance well or relate to true teacher performance. In this study, we investigate how well the new system has addressed these issues by analyzing the degree to which FFT scores differentiate performance, are internally consistent, and correlate with teachers' contributions to student achievement growth as measured by VAM scores. This report is based on data from a pilot of the new system covering 6,676 teachers from 269 districts in the state of Pennsylvania, including Pittsburgh public schools. The data include the 22 components of the FFT, each of which is designed to capture a separate teaching practice. We used these data to estimate four domain scores and one overall Professional Practice Rating (PPR) score. We also merged these scores with data on teachers' estimated contributions to student achievement growth. Based on these pilot data from the 2012--2013 school year, we estimate that, although less than 13 percent of teachers received the top rating ("distinguished") for the overall PPR score, almost 85 percent were rated in the second highest category ("proficient") (Figure S.1). Less than 0.1 percent were rated in the bottom category ("failing"). The remaining teachers (around 2.6 percent) were given "needs improvement" ratings. We found that FFT scores were internally consistent, meaning that the domains and the components within each domain appear to be measuring similar concepts. We also found that teachers with higher FFT scores tended to produce greater student achievement growth. The correlations of the FFT scores with VAM scores were all positive and generally statistically significant, ranging from 0.19 to 0.22 by domain. We compared the results based on the 2012--2013 data with results based on 2011-2012 data from a previous pilot phase. For the most part, the findings were similar. More than 90 percent of teachers were rated in the top two performance categories in both phases, although the fraction of ratings in the top two categories decreased somewhat in Pittsburgh (which contributed more teachers to the pilot than any other district). The levels of internal consistency were in the acceptable to good ranges in both phases, with the overall PPR score having higher consistency than any of the domain scores in both phases. The correlations between parts of the FFT and VAM scores were almost always positive but also below 0.30 in both phases. The lowest correlations in 2011-2012 were slightly improved in 2012-2013. In sum, although FFT scores are overwhelmingly concentrated in the top two performance categories, the positive correlations with VAM suggest that the FFT provides some meaningful differentiation and captures aspects of teacher skills related to student achievement growth.

Responding to federal and state prompting, school districts across the country are implementing new teacher evaluation systems that aim to increase the rigor of evaluation ratings, better differentiate effective teaching, and support personnel and staff development initiatives that promote teacher effectiveness and ultimately improve student achievement. Pittsburgh Public Schools (PPS) has been working for the last several years to develop richer and more-comprehensive measures of teacher effectiveness in support of a larger effort to promote effective teaching. In partnership with PPS, REL Mid-Atlantic collected data from Pittsburgh on three different types of teacher performance measures: professional practice measures derived from the Danielson Framework for Teaching; Tripod student survey measures; and value-added measures designed to assess each teacher's contribution to student achievement growth. The study found that each of the three types of measures has the potential to differentiate the performance levels of different teachers. Moreover, the three types of measures are positively but modestly correlated with each other, suggesting that they are valid and complementary measures of teacher effectiveness and that they can be combined to produce a measure that is more comprehensive than any single measure. School-level variation in the ratings on the professional practice measure, however, suggests that different principals may have different standards in assigning ratings, which in turn suggests that the measure might be improved by using more than one rater of professional practice for each teacher. Five appendices present: (1) Descriptions of the three teacher evaluation measures; (2) Detailed tables; (3) Principal components analyses; (4) Correlating the Research-based Inclusive System of Evaluation and 7Cs ratings with partly concurrent value-added measure estimates; and (5) Checking Research-based Inclusive System of Evaluation ratings within and between schools. [This report was prepared for the Institute of Education Sciences (IES) by Regional Educational Laboratory Mid-Atlantic administered by ICF International.]